Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
Akshita Gupta 6bf2a53b27 | 3 years ago | |
---|---|---|
classifiers | 3 years ago | |
datasets | 3 years ago | |
networks | 3 years ago | |
readme_images | 3 years ago | |
scripts | 3 years ago | |
.DS_Store | 3 years ago | |
.gitignore | 3 years ago | |
LICENSE | 3 years ago | |
README.md | 3 years ago | |
config_actions.py | 3 years ago | |
config_images.py | 3 years ago | |
environment.yml | 3 years ago | |
train_actions.py | 3 years ago | |
train_images.py | 3 years ago |
(* denotes equal contribution)
Paper: https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123670477.pdf
Video Presentation: Short summary , Overview
Finetuned features: https://drive.google.com/drive/folders/13-eyljOmGwVRUzfMZIf_19HmCj1yShf1?usp=sharing
Webpage: https://akshitac8.github.io/tfvaegan/
Zero-shot learning strives to classify unseen categories for which no data is available during training. In the generalized variant, the
test samples can further belong to seen or unseen categories. The stateof-the-art relies on Generative Adversarial Networks that synthesize unseen class features by leveraging class-specific semantic embeddings. During training, they generate semantically consistent features, but discard this constraint during feature synthesis and classification. We propose to enforce semantic consistency at all stages of (generalized) zero-shot learning: training, feature synthesis and classification. We first introduce a feedback loop, from a semantic embedding decoder, that iteratively refines the generated features during both the training and feature synthesis stages. The synthesized features together with their corresponding latent embeddings from the decoder are then transformed into discriminative features and utilized during classification to reduce ambiguities among categories. Experiments on (generalized) zero-shot object and action classification reveal the benefit of semantic consistency and iterative feedback, outperforming existing methods on six zero-shot learning benchmarks
Overall Framework for TF-Vaegan
A feedback module, which utilizes the auxiliary decoder during both training and feature synthesis stages for improving semantic quality of synthesized feature. |
A discriminative feature transformation that utilizes the auxiliary decoder during the classification stage for enhancing zero-shot classification. |
The model is built in PyTorch 0.3.1 and tested on Ubuntu 16.04 environment (Python3.6, CUDA9.0, cuDNN7.5).
For installing, follow these intructions
conda create -n tfvaegan python=3.6
conda activate tfvaegan
pip install https://download.pytorch.org/whl/cu90/torch-0.3.1-cp36-cp36m-linux_x86_64.whl
pip install torchvision==0.2.0 scikit-learn==0.22.1 scipy==1.4.1 h5py==2.10 numpy==1.18.1
Download CUB, AWA, FLO and SUN features from the drive link shared below.
link: https://drive.google.com/drive/folders/16Xk1eFSWjQTtuQivTogMmvL3P6F_084u?usp=sharing
Download UCF101 and HMDB51 features from the drive link shared below.
link: https://drive.google.com/drive/folders/1pNlnL3LFSkXkJNkTHNYrQ3-Ie4vvewBy?usp=sharing
Extract them in the datasets
folder.
import scipy.io as io
io.savemat('temp',feat)
CUB : python scripts/run_cub_tfvaegan.py
AWA : python scripts/run_awa_tfvaegan.py
FLO : python scripts/run_flo_tfvaegan.py
SUN : python scripts/run_sun_tfvaegan.py
HMDB51 : python scripts/run_hmdb51_tfvaegan.py
UCF101 : python scripts/run_ucf101_tfvaegan.py
If you find this useful, please cite our work as follows:
@inproceedings{narayan2020latent,
title={Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification},
author={Narayan, Sanath and Gupta, Akshita and Khan, Fahad Shahbaz and Snoek, Cees GM and Shao, Ling},
booktitle={ECCV},
year={2020}
}
No Description
Python
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》