Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
ariaotp b5b5a793e4 | 2 years ago | |
---|---|---|
scripts | 2 years ago | |
src | 2 years ago | |
README.md | 2 years ago | |
README_CN.md | 2 years ago | |
eval.py | 2 years ago | |
export.py | 2 years ago | |
mindspore_hub_conf.py | 2 years ago | |
requirements.txt | 2 years ago | |
train.py | 2 years ago |
Deep Embedding Model (DEM) proposes a new zero shot learning (ZSL) model, which maps the semantic space to the visual feature space. That's to say, DEM maps the low dimensional space to the high dimensional space, thus avoiding the hubness problem. And a multi-modal semantic feature fusion method is proposed, which is used for joint optimization in an end-to-end manner.
Paper: Li Zhang, Tao Xiang, Shaogang Gong."Learning a Deep Embedding Model for Zero-Shot Learning" Proceedings of the CVPR.2017.
DEM uses googlenet to extract features, and then uses multimodal fusion method to train in attribute space, wordvector space and fusion space.
Dataset used: AwA, CUB. Download data from here
- Note:Data will be processed in dataset.py
└─data
├─AwA_data
│ ├─attribute #attribute data
│ ├─wordvector #wordvector data
│ ├─test_googlenet_bn.mat
│ ├─test_labels.mat
│ ├─testclasses_id.mat
│ └─train_googlenet_bn.mat
└─CUB_data #The directory is similar to AwA_ data
After installing MindSpore via the official website, you can start training and evaluation as follows:
# Install necessary package
pip install -r requirements.txt
# Place dataset in '/data/DEM_data', rename and unzip
mv data.zip DEM_data.zip
mv ./DEM_data.zip /data
cd /data
unzip DEM_data.zip
#1p example
# Enter the script dir and start training
sh run_standalone_train_ascend.sh CUB att /data/DEM_data ../output
# Enter the script dir and start evaluating
sh run_standalone_eval_ascend.sh CUB att /data/DEM_data ../output/train.ckpt
#8p example
sh run_distributed_train_ascend.sh [hccl configuration,.json format] CUB att /data/DEM_data
sh run_standalone_eval_ascend.sh CUB att /data/DEM_data ../train_parallel/7/auto_parallel-120_11.ckpt
#Note: Word and fusion training in CUB dataset are not supported
├── cv
├── DEM
├── README.md // descriptions about DEM
├── README_CN.md // descriptions about DEM in Chinese
├── requirements.txt // package needed
├── scripts
│ ├──run_distributed_train_ascend.sh // train in ascend with 8p
│ ├──run_standalone_train_ascend.sh // train in ascend with 1p
│ └──run_standalone_eval_ascend.sh // evaluate in ascend
├── src
│ ├──dataset.py // load dataset
│ ├──demnet.py // DEM framework
│ ├──config.py // parameter configuration
│ ├──kNN.py // k-Nearest Neighbor
│ ├──kNN_cosine.py // k-Nearest Neighbor cosine
│ ├──accuracy.py // compute accuracy
│ ├──set_parser.py // basic parameters
│ └──utils.py // functions used
├── train.py // training script
├── eval.py // evaluation script
└── export.py // exportation script
# Major parameters in train.py and set_parser.py as follows:
--device_target:Device target, default is "Ascend"
--device_id:Device ID
--distribute:Whether train under distributed environment
--device_num:The number of device used
--dataset:Dataset used, choose from "AwA", "CUB"
--train_mode:Train_mode, choose from "att"(attribute), "word"(wordvector), "fusion"
--batch_size:Training batch size.
--interval_step:the interval of printing loss
--epoch_size:The number of training epoch
--data_path:Path where the dataset is saved
--save_ckpt:Path where the ckpt is saved
--file_format:Model transformation format
python train.py --data_path=/YourDataPath --save_ckpt=/YourCkptPath --dataset=[AwA|CUB] --train_mode=[att|word|fusion]
# or enter script dir, and run the script
sh run_standalone_train_ascend.sh [AwA|CUB] [att|word|fusion] [DATA_PATH] [SAVE_CKPT]
# 1p example:
sh run_standalone_train_ascend.sh CUB att /data/DEM_data ../train.ckpt
# 8p example:
sh run_distributed_train_ascend.sh [hccl configuration,.json format] CUB att /data/DEM_data ../train.ckpt
After training, the loss value will be achieved as follows:
============== Starting Training ==============
epoch: 1 step: 100, loss is 0.24551314
epoch: 2 step: 61, loss is 0.2861852
epoch: 3 step: 22, loss is 0.2151301
...
epoch: 16 step: 115, loss is 0.13285707
epoch: 17 step: 76, loss is 0.15123637
...
The model checkpoint will be saved in [SAVE_CKPT], which has been designated in the script.
Before running the command below, please check the checkpoint path used for evaluation.
python eval.py --data_path=/YourDataPath --save_ckpt=/YourCkptPath --dataset=[AwA|CUB] --train_mode=[att|word|fusion]
# or enter script dir, and run the script
sh run_standalone_eval_ascend.sh [AwA|CUB] [att|word|fusion] [DATA_PATH] [SAVE_CKPT]
# Example:
sh run_standalone_eval_ascend.sh CUB att /data/DEM_data ../output/train.ckpt
The accuracy of the test dataset is as follows:
============== Starting Evaluating ==============
accuracy _ CUB _ att = 0.58984
Parameters | DEM_AwA | DEM_CUB |
---|---|---|
Resource | Ascend 910; CPU 2.60GHz, 192cores; Memory 755G;OS CentOS8.2 | Ascend 910; CPU 2.60GHz, 192cores; Memory 755G;OS CentOS8.2 |
uploaded Date | 06/18/2021 (month/day/year) | 04/26/2021 (month/day/year) |
MindSpore Version | 1.2.0 | 1.2.0 |
Dataset | AwA | CUB |
Training Parameters | epoch = 100, batch_size = 64, lr=1e-5 / 1e-4 / 1e-4 | epoch = 100, batch_size = 100, lr=1e-5 |
Optimizer | Adam | Adam |
Loss Function | MSELoss | MSELoss |
outputs | probability | probability |
Training mode | attribute, wordvector, fusion | attribute |
Speed | 24.6ms/step, 7.3ms/step, 42.1ms/step | 51.3ms/step |
Total time | 951s / 286s / 1640s | 551s |
Checkpoint for Fine tuning | 3040k / 4005k / 7426k (.ckpt file) | 3660k (.ckpt file) |
Accuracy calculation method | kNN / kNN_cosine / kNN_cosine | kNN |
In train.py, we use "dataset.Generator(shuffle=True)" to shuffle dataset.
Please check the official homepage.
Deep Embedding Model for ZSL
Python Markdown Shell other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》