Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
ZubinGou 2817a164e1 | 2 years ago | |
---|---|---|
config | 2 years ago | |
data | 2 years ago | |
images | 2 years ago | |
log_output | 2 years ago | |
src | 2 years ago | |
tools | 2 years ago | |
.gitignore | 2 years ago | |
LICENSE | 2 years ago | |
README.md | 2 years ago | |
main_ce.py | 2 years ago | |
main_supcon.py | 2 years ago | |
requirements.txt | 2 years ago |
A PaddlePaddle implementation of SupContrast: Supervised Contrastive Learning
This repo covers an reference implementation for the following papers in PaddlePaddle 2.x, using CIFAR as an illustrative example:
(1) Supervised Contrastive Learning. Paper
(2) A Simple Framework for Contrastive Learning of Visual Representations. Paper
The loss function SupConLoss
in supcon.py
takes features
(L2 normalized) and labels
as input, and return the loss. If labels
is None
or not passed to the it, it degenerates to SimCLR.
Usage:
from supcon import SupConLoss
# define loss with a temperature `temp`
criterion = SupConLoss(temperature=temp)
# features: [bsz * n_views, f_dim]
# `n_views` is the number of crops from each image
# better be L2 normalized in f_dim dimension
features = ...
# labels: [bsz]
labels = ...
# SupContrast
loss = criterion(features, labels)
# or SimCLR
loss = criterion(features)
...
Results on CIFAR-10:
Arch | Setting | Loss | Paper Acc(%) | Our Acc(%) | abs. improv. | |
---|---|---|---|---|---|---|
SupCrossEntropy | ResNet50 | Supervised | Cross Entropy | 95.0 | 96.9 (-*) |
1.9 |
SupContrast | ResNet50 | Supervised | Contrastive | 96.0 | 97.3 (96.8*) |
1.3 |
SimCLR | ResNet50 | Unsupervised | Contrastive | 93.6 | - | - |
*for no
cutout
Accuracy
You might use CUDA_VISIBLE_DEVICES
to set proper number of GPUs.
We released 3 models, please download from cowtransfer with code 461254
:
./logs
|-- resnet50-ce-final/final # SupCrossEntropy (Acc: 96.9)
|-- resnet50-supcon-final/final # SupContrast Pretrained
|-- resnet50-linear-final/final # SupContrast Linear Fine-tuned (Acc: 97.3)
(0) Data Preparing
cd data
wget https://dataset.bj.bcebos.com/cifar/cifar-10-python.tar.gz
(1) Standard Cross-Entropy
python main_ce.py -y config/resnet50_ce.yml
config the continue_from
in config/resnet50_ce.yml
to specify the checkpoint path, then run:
python main_ce.py -y config/resnet50_ce.yml --test
you will get:
(2) Supervised Contrastive Learning
Pretraining stage:
python main_supcon.py -y config/resnet50_supcon.yml
Linear evaluation stage:
config the from_supcon
in config/resnet50_linear.yml
to specify the checkpoint path, then run:
python main_ce.py -y config/resnet50_linear.yml
config the continue_from
in config/resnet50_linear.yml
to specify the checkpoint path, then run:
python main_ce.py -y config/resnet50_linear.yml --test
you will get:
config/
for configuration details@Article{khosla2020supervised,
title = {Supervised Contrastive Learning},
author = {Prannay Khosla and Piotr Teterwak and Chen Wang and Aaron Sarna and Yonglong Tian and Phillip Isola and Aaron Maschinot and Ce Liu and Dilip Krishnan},
journal = {arXiv preprint arXiv:2004.11362},
year = {2020},
}
A PaddlePaddle implementation of SupContrast: Supervised Contrastive Learning
Python other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》