Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
liuxinchen3 22a8a748d2 | 2 years ago | |
---|---|---|
.. | ||
images | 2 years ago | |
videos | 2 years ago | |
README.md | 2 years ago | |
__init__.py | 2 years ago | |
build.py | 2 years ago | |
common.py | 2 years ago |
A dataset can be used by wrapping it into a torch Dataset. This document explains how to setup the builtin datasets so they can be used by X-modaler.
X-modaler has builtin support for a few datasets (e.g., MSCOCO or MSVD). The corresponding dataset wrappers are provided in ./xmodaler/datasets
:
xmodaler/datasets/
images/
mscoco.py
videos/
msvd.py
You can specify which dataset wrapper to use by DATASETS.TRAIN
, DATASETS.VAL
and DATASETS.TEST
in the config file.
First, download the dataset files, pre-trained models and coco_caption.
xmodaler
coco_caption
open_source_dataset/
mscoco_dataset
msvd_dataset
msrvtt_dataset
ConceptualCaptions
VQA
VCR
flickr30k
pretrain/
BERT
TDEN
Uniter
mscoco_dataset/
mscoco_caption_anno_train.pkl
mscoco_caption_anno_val.pkl
mscoco_caption_anno_test.pkl
vocabulary.txt
captions_val5k.json
captions_test5k.json
# image files that are mentioned in the corresponding json
features/
up_down/
*.npz
msvd_dataset/
msvd_caption_anno_train.pkl
msvd_caption_anno_val.pkl
msvd_caption_anno_test.pkl
vocabulary.txt
captions_val.json
captions_test.json
# videos files that are mentioned in the corresponding json
features/
resnet152/
*.npy
msrvtt_dataset/
msrvtt_caption_anno_train.pkl
msrvtt_caption_anno_val.pkl
msrvtt_caption_anno_test.pkl
vocabulary.txt
captions_val.json
captions_test.json
# videos files that are mentioned in the corresponding json
msrvtt_torch/
feature/
resnet152/
*.npy
When the dataset wrapper and data files are ready, you need to specify the corresponding paths to these data files in the config file. For example,
DATALOADER:
FEATS_FOLDER: '../open_source_dataset/mscoco_dataset/features/up_down' # feature folder
ANNO_FOLDER: '../open_source_dataset/mscoco_dataset' # annotation folders
INFERENCE:
VOCAB: '../open_source_dataset/mscoco_dataset/vocabulary.txt'
VAL_ANNFILE: '../open_source_dataset/mscoco_dataset/captions_val5k.json'
TEST_ANNFILE: '../open_source_dataset/mscoco_dataset/captions_test5k.json'
业界首个模块化、标准化的跨模态视觉分析代码库。支持各种多模态任务,轻松复现视觉语言领域主流技术,促进学术界在视觉语言领域的发展。
Python Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》