Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
You can not select more than 25 topics
Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
yangzhouxin
51dd4a165f
|
3 years ago | |
|---|---|---|
| .. | ||
| META-INF | 3 years ago | |
| __pycache__ | 3 years ago | |
| data | 3 years ago | |
| eval_results | 3 years ago | |
| log | 3 years ago | |
| misc | 3 years ago | |
| models | 3 years ago | |
| scripts | 3 years ago | |
| vis | 3 years ago | |
| .DS_Store | 3 years ago | |
| ADVANCED.md | 3 years ago | |
| HSPLAN.py | 3 years ago | |
| ImageCaptionModel.py | 3 years ago | |
| LICENSE | 3 years ago | |
| data_test.py | 3 years ago | |
| dataloader.py | 3 years ago | |
| dataloaderraw.py | 3 years ago | |
| eval.py | 3 years ago | |
| eval_ensemble.py | 3 years ago | |
| eval_utils.py | 3 years ago | |
| opts.py | 3 years ago | |
| readme.md | 3 years ago | |
| test-best.sh | 3 years ago | |
| test-last.sh | 3 years ago | |
| test.py | 3 years ago | |
| test_HSPLAN.py | 3 years ago | |
| train-wo-refining.sh | 3 years ago | |
| train.py | 3 years ago | |
| train.sh | 3 years ago | |
readme.md
HSPLAN
基于transformer的图文识别模型(Image Caption Model)
项目简介
1. 功能
该模型实现了从图像特征中生成文本描述的功能，通过从原始图像中提取的图像特征，可以生成根据图像特征描述该图像的语句。
2. 性能
代码基于pytorch框架，在生成语句阶段，可以输出语法正确的连贯语句。
3. 评估指标
CIDEr-D(Consensus-based Image Description Evaluation), BLEU(Bilingual Evaluation Understudy), METEOR(Metric for Evaluation of Translation with Explicit ORdering), ROUGE-L(Recall-Oriented Understudy for Gisting Evaluation), SPICE(Semantic Propositional Image Caption Evaluation)
4. 使用数据集
使用MSCOCO数据集提取的用bottom up attention的预训练特征(https://github.com/peteanderson80/bottom-up-attention)
运行与环境依赖
代码运行的环境与依赖：
| 类别 | 名称 | 版本 |
|---|---|---|
| 操作系统 | Ubuntu | 16.04 |
| 编程语言 | Python | 3.6 |
| 底层驱动 | CUDA | 9.0 |
| 深度学习框架 | Pytorch | 1.0 |
输入与输出
代码的输入与输出。如下所示：
| 名称 | 说明 |
|---|---|
| 输入 | fc_feats[图像中提取的整体特征],att_feats[图像中提取的局部特征],att_masks[图像特征的mask] |
| 输出 | 对应于图像特征的图像caption以及运行的时间 |
运行方式
在terminal下运行以下命令。
cd project_dir
python HSPLAN.py