Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
leeyy2020 eec0299bd4 | 2 years ago | |
---|---|---|
.. | ||
label_normalized | 2 years ago | |
README.md | 2 years ago | |
data.py | 2 years ago | |
evaluate.py | 2 years ago | |
model.py | 2 years ago | |
pet.py | 2 years ago | |
predict.py | 2 years ago |
PET (Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference) 提出将输入示例转换为完形填空式短语,以帮助语言模型理解给定的任务
|—— pet.py # PET 策略的训练、评估主脚本
|—— dataset.py # PET 策略针对 FewCLUE 9 个数据集的任务转换逻辑,以及明文 -> 训练数据的转换
|—— model.py # PET 的网络结构
|—— evaluate.py # 针对 FewCLUE 9 个数据集的评估函数
|—— predict.py # 针对 FewCLUE 9 个数据集进行预测
PaddleNLP 内置了 FewCLUE 数据集,可以直接用来进行 PET 策略训练、评估、预测,并生成 FewCLUE 榜单的提交结果,参与 FewCLUE 竞赛。
基于 FewCLUE 数据集进行实验只需要 1 行代码,这部分代码在 pet.py
脚本中
from paddlenlp.datasets import load_dataset
# 通过指定 "fewclue" 和数据集名字 name="tnews" 即可一键加载 FewCLUE 中的 tnews 数据集
train_ds, dev_ds, public_test_ds = load_dataset("fewclue", name="tnews", splits=("train_0", "dev_0", "test_public"))
通过如下命令,指定 GPU 0 卡, 在 FewCLUE 的 tnews
数据集上进行训练&评估
python -u -m paddle.distributed.launch --gpus "0" \
pet.py \
--task_name "tnews" \
--device gpu \
--pattern_id 0 \
--save_dir ./tnews \
--index 0 \
--batch_size 16 \
--learning_rate 1E-4 \
--epochs 10 \
--max_seq_length 512 \
--language_model "ernie-1.0" \
--rdrop_coef 0 \
参数含义说明
task_name
: FewCLUE 中的数据集名字device
: 使用 cpu/gpu 进行训练pattern_id
完形填空的模式save_dir
: 模型存储路径max_seq_length
: 文本的最大截断长度rdrop_coef
: R-Drop 策略 Loss 的权重系数,默认为 0, 若为 0 则未使用 R-Drop 策略模型每训练 1 个 epoch, 会在验证集上进行评估
通过如下命令,指定 GPU 0 卡, 在 FewCLUE
的 tnews
数据集上进行预测
python -u -m paddle.distributed.launch --gpus "0" predict.py \
--task_name "tnews" \
--device gpu \
--init_from_ckpt "./tnews/model_120/model_state.pdparams" \
--output_dir "./tnews/output" \
--batch_size 32 \
--max_seq_length 512
[1] Schick, Timo, and Hinrich Schütze. “Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference.” ArXiv:2001.07676 [Cs], January 25, 2021. http://arxiv.org/abs/2001.07676.
黑客松task_55,在PaddleNLP的Roberta中,新增 MultipleChoice,MaskedLM 和 CausalLM三个类,7个模型权重. ,新增BPETokenizer
Python C++ Cuda Text Shell other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》