使用100G中文高质量语料,128张V100,训练的中文Megatron模型,参数量2.6B
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
zhanghangit bf716fb7c0 add nvidia megatron code 1 month ago
..
bpe_3w_new add nvidia megatron code 1 month ago
data add nvidia megatron code 1 month ago
examples add nvidia megatron code 1 month ago
images add nvidia megatron code 1 month ago
megatron add nvidia megatron code 1 month ago
optimizer_ add nvidia megatron code 1 month ago
tasks add nvidia megatron code 1 month ago
tools add nvidia megatron code 1 month ago
.gitignore add nvidia megatron code 1 month ago
eval_gpt2.py add nvidia megatron code 1 month ago
get_ib_throughput.sh add nvidia megatron code 1 month ago
ib_speed_stat.sh add nvidia megatron code 1 month ago
preprocess_each_dataset_dev.sh add nvidia megatron code 1 month ago
pretrain_bert.py add nvidia megatron code 1 month ago
pretrain_gpt2.py add nvidia megatron code 1 month ago
pretrain_ict.py add nvidia megatron code 1 month ago
requirements.txt add nvidia megatron code 1 month ago
setup.py add nvidia megatron code 1 month ago