You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
zhanghangit bf716fb7c0 add nvidia megatron code 3 years ago
..
bpe_3w_new add nvidia megatron code 3 years ago
data add nvidia megatron code 3 years ago
examples add nvidia megatron code 3 years ago
images add nvidia megatron code 3 years ago
megatron add nvidia megatron code 3 years ago
optimizer_ add nvidia megatron code 3 years ago
tasks add nvidia megatron code 3 years ago
tools add nvidia megatron code 3 years ago
.gitignore add nvidia megatron code 3 years ago
eval_gpt2.py add nvidia megatron code 3 years ago
get_ib_throughput.sh add nvidia megatron code 3 years ago
ib_speed_stat.sh add nvidia megatron code 3 years ago
preprocess_each_dataset_dev.sh add nvidia megatron code 3 years ago
pretrain_bert.py add nvidia megatron code 3 years ago
pretrain_gpt2.py add nvidia megatron code 3 years ago
pretrain_ict.py add nvidia megatron code 3 years ago
requirements.txt add nvidia megatron code 3 years ago
setup.py add nvidia megatron code 3 years ago

使用100G中文高质量语料,128张V100,训练的中文Megatron模型,参数量2.6B

Python C++ Shell Cuda TeX other

Contributors (2)