Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
i-robot b49e04359d | 4 days ago | |
---|---|---|
.. | ||
bert | 4 months ago | |
bloom | 4 months ago | |
clip | 4 months ago | |
codegeex2 | 3 months ago | |
codellama | 5 days ago | |
convert_config | 2 months ago | |
general | 1 month ago | |
glm | 4 months ago | |
glm2 | 5 days ago | |
glm3 | 5 days ago | |
gpt2 | 3 weeks ago | |
llama | 1 month ago | |
llama2 | 4 days ago | |
mae | 4 months ago | |
pangualpha | 4 months ago | |
qa | 4 months ago | |
sam | 2 months ago | |
swin | 4 months ago | |
t5 | 4 months ago | |
tokcls | 4 months ago | |
txtcls | 4 months ago | |
vit | 2 months ago | |
README.md | 1 week ago |
configs统一在run_xxx.yaml中,排序按照修改频率的顺序和一般的模型训练流程顺序(数据集->模型->训练、评估、推理),具体顺序如下
load_checkpoint=path/to/dir/
,其中dir路径下包含{BASE_MODEL}.ckpt
、{LORA_MODEL}.ckpt
。需要满足实际运行的卡数 device_num = data_parallel × model_parallel × pipeline_stage。自动并行下无此约束,但要保证stage内的卡数 stage_device_num是2的幂
type: 模型参数配置类
checkpoint_name_or_path: 评估时不指定权重,模型默认加载的权重名
# 以下配置针对大规模语言模型推理
top_k: 从概率最大的top_k个tokens中采样
top_p: 从概率最大且概率累计不超过top_p的tokens中采样
do_sample: 使能top_k或top_p采样,为False时top_k和top_p均重置为1
use_past: 使能增量推理,为True时为增量推理,否则为自回归推理,当前开启后会使用Paged Attention进行计算,使用时请参考模型支持列表
max_decode_length: 文本生成最大长度(输入长度统计在内)
max_length: 文本生成最大长度(输入长度统计在内),效果等同于max_decode_length,同时存在时以max_length为准
max_new_tokens: 文本新生成的最大长度(输入长度不统计在内),与max_length同时设置时,以max_new_tokens为准
min_length: 文本生成最小长度(输入长度统计在内)
min_new_tokens: 文本新生成最小长度(输入长度不统计在内),与min_length同时设置时,以min_new_tokens为准
repetition_penalty: 重复文本惩罚系数,该值不小于1,等于1时不惩罚
block_size: 使用Paged Attention推理时需设置,每块block的大小
num_blocks: 使用Paged Attention推理时需设置,blocks的总数。当前配置需要保证batch_sizeseq_length<=block_sizenum_blocks,否则运行过程中会提示PA的内存池不足
No Description
Jupyter Notebook Python Markdown Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》