Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
whcao aac7f578e0 | 2 days ago | |
---|---|---|
.github | 8 months ago | |
docs | 5 days ago | |
examples | 1 month ago | |
requirements | 2 weeks ago | |
xtuner | 3 days ago | |
.gitignore | 7 months ago | |
.owners.yml | 8 months ago | |
.pre-commit-config-zh-cn.yaml | 9 months ago | |
.pre-commit-config.yaml | 1 month ago | |
LICENSE | 8 months ago | |
MANIFEST.in | 8 months ago | |
README.md | 2 days ago | |
README_zh-CN.md | 2 days ago | |
requirements.txt | 4 months ago | |
setup.cfg | 8 months ago | |
setup.py | 3 months ago |
system
context. More information can be found on Docs!XTuner is an efficient, flexible and full-featured toolkit for fine-tuning large models.
Efficient
Flexible
Full-featured
Models | SFT Datasets | Data Pipelines | Algorithms |
It is recommended to build a Python-3.10 virtual environment using conda
conda create --name xtuner-env python=3.10 -y
conda activate xtuner-env
Install XTuner via pip
pip install -U xtuner
or with DeepSpeed integration
pip install -U 'xtuner[deepspeed]'
Install XTuner from source
git clone https://github.com/InternLM/xtuner.git
cd xtuner
pip install -e '.[all]'
XTuner supports the efficient fine-tune (e.g., QLoRA) for LLMs. Dataset prepare guides can be found on dataset_prepare.md.
Step 0, prepare the config. XTuner provides many ready-to-use configs and we can view all configs by
xtuner list-cfg
Or, if the provided configs cannot meet the requirements, please copy the provided config to the specified directory and make specific modifications by
xtuner copy-cfg ${CONFIG_NAME} ${SAVE_PATH}
vi ${SAVE_PATH}/${CONFIG_NAME}_copy.py
Step 1, start fine-tuning.
xtuner train ${CONFIG_NAME_OR_PATH}
For example, we can start the QLoRA fine-tuning of InternLM2-Chat-7B with oasst1 dataset by
# On a single GPU
xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
# On multiple GPUs
(DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
(SLURM) srun ${SRUN_ARGS} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2
--deepspeed
means using DeepSpeed 🚀 to optimize the training. XTuner comes with several integrated strategies including ZeRO-1, ZeRO-2, and ZeRO-3. If you wish to disable this feature, simply remove this argument.
For more examples, please see finetune.md.
Step 2, convert the saved PTH model (if using DeepSpeed, it will be a directory) to Hugging Face model, by
xtuner convert pth_to_hf ${CONFIG_NAME_OR_PATH} ${PTH} ${SAVE_PATH}
XTuner provides tools to chat with pretrained / fine-tuned LLMs.
xtuner chat ${NAME_OR_PATH_TO_LLM} --adapter {NAME_OR_PATH_TO_ADAPTER} [optional arguments]
For example, we can start the chat with
InternLM2-Chat-7B with adapter trained from oasst1 dataset:
xtuner chat internlm/internlm2-chat-7b --adapter xtuner/internlm2-chat-7b-qlora-oasst1 --prompt-template internlm2_chat
LLaVA-InternLM2-7B:
xtuner chat internlm/internlm2-chat-7b --visual-encoder openai/clip-vit-large-patch14-336 --llava xtuner/llava-internlm2-7b --prompt-template internlm2_chat --image $IMAGE_PATH
For more examples, please see chat.md.
Step 0, merge the Hugging Face adapter to pretrained LLM, by
xtuner convert merge \
${NAME_OR_PATH_TO_LLM} \
${NAME_OR_PATH_TO_ADAPTER} \
${SAVE_PATH} \
--max-shard-size 2GB
Step 1, deploy fine-tuned LLM with any other framework, such as LMDeploy 🚀.
pip install lmdeploy
python -m lmdeploy.pytorch.chat ${NAME_OR_PATH_TO_LLM} \
--max_new_tokens 256 \
--temperture 0.8 \
--top_p 0.95 \
--seed 0
🔥 Seeking efficient inference with less GPU memory? Try 4-bit quantization from LMDeploy! For more details, see here.
We appreciate all contributions to XTuner. Please refer to CONTRIBUTING.md for the contributing guideline.
@misc{2023xtuner,
title={XTuner: A Toolkit for Efficiently Fine-tuning LLM},
author={XTuner Contributors},
howpublished = {\url{https://github.com/InternLM/xtuner}},
year={2023}
}
This project is released under the Apache License 2.0. Please also adhere to the Licenses of models and datasets being used.
No Description
Python Markdown
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》