Announcing XTuner: An Efficient Finetune Toolkit for LLM

The advent of ChatGPT has provided a glimpse into the dawn of general artificial intelligence. Meanwhile, tech companies are making their own large language models open source one after another.

However, the high hardware costs associated with large models often pose a significant barrier to many researchers.

To democratize access to these powerful models and empower various industries, Shanghai Artificial Intelligence Laboratory has developed XTuner, a low-cost toolkit for training large models. With XTuner, 8GB GPU memory is all you need to create your own AI assistant.

https://github.com/InternLM/xtuner

XTuner offers a wealth of features that can be freely composed and matched. In addition to standalone functions, XTuner introduces three breakthrough technologies allowing developers to truly work “data-centric”:

1. Efficient Data Engine: XTuner supports several popular open-source dataset formats and allows for mixed usage:

Alpaca format

MOSS format

Gunacao format

OpenAI format

…and more being added continuously

pip install xtuner

# To train data in a mixture of Alpaca and Gunacao datasets
xtuner train internlm_7b_qlora_alpaca_enzh_oasst1_e3

Also, XTuner offers full decoupling for different dataset formats based on the characteristics of LLM-Training data. This allows fine-tuning without disrupting the dialogue templates of Chat models.

2. Multiple Training Engines: XTuner pioneers the integration of HuggingFace and OpenMMLab, balancing usability and configurability. It supports both MMEngine Runner and HuggingFace Trainer. Thus, developers with deep customization needs can flexibly configure according to their habits.

pip install xtuner

# To use MMEngine Runner for training
xtuner train internlm_7b_qlora_oasst1_e3

# To use HuggingFace Trainer for training
xtuner train internlm_7b_qlora_oasst1_e3_hf

3. One-click Training Start: XTuner integrates standard procedures for incremental pre-training, single-round & multi-round dialogue instruction tuning. Developers only need to focus on the data. Furthermore, XTuner incorporates technologies like QLoRA, DeepSpeed, and FSDP, offering solutions for varying model sizes under different hardware specifications. Just 8GB GPU memory can fine-tune a 7B model.

pip install 'xtuner[deepspeed]'

# Fine-tuning a Llama2-7B model with only 8GB GPU memory
xtuner train llama2_7b_qlora_oasst1_512_e3 --deepspeed deepspeed_zero2

Developers can focus on the data and leave the rest to XTuner, freeing up more energy to explore the vast universe of large models!

With XTuner, developers can add plugins to large models to supplement their capabilities, or even gain some skills exclusive to ChatGPT. XTuner provides a rich selection of large model plugins on the HuggingFace Hub. Feel free to download and try them out!

https://huggingface.co/xtuner

Inspired by Pac-Man, the XTuner logo reflects a playful spirit. The development team and the open-source community aim to have fun exploring large models and developing a variety of entertaining applications. Your ideas are always welcome in the XTuner discussions, where you can find like-minded partners. The XTuner team will continue to replicate popular community projects at a low cost, inviting more people to ride the wave of large models.

https://github.com/InternLM/xtuner/discussions