{"id":5647923,"date":"2023-05-19T11:01:21","date_gmt":"2023-05-19T15:01:21","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5647923"},"modified":"2023-07-21T09:41:58","modified_gmt":"2023-07-21T13:41:58","slug":"how-to-finetune-gpt-like-large-language-models-on-a-custom-dataset","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/blog\/how-to-finetune-gpt-like-large-language-models-on-a-custom-dataset\/","title":{"rendered":"How To Finetune GPT Like Large Language Models on a Custom Dataset"},"content":{"rendered":"

Takeaways<\/h3> Learn how to finetune large language models (LLMs) on a custom dataset. We will be using Lit-GPT<\/a>, an optimized collection of open-source LLMs for finetuning and inference. It supports – LLaMA 2, Falcon, StableLM, Vicuna, LongChat, and a couple of other top performing open source large language models.<\/div>\n

The AI community’s effort has led to the development of many high-quality open-source LLMs, including but not limited to LLaMA 2, Falcon, StableLM, and Pythia. You can finetune these models on a custom instruction dataset to adapt to your specific task, such as training a chatbot to answer financial questions.<\/p>\n

Lightning AI recently launched Lit-GPT, the second LLM implementation in the Lit-* series after Lit-LLaMA<\/a>. The goal of these Lit-* series is to provide the AI\/ML community with a clean, solid, and optimized implementation of large language models with pretraining and finetuning support using LoRA<\/a> and Adapter<\/a>.<\/p>\n

We will guide you through the process step by step, from installation to model download and data preparation to finetuning. If you have already completed a step or are confident about it, feel free to skip it.<\/p>\n

Installing Lit-GPT \ud83e\udd9c<\/h2>\n

The Lit-GPT repository is available in the Lightning AI GitHub organization here<\/a>. To get started, clone the repository and install its dependencies.<\/p>\n

git clone <https:\/\/github.com\/Lightning-AI\/lit-gpt\r\ncd lit-gpt\r\n\r\n<\/pre>\n
We are using FlashAttention<\/a>, a fast and memory-efficient implementation of attention, which is only available in PyTorch Nightly 2.1 at the moment of writing this article.<\/p>\n
# for cuda\r\npip install --index-url <https:\/\/download.pytorch.org\/whl\/nightly\/cu118> --pre 'torch>=2.1.0dev'<\/pre>\n
# for cpu\r\npip install --index-url <https:\/\/download.pytorch.org\/whl\/nightly\/cpu> --pre 'torch>=2.1.0dev'\r\n\r\n<\/pre>\n
Finally, install the dependencies using pip install -r requirements.txt<\/code> .<\/p>\n
Downloading the model weights<\/h2>\n
In order to use the model or finetune it we need a pre-trained weight. Thanks to the effort of open source teams, we have a bunch of open source weights that we can use for commercial purposes. Lit-GPT supports various LLMs including Llama 2, Falcon, Vicuna, and RedPajama-INCITE. You can check all the supported models here<\/span><\/a>. We use the RedPajama-INCITE 3B parameter weights in this tutorial. You can find the instructions to download other supported weights like LLaMA 2 and Falcon in this tutorial section<\/span><\/a>.<\/p>\n
\r\n# download the model weights\r\npython scripts\/download.py --repo_id togethercomputer\/RedPajama-INCITE-Base-3B-v1<\/pre>\n
# convert the weights to Lit-GPT format\r\npython scripts\/convert_hf_checkpoint.py --checkpoint_dir checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1\r\n\r\n<\/pre>\n
\"\"<\/p>\n
You will see, gpt_neox<\/code> layers being mapped to the Lit-GPT layers in the terminal. After this step, you can find the downloaded weights in the checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1<\/code> folder.<\/p>\n
Prepare the dataset<\/h2>\n
In this tutorial, we will use the Dolly 2.0 instruction dataset<\/a> by Databricks for finetuning. Finetuning involves two main steps- first, we process the dataset in the Lit-GPT format and then we run the finetuning script on the processed dataset.<\/p>\n
Instruction datasets typically have three keys: instruction, input (optional context for the given instruction), and the expected response from the LLM. Below is a sample example of instruction data:<\/p>\n
[\r\n  {\r\n    \"instruction\": \"Arrange the given numbers in ascending order.\",\r\n    \"input\": \"2, 4, 0, 8, 3\",\r\n    \"output\": \"0, 2, 3, 4, 8\"\r\n  },\r\n...\r\n]<\/pre>\n
The dolly 2.0 dataset<\/a> comes in JSON Lines<\/a> format, which is plainly speaking a text file with rows of JSON data. It is a convenient format when processing one record at a time. The Dolly dataset contains the following keys –<\/p>\n
{\r\n  \"instruction\": \"When did Virgin Australia start operating?\",\r\n  \"context\": \"Virgin Australia, the trading name of Virgin Australia Airlines Pty Ltd, is an Australian-based airline. It is the largest airline by fleet size to use the Virgin brand. It commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route. It suddenly found itself as a major airline in Australia's domestic market after the collapse of Ansett Australia in September 2001. The airline has since grown to directly serve 32 cities in Australia, from hubs in Brisbane, Melbourne and Sydney.\",\r\n  \"response\": \"Virgin Australia commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.\",\r\n  \"category\": \"closed_qa\"\r\n}<\/pre>\n
We need to rename context<\/code> to input<\/code> and response<\/code> to output<\/code> and we are all set to process our data.<\/p>\n
with open(file_path, \"r\") as file:\r\n    data = file.readlines()\r\n    data = [json.loads(line) for line in data]\r\nfor item in data:\r\n    item[\"input\"] = item.pop(\"context\")\r\n    item[\"output\"] = item.pop(\"response\")\r\n\r\n<\/pre>\n
We can modify the existing Alpaca script<\/a> for our data preparation. This script downloads data from tloen\u2019s Alpaca-lora<\/a> project and saves the processed data. It includes a prepare<\/code> function that loads the raw instruction
\ndataset, creates prompts, and tokenizes them using the model tokenizer provided in the checkpoint_dir<\/code>. The tokenized data is split into training and test sets based on the test_split_size<\/code> provided and saved to the destination_path<\/code>.<\/p>\n
To modify the Alpaca script, open it from here<\/a> and edit the prepare<\/code> function. This is how our final function would look after mapping the keys appropriately.<\/p>\n
DATA_FILE = \"https:\/\/huggingface.co\/datasets\/databricks\/databricks-dolly-15k\/resolve\/main\/databricks-dolly-15k.jsonl\"\r\nDATA_FILE_NAME = \"dolly_data_cleaned_archive.json\"<\/pre>\n
def prepare(\r\n    destination_path: Path = Path(\"data\/dolly\"),\r\n    checkpoint_dir: Path = Path(\"checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1\"),\r\n    test_split_size: int = 2000,\r\n    max_seq_length: int = 256,\r\n    seed: int = 42,\r\n    mask_inputs: bool = False, # as in alpaca-lora\r\n    data_file_name: str = DATA_FILE_NAME,\r\n) -> None:\r\n    \"\"\"Prepare the Dolly dataset for instruction tuning.\r\n    The output is a training and validation dataset saved as `train.pt` and `val.pt`,\r\n    which stores the preprocessed and tokenized prompts and labels.\r\n    \"\"\"\r\n    destination_path.mkdir(parents=True, exist_ok=True)\r\n    file_path = destination_path \/ data_file_name\r\n    download(file_path)\r\n    tokenizer = Tokenizer(checkpoint_dir \/ \"tokenizer.json\", checkpoint_dir \/ \"tokenizer_config.json\")\r\n    with open(file_path, \"r\") as file:\r\n        data = file.readlines()\r\n        data = [json.loads(line) for line in data]\r\n    for item in data:\r\n    item[\"input\"] = item.pop(\"context\")\r\n    item[\"output\"] = item.pop(\"response\")\r\n    # Partition the dataset into train and test\r\n    train_split_size = len(data) - test_split_size\r\n    train_set, test_set = random_split(\r\n        data, lengths=(train_split_size, test_split_size), generator=torch.Generator().manual_seed(seed)\r\n        )\r\n    train_set, test_set = list(train_set), list(test_set)\r\n    print(f\"train has {len(train_set):,} samples\")\r\n    print(f\"val has {len(test_set):,} samples\")\r\n    print(\"Processing train split ...\")\r\n    train_set = [prepare_sample(sample, tokenizer, max_seq_length, mask_inputs) for sample in tqdm(train_set)]\r\n    torch.save(train_set, file_path.parent \/ \"train.pt\")\r\n    print(\"Processing test split ...\")\r\n    test_set = [prepare_sample(sample, tokenizer, max_seq_length, mask_inputs) for sample in tqdm(test_set)]\r\n    torch.save(test_set, file_path.parent \/ \"test.pt\")\r\n\r\n<\/pre>\n
Finally, let\u2019s run the script by providing the data path and the model checkpoint directory.<\/p>\n
python scripts\/prepare_yourscript.py \\\r\n  --destination_path data\/dolly \\\r\n  --checkpoint_dir checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1\r\n\r\n<\/pre>\n
\"\"<\/p>\n
<\/h2>\n
Finetuning the RedPajama-INCITE model<\/h2>\n
Once you have completed all the above steps, it is straightforward to start finetuning. You need to run the
\nfinetune\/adapter.py<\/code> script by providing your data path.<\/p>\n
python finetune\/adapter.py \\\r\n  --data_dir data\/dolly \\\r\n  --checkpoint_dir checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1\r\n  --out_dir out\/adapter\/dolly<\/pre>\n
\"\"<\/p>\n
You can update the default number of GPUs, micro-batch size, and all the other hyperparameters in the finetuning script here<\/a>.<\/p>\n
You can play with your finetuned model using the generate\/adapter.py<\/code> script by trying different prompts and turning the model temperature.<\/p>\n
python generate\/adapter.py \\\r\n  --adapter_path out\/adapter\/dolly\/lit_model_adapter_finetuned.pth \\\r\n  --checkpoint_dir checkpoints\/togethercomputer\/RedPajama-INCITE-Base-3B-v1 \\\r\n  --prompt \"who is the author of Game of thrones?\"\r\n\r\n<\/pre>\n
\"\"<\/p>\n
<\/h2>\n
Learn more about large language models and efficient finetuning techniques \ud83d\udc47<\/h2>\n
This article provided you with short step-by-step instructions to finetune your own large language model. We saw that thanks to parameter-efficient finetuning techniques, it\u2019s possible to do it on a single GPU. If you want to learn more about these techniques, check out our more in-depth guides below.<\/p>\n