{"id":5648254,"date":"2023-06-14T22:10:05","date_gmt":"2023-06-15T02:10:05","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5648254"},"modified":"2023-06-22T14:37:03","modified_gmt":"2023-06-22T18:37:03","slug":"finetuning-falcon-efficiently","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/finetuning-falcon-efficiently\/","title":{"rendered":"Finetuning Falcon LLMs More Efficiently With LoRA and Adapters"},"content":{"rendered":"

Key takeaway<\/h3>
\nUsing parameter-efficient finetuning methods outlined in this article, it’s possible to finetune an open-source LLM like Falcon in 1 hour on a single GPU instead of a day on 6 GPUs.<\/div>\n

Finetuning allows us to adapt pretrained LLMs in a cost-efficient manner. But which method should we use? This article compares different parameter-efficient finetuning methods for the latest top-performing open-source LLM, Falcon.<\/p>\n

 <\/p>\n

Pretraining and Finetuning LLMs<\/span><\/h2>\n

Before we dive into the LLM finetuning details, let’s briefly recap how we train LLMs in general.<\/span><\/p>\n

LLMs are trained in two stages. The first stage is an expensive pretraining step to train the models on a large, unlabeled dataset containing trillions of words. The resulting models are often called <\/span>foundation<\/span><\/em><\/span> models since they have general capabilities and can be adapted for various downstream tasks. A classic example of a pretrained model is GPT-3.<\/span><\/p>\n

The second stage is finetuning such a foundation model. This typically involves training the pretrained model to follow instructions or perform another specific target task (for example, sentiment classification). ChatGPT (which started as a finetuned version of the GPT-3 foundation model) is a typical example of a model that was finetuned to follow instructions. Using parameter-efficient finetuning methods outlined in this article, it’s possible to finetune an LLM in 1 hour on a single GPU instead of a day on 6 GPUs.<\/span><\/p>\n

 <\/p>\n

\n
\"\"

Finetuning a pretrained LLM to follow instructions<\/p><\/div>\n<\/div>\n

Finetuning also allows the model to better adapt to specific domains or types of text that were not well represented in its original training data. For example, we might finetune a model on medical literature if we want it to understand and generate medical texts. <\/span><\/p>\n

Besides building custom chatbots, finetuning allows customization of these models to specific business needs to offer superior performance in targeted applications. Furthermore, it can also provide a data privacy advantage when data cannot be uploaded or shared with cloud APIs.<\/span><\/p>\n

This article aims to illustrate how to finetune a top-performing LLM efficiently and cost-effectively in a few hours on a single GPU.<\/span><\/strong><\/span><\/p>\n

 <\/p>\n

Finetuning Versus ChatGPT<\/span><\/h2>\n

In the era of ChatGPT, why do we care about finetuning models in the first place?<\/span><\/p>\n

The problem with closed models such as OpenAI’s ChatGPT and Google’s Bard is that they cannot be readily customized, which makes them less attractive for many use cases. However, fortunately, we have seen a large number of open-source LLMs emerging in recent months. (While ChatGPT and Bard have strong in-context learning capabilities, finetuned models outperform generalist models on specific tasks; recent research examples highlighting this include <\/span>Goat<\/span><\/a><\/span> and <\/span>Gorilla<\/span><\/a><\/span>.)<\/span><\/p>\n

 <\/p>\n

Open Source LLMs and the Falcon Architecture<\/span><\/h2>\n

Finetuning open-source LLMs has several benefits, such as better customization capabilities and task performance. Furthermore, open-source LLMs are an excellent testbed for researchers to develop novel techniques. But if we adopt an open-source model today, which one should it be?<\/span><\/p>\n

As of this writing, the Falcon model, developed by <\/span>Technology Innovation Institute<\/span><\/a><\/span>, is currently the top-performing open-source LLM. And in this article, we will learn how to finetune it efficiently, for example, on your custom dataset.<\/span><\/p>\n

 <\/p>\n

\n
\"\"

Excerpt from the OpenLLM leaderboard<\/a><\/p><\/div>\n<\/div>\n

Falcon LLMs come in different sizes: as of this writing, there’s a 7 billion parameter (Falcon 7B) variant and a 40 billion parameter variant (Falcon 40B). Furthermore, each size comes as a foundation (Falcon 7B and Falcon 40B) and instruction-tuned model (Falcon 7B-instruct and Falcon 40B-instruct). The instruction-tuned models are already finetuned for general-purpose tasks (similar to ChatGPT), but they can be further finetuned on domain-specific data if needed. (PS: A <\/span>180B version is also in the works<\/span><\/a><\/span>.)<\/span><\/p>\n

Note that the Falcon model is fully open-source and was released under a permissive <\/span>Apache version 2.0<\/span><\/a><\/span> license, which permits unrestricted commercial use — it’s the same license PyTorch Lightning, TensorFlow, and OpenOffice use, for example.<\/span><\/p>\n

How is Falcon different from other LLMs such as GPT or LLaMA?<\/span><\/strong><\/span><\/p>\n

Besides the better performance on the OpenLLM leaderboard, as highlighted above, there are also some small architectural differences between Falcon, LLaMA, and GPT. LLaMA (<\/span>Touvron et al. 2023<\/span><\/a><\/span>) introduced the following architecture improvements, which likely contributed to LLaMA’s better performance over GPT-3 (<\/span>Brown at al. 2020<\/span><\/a><\/span>):<\/span><\/p>\n