{"id":5647720,"date":"2023-04-12T22:09:40","date_gmt":"2023-04-13T02:09:40","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5647720"},"modified":"2023-06-22T13:31:06","modified_gmt":"2023-06-22T17:31:06","slug":"understanding-llama-adapters","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/article\/understanding-llama-adapters\/","title":{"rendered":"Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters"},"content":{"rendered":"

Key takeaway<\/h3> Learn how popular parameter-efficient finetuning methods for LLM work: prefix tuning, adapters, and LLaMA-Adapter. <\/div>\n

In the rapidly evolving field of artificial intelligence, utilizing large language models in an efficient and effective manner has become increasingly important.<\/span><\/p>\n

Parameter-efficient finetuning stands at the forefront of this pursuit, allowing researchers and practitioners to reuse pretrained models while minimizing their computational and resource footprints. It also allows us to train AI models on a broader range of hardware, including devices with limited computational power, such as laptops, smartphones, and IoT devices. Lastly, with the increasing focus on environmental sustainability, parameter-efficient finetuning reduces the energy consumption and carbon footprint associated with training large-scale AI models.<\/span><\/p>\n

This article explains the broad concept of finetuning and discusses popular parameter-efficient alternatives like prefix tuning and adapters. Finally, we will look at the recent LLaMA-Adapter method and see how we can use it in practice.<\/span><\/p>\n

 <\/p>\n

Finetuning Large Language Models<\/h2>\n

Since GPT-2 (<\/span>Radford et al.<\/span><\/a><\/span>) and GPT-3 (<\/span>Brown et al.<\/span><\/a><\/span>), we have seen that generative large language models (LLMs) pretrained on a general text corpus are capable of in-context learning, which doesn’t require us to further train or finetune pretrained LLMs if we want to perform specific or new tasks that the LLM wasn’t explicitly trained on. Instead, we can directly provide a few examples of a target task via the input prompt, as illustrated in the example below.<\/p>\n

 
\n<\/span><\/p>\n

\"In-context
\n <\/p>\n

In-context learning is a valuable and user-friendly method for situations where direct access to the large language model (LLM) is limited, such as when interacting with the LLM through an API or user interface.
\nHowever, if we have access to the LLM, adapting and finetuning it on a target task using data from a target domain usually leads to superior results. So, how can we adapt a model to a target task? There are three conventional approaches outlined in the figure below.<\/p>\n

 
\n\"the
\n <\/p>\n

These methods above are compatible with generative (decoder-style) models such as GPT and embedding-focused (encoder-style) models such as BERT. In contrast to these three approaches, in-context learning only applies to generative models. It’s also worth highlighting that when we finetune generative models, we work with and build on the embeddings they create instead of the generated output texts.<\/span><\/p>\n

 
\nFeature-based approach<\/span><\/strong><\/p>\n

In the feature-based approach, we load a pretrained LLM and apply it to our target dataset. Here, we are particularly interested in generating the output embeddings for the training set, which we can use as input features to train a classification model. While this approach is particularly common for embedding-focused like BERT, we can also extract embeddings from generative GPT-style models (you can find an example in our blog post\u00a0<\/span>here<\/span><\/a>).<\/span><\/p>\n

The classification model can then be a logistic regression model, a random forest, or XGBoost — whatever our hearts desire. (However, based on my experience, linear classifiers like logistic regression perform best here.)<\/span><\/p>\n

Conceptually, we can illustrate the feature-based approach with the following code:<\/span><\/p>\n

 <\/p>\n


\nmodel = AutoModel.from_pretrained(\"distilbert-base-uncased\")\n\n# ...
\n# tokenize dataset
\n# ...\n\n# generate embeddings
\n@torch.inference_mode()
\ndef get_output_embeddings(batch):
\n    output = model(
\n        batch[\"input_ids\"],
\n        attention_mask=batch[\"attention_mask\"]
\n    ).last_hidden_state[:, 0]
\nreturn {\"features\": output}\n\ndataset_features = dataset_tokenized.map(
\n  get_output_embeddings, batched=True, batch_size=10)\n\nX_train = np.array(dataset_features[\"train\"][\"features\"])
\ny_train = np.array(dataset_features[\"train\"][\"label\"])\n\nX_val = np.array(dataset_features[\"validation\"][\"features\"])
\ny_val = np.array(dataset_features[\"validation\"][\"label\"])\n\nX_test = np.array(dataset_features[\"test\"][\"features\"])
\ny_test = np.array(dataset_features[\"test\"][\"label\"])\n\n# train classifier
\nfrom sklearn.linear_model import LogisticRegression\n\nclf = LogisticRegression()
\nclf.fit(X_train, y_train)\n\nprint(\"Training accuracy\", clf.score(X_train, y_train))
\nprint(\"Validation accuracy\", clf.score(X_val, y_val))
\nprint(\"test accuracy\", clf.score(X_test, y_test))
\n<\/code>