Introduction: Getting the Most out of LoRA I’ve run hundreds, if not thousands, of experiments involving LoRA over the past few months. A few weeks ago, I took the time to delve deeper into some of the hyperparameter choices. This is more of an experimental diary presented in sequential order. I hope it proves … Continue reading Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments