Reflect, Ripple, Reinovate

Category: fine tuning

Finetuning Llama 3 with Odds Ratio Preference Optimization

May 2, 2024

—

by

ross

in Advanced, blogathon, dataset, fine tuning, Generative AI, Guide, HuggingFace, Kaggle, language models, Large Language Models, LLM, LLMs, Models, optimization, PyTorch, Supervised, time, training, Unsupervised

Introduction Large Language Models are often trained rather than built, requiring multiple steps to perform well. These steps, including Supervised Fine Tuning (SFT) and Preference Alignment, are crucial for learning new things and aligning with human responses. However, each step takes a significant amount of time and computing resources. One solution is the Odd Ratio…