The Forgotten Pioneer: The Legendary Four Open-Source Models That First Topped the Chatbot Arena
Four early open-source models—Vicuna-13B, Guanaco-33B, Vicuna-33B, and WizardLM-70B—briefly dominated the Chatbot Arena, outperforming early commercial offerings. Vicuna-13B, trained for $300, pioneered the use of ChatGPT conversation data for fine-tuning and indirectly led to the creation of the Chatbot Arena platform. Guanaco-33B demonstrated the power of QLoRA for efficient fine-tuning on consumer hardware, a technique that revolutionized open-source model development. WizardLM-70B, developed by Microsoft, introduced the Evol-Instruct method for generating complex training data, though its successor, WizardLM-2, was mysteriously removed from public access shortly after its release. AI
IMPACT These early open-source models pioneered efficient training and data generation techniques, paving the way for today's advanced LLMs.