LLMs fine-tuned with MinHash curriculum for neural architecture synthesis

Researchers have developed a novel framework for neural architecture search (NAS) that utilizes a MinHash-based similarity scheduling approach to create a progressive curriculum for fine-tuning large language models (LLMs). This method partitions neural architecture code into similarity bands, presenting them in increasing heterogeneity to guide the LLM. When evaluated on the OlympicCoder-7B model within the LEMUR benchmark for CIFAR-10 image classification, the curriculum achieved a peak success rate of 60% at a high-similarity level without post-processing repair. Ablation studies indicated that while the base model without repair performed better than the curriculum model in certain diverse scenarios, interface repair was crucial for improving performance in those cases, suggesting distinct failure modes addressed by curriculum scheduling and repair. AI

IMPACT Introduces a novel curriculum learning approach for LLM-based neural architecture search, potentially improving efficiency and success rates in generating new model architectures.

RANK_REASON The cluster contains an academic paper detailing a new method for neural architecture synthesis using LLMs.

Read on arXiv cs.CV →

• Anujaya Vijayakumar
• CIFAR-10
• Lemur
• Lora
• MinHash
• OlympicCoder-7B
• The Street View House Numbers Dataset

• paper
• model release

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →