Towards Code-Oriented LM Embeddings for Surrogate-Assisted Neural Architecture Search
Researchers have developed a novel method called Code-Oriented LM Embeddings (COLE) to improve Neural Architecture Search (NAS). This technique uses off-the-shelf language models to generate embeddings from code representations of neural architectures, bypassing the need for expensive fine-tuning or complex feature engineering. Experiments on NAS-Bench-201 and einspace demonstrated that COLE embeddings outperform other text-based encodings and significantly reduce the evaluation budget required to find high-performing architectures. AI
IMPACT Introduces a more efficient method for designing neural networks, potentially accelerating AI model development.