New AI inference optimization flow uses LLMs and execution traces

Researchers have developed TraceDSE, a novel agentic design space exploration flow for optimizing AI inference workloads on heterogeneous edge Systems-on-Chip (SoCs). This approach jointly maps workloads to processing units (PUs) and configures them, addressing the combinatorial complexity that makes exhaustive search infeasible. Unlike previous methods that rely on limited feedback, TraceDSE utilizes richer system execution traces, enabling an LLM proposer to generate candidates and an LLM critic to analyze traces for bottlenecks and suggest refinements. This iterative process significantly improves decision quality and search effectiveness, outperforming state-of-the-art black-box optimization tools by up to 68% in hypervolume improvement while requiring substantially fewer hardware evaluations. AI

IMPACT This research could lead to more efficient AI deployment on edge devices by optimizing hardware configuration and workload mapping.

RANK_REASON The item is a research paper detailing a new method for optimizing AI inference workloads. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AI inference workloads
• Bayesian optimization
• Geetha Prasuna Yarramneni
• Heterogeneous Edge SoCs
• Intel Meteor Lake SoC
• LLM
• TraceDSE

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →