SPARTA accelerator boosts LLM edge inference with sparse ternary design · 1 source tracked

Researchers have developed SPARTA, a novel accelerator designed for efficient long-context inference of sparse ternary Large Language Models (LLMs) on edge devices. SPARTA features a phase-aware sparse dataflow and a unified compute fabric that maps ternary operations and attention mechanisms onto shared processing elements, exploiting various forms of sparsity. An adaptive two-stage attention engine further reduces KV-cache traffic by up to 4.8x compared to full attention, with minimal quality loss. Benchmarks show SPARTA significantly outperforms existing hardware like NVIDIA Jetson AGX Orin and AMD Ryzen 7 9700X in generation throughput and energy efficiency. AI

IMPACT Enables more powerful LLM capabilities on resource-constrained edge devices, potentially expanding AI applications in mobile and embedded systems.

RANK_REASON The cluster contains an academic paper detailing a new hardware architecture for LLM inference. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AMD Ryzen 7 9700X
• arXiv
• FIGLUT
• Hugging Face
• NVIDIA Jetson AGX Orin 64GB
• SPARTA
• Tellme Networks

• infra
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →