A new research paper introduces GRACE, a system designed to accelerate generative recommenders for real-time ad retrieval. GRACE addresses two key challenges: ensuring ad eligibility through a novel Generative Target Matching (GTM) technique and optimizing compute costs and latency for encoder-decoder Transformers. The system redesigns the decoder for wide-beam, short-sequence generation and includes optimizations for attention kernels, KV cache, and beam search, significantly reducing latency and keeping generative retrieval within strict requirements. AI
IMPACT Optimizes generative models for real-time applications, potentially improving ad targeting efficiency and reducing computational costs.
RANK_REASON The cluster contains a research paper detailing a new system and methodology. [lever_c_demoted from research: ic=1 ai=1.0]
Read on arXiv cs.IR (Information Retrieval) →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
