PulseAugur
中
实时 07:40:20

New research enables faster, more efficient LLMs on mobile devices

作者 PulseAugur 编辑部 · [2 个来源] ·

Researchers have developed new methods for deploying large language models on mobile devices, focusing on reducing latency and memory usage. One approach, MobileLLM-Flash, uses hardware-in-the-loop architecture search and attention skipping to create efficient models that can be deployed on standard mobile runtimes. Another framework integrates application-specific LoRAs into a single frozen inference graph, enabling dynamic task switching and multi-stream decoding for faster response generation on devices like the Samsung Galaxy S24 and S25. AI

影响 Advances in on-device LLM efficiency could accelerate the integration of generative AI into mobile applications and edge computing.

排序理由 The cluster contains two arXiv papers detailing novel research on on-device LLM design and acceleration.

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

New research enables faster, more efficient LLMs on mobile devices

报道来源 [2]

  1. arXiv cs.LG TIER_1 English(EN) · Hanxian Huang, Igor Fedorov, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghurama ·

    MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment

    arXiv:2603.15954v2 Announce Type: replace Abstract: Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce near-real-time responses and exhibit broad hardware…

  2. arXiv cs.CL TIER_1 English(EN) · Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala, Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungB ·

    Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM

    arXiv:2604.18655v2 Announce Type: replace-cross Abstract: Deploying large language models (LLMs) on smartphones poses significant engineering challenges due to stringent constraints on memory, latency, and runtime flexibility. In this work, we present a hardware-aware framework f…

相关实体

相关话题