A small, open-source language model has been tested for its ability to run on budget Android phones with limited RAM. The model successfully operated on a Galaxy A05s with 4 GB of RAM, answering a stock question in 4.25 seconds. However, the performance varied depending on whether the network was active and the context length, with speeds dropping to 11.0 tokens/sec when 768 tokens were already in context. The model is specifically trained for pharmacy software and is not suitable for general chatbot applications, with its accuracy needing further evaluation on the target device. AI
IMPACT Enables the potential for on-device AI applications on lower-end hardware, expanding accessibility for specialized tasks.
RANK_REASON The item discusses the performance of a specific LLM on a consumer device, which falls under tooling and infrastructure rather than a frontier release or significant industry event.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →