I Crammed RAG, a Vector Database, and a Gemma LLM into a Mobile App. Here’s What Happened.
A developer built a mobile app called Smart Notes that allows users to query their personal notes without an internet connection. The app utilizes two Gemma models for local inference and embedding generation, storing vector data in an on-device database. This approach ensures user privacy by keeping all data and processing entirely on the mobile device, avoiding the need for cloud APIs or network access after the initial model download. AI
IMPACT Enables private, offline querying of personal data using on-device LLMs, reducing reliance on cloud services for note-taking applications.