This article details a method for enabling tool-calling capabilities in small, on-device Large Language Models (LLMs) that do not natively support this feature. The approach involves describing available tools within the model's system prompt using an OpenAI-compatible JSON schema format. The core challenge lies in parsing the model's output, as different LLM families respond with varied syntaxes, deviating from the expected `<tool_call>` format. The solution implements a flexible parser that can interpret these diverse responses, allowing agents to reliably invoke tools like web searches or data saving functions even with limited local models. AI
IMPACT Enables broader adoption of agentic workflows on resource-constrained devices by overcoming limitations in smaller LLMs.
RANK_REASON The article describes a technical workaround for enabling a specific feature (tool calling) in existing LLM frameworks for smaller, on-device models, rather than a new model release or core research.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
