A developer explored the capabilities and limitations of macOS 27's on-device model, referred to as 'fm', by building a coding agent called fm-mcp. The agent was designed to delegate tasks to the model, such as summarizing logs or extracting data, but encountered several challenges. These included managing the model's context window, which is shared between input and output, and understanding that log data consumes significantly more tokens than standard prose. The developer also found that the model could generate excessive output when asked for non-existent data fields, a problem mitigated by explicitly defining nullable types and unique titles in the JSON schema for structured output. AI
IMPACT Provides insights into practical limitations and prompt engineering techniques for on-device LLMs.
RANK_REASON Developer's personal project detailing technical challenges with a specific OS feature.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →