A developer shared their experience building a production service using open-source LLMs for appointment scheduling, ultimately deciding to shut down the project due to significant challenges. While open-source models have improved and are suitable for individual use, integrating them into a multi-party service proved problematic. Issues included unreliable API uptime from providers, difficulties with PydanticAI's asynchronous design in a synchronous environment, and models frequently returning broken structured data that validation could not fix, leading to a frustrating user experience. AI
IMPACT Highlights the difficulties of deploying LLMs in multi-party production environments due to reliability and data integrity issues.
RANK_REASON Developer's personal account of challenges using LLMs in a production service.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
