llama-dash - Local LLM Ops
The developer has created llama-dash, a dashboard and logging proxy designed for self-hosted local LLM operations. This tool aims to provide visibility into model usage, request logging, and scoped access control for local inference stacks. It functions by proxying OpenAI and Anthropic compatible API endpoints, logging requests with cost estimates, and offering features like rate limiting, model allow-lists, and UI-based model management. AI
IMPACT Provides enhanced control and visibility for users running local LLM inference stacks.