A new paper introduces the Foundation Model Deployment Portfolio (FMDP) problem, which addresses how transportation management centers can optimally deploy various foundation models like LLMs and VLMs. The FMDP problem aims to minimize total cost of ownership while meeting quality, latency, and safety constraints on shared GPU resources. Researchers developed a greedy heuristic to solve this NP-hard problem, demonstrating in a case study that a mixed portfolio of open-source and closed-API models could significantly reduce costs compared to an all-closed-API approach. AI
IMPACT Provides a framework for cost-effective integration of LLMs and VLMs in specialized operational environments.
RANK_REASON The cluster contains an academic paper detailing a new problem formulation and heuristic for optimizing foundation model deployment. [lever_c_demoted from research: ic=1 ai=1.0]
- 0-1 knapsack problem
- arXiv
- Foundation Model Deployment Portfolio
- graphics processing unit
- Hugging Face
- large language models
- Transportation Management Center of the Ozarks
- vision-language models
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
