A new toolkit called HoliBench has been developed to address the challenges of deploying foundation models, including large language models, on resource-constrained CPS-IoT applications. This toolkit offers a unified workflow for evaluating model accuracy, latency, and energy consumption across a range of devices, from single-board computers to GPU servers. HoliBench's platform abstraction layer and support for multiple model modalities and inference engines enable comprehensive cross-device measurement, revealing trade-offs that traditional tools miss. The open-source infrastructure aims to facilitate deployment-aware evaluation of foundation models. AI
IMPACT Enables more efficient deployment of foundation models on edge devices by providing a unified evaluation framework.
RANK_REASON The item is a research paper detailing a new toolkit for evaluating foundation models. [lever_c_demoted from research: ic=1 ai=1.0]
- CPS-IoT applications
- GPU servers
- HoliBench
- Inesh Chakrabarti
- large-language models
- single-board computers
- Time Series Foundation Models
- vision-language model
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
