Terastal: Layer-Variant-based Scheduling for Real-Time Multi-DNN Workloads on Heterogeneous Accelerators
Researchers have developed Terastal, a new framework designed to improve the scheduling of multiple deep neural networks (DNNs) on heterogeneous accelerators for soft real-time applications. The system addresses latency differences between accelerators by creating customized "layer variants," which are optimized implementations of DNN layers. Terastal combines offline design and online scheduling to balance timing and accuracy, reportedly reducing deadline misses by over 30% compared to existing methods while maintaining high accuracy. AI
IMPACT Optimizes real-time DNN execution on specialized hardware, potentially improving performance and reliability for AI applications.