New research reveals instability in Dynamic Tensor Rematerialization for DNN training

Researchers have identified deterministic instability and feasibility inversions within Dynamic Tensor Rematerialization (DTR), an eviction policy for training deep neural networks under memory constraints. Their analysis, conducted on a reference simulator using execution traces, revealed that subtle differences in memory budgets can lead to vastly different execution speeds and even cause out-of-memory errors. The study suggests these issues stem from specific scoring terms within DTR, indicating distinct pathologies rather than a single underlying mechanism. AI

IMPACT Identifies potential performance and stability issues in memory management techniques crucial for large-scale AI model training.

RANK_REASON Academic paper detailing a new finding about an AI infrastructure technique. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Dynamic Tensor Rematerialization
• long short-term memory
• Mahesh Reddy Pagadala
• ResNet-32
• simrd

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →