A new paper identifies numerical issues in standard divergence operations between nearly equal distributions, including Kullback-Leibler (KL) and Jensen-Shannon divergences. These problems arise from rounding and cancellation errors in forward and gradient computations, impacting applications like knowledge distillation and reinforcement learning. The authors propose stable reformulations and provide validated implementations to address these numerical challenges. AI
IMPACT Addresses fundamental numerical stability issues in common machine learning optimization techniques.
RANK_REASON Academic paper detailing numerical issues and proposing solutions. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- Influence Flower
- Jensen-Shannon divergence
- knowledge distillation
- Kullback--Leibler divergence
- Monte Carlo
- Proximal Methods for Plant Stress Detection Using Optical Sensors and Machine Learning
- reinforcement learning
- ScienceCast
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →