研究人员推出了一种新颖的基于粒子的变分推理方法——熵正则化最优输运下降法（Entropic Transport Descent, ETD），该方法利用熵正则化最优输运来改进对难解分布的近似。与先前可能在高维情况下遭受方差坍塌的方法不同，ETD的全局协调机制能够保留多模态结构，并在各种实验中与SVGD等现有技术相媲美或表现更优。同时，对变分深度高斯过程（Variational Deep Gaussian Processes, VDGPs）的独立分析表明，后验坍塌（一种变分后验与先验匹配的常见问题）与特定的参数化和初始化有关。该研究提出了一种替代的初始化策略，可以在不影响预测性能的情况下缓解这种坍塌并提高训练稳定性。
AI
arXiv:2606.25265v1 Announce Type: new Abstract: Particle-based variational inference (ParVI) methods approximate an intractable target distribution by evolving an ensemble of interacting samples. Existing approaches rely predominantly on kernel-based repulsion (e.g., SVGD), which…
arXiv cs.LG
TIER_1English(EN)·Francisco Javier S\'aez-Maldonado, Juan Maro\~nas, Daniel Hern\'andez-Lobato·
arXiv:2606.25882v1 Announce Type: new Abstract: DGPs are probabilistic models with remarkable prediction performance that concatenate GPs across several layers. Exact inference in DGPs is intractable, and variational inference is often used to approximate the posterior with a par…
DGPs are probabilistic models with remarkable prediction performance that concatenate GPs across several layers. Exact inference in DGPs is intractable, and variational inference is often used to approximate the posterior with a parametric distribution tuned by minimizing the Kul…
Mean Field Variational Inference (MFVI) is widely understood to underestimate posterior variance. By analysing conjugate Bayesian Linear Regression (BLR), we show that this characterization is incomplete: while MFVI underestimates the variance in parameter space, it can overestim…
Particle-based variational inference (ParVI) methods approximate an intractable target distribution by evolving an ensemble of interacting samples. Existing approaches rely predominantly on kernel-based repulsion (e.g., SVGD), which suffers from variance collapse in high dimensio…
arXiv stat.ML
TIER_1English(EN)·James Odgers, Ben Riegler, Siddharth Swaroop, Vincent Fortuin·
arXiv:2606.25745v1 Announce Type: new Abstract: Mean Field Variational Inference (MFVI) is widely understood to underestimate posterior variance. By analysing conjugate Bayesian Linear Regression (BLR), we show that this characterization is incomplete: while MFVI underestimates t…
Mean Field Variational Inference (MFVI) is widely understood to underestimate posterior variance. By analysing conjugate Bayesian Linear Regression (BLR), we show that this characterization is incomplete: while MFVI underestimates the variance in parameter space, it can overestim…
arXiv stat.ML
TIER_1English(EN)·Jinlin Lai, Antonio Linero, Yuling Yao·
arXiv:2410.14843v4 Announce Type: replace Abstract: Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational i…
