Toward Scalable and Valid Conditional Independence Testing with Spectral Representations
Researchers have developed a new method for testing conditional independence (CI) using spectral representations derived from partial covariance operators. This approach aims to overcome the limitations of existing CI tests, which often rely on restrictive assumptions or suffer from poor scalability. The proposed technique involves learning representations through a bi-level contrastive algorithm and is supported by theoretical guarantees linking representation learning error to test performance. Experiments indicate that this method provides a statistically sound and scalable pathway for CI testing, integrating principles from kernel-based theory with modern representation learning. AI
IMPACT Introduces a novel statistical technique that could improve feature selection and causal inference in AI models.