When Do Fewer Coordinates Suffice in DP-SGD?
Researchers have developed a new method called TP-TopK DP-SGD to improve the efficiency of differentially private stochastic gradient descent. This technique aims to reduce the computational overhead by updating fewer coordinates during private training without sacrificing optimization signal. The method involves a two-phase approach where an initial private phase identifies relevant coordinates for the main training phase, potentially reducing noise impact from the full parameter dimension to a smaller active dimension. AI
IMPACT This research could lead to more efficient and scalable differentially private machine learning models.