Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacy
Researchers have developed a method to apply differential privacy to two-layer ReLU neural networks, a significant step beyond current limitations to convex problems. This new approach uses a stochastic approximation of a dual formulation to create a strongly convex problem, enabling more accurate privacy bounds for methods like NoisyCGD. Empirical tests show that this technique achieves privacy-utility trade-offs comparable to DP-SGD on benchmark classification tasks. AI
IMPACT Expands the applicability of differential privacy to more complex neural network architectures, potentially enabling more secure AI development.