FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data
Researchers have developed FGRPO, a new framework for federated learning that enables decentralized fine-tuning of reasoning models while preserving data privacy. This approach addresses the privacy risks associated with centralizing data from distributed owners by using group relative policy optimization (GRPO). FGRPO incorporates an adaptive aggregation mechanism to manage instability caused by varying reward scales across different tasks, ensuring robust convergence on non-independent and identically distributed (non-IID) data. AI
IMPACT Enhances privacy in decentralized AI model training, potentially enabling broader collaboration on sensitive datasets.