OpenRFM: Dissecting Relational In-Context Learning
Researchers have introduced OpenRFM, a new framework designed to improve Relational Foundation Models (RFMs) for in-context learning. The study identifies key limitations in existing models, such as the Relational Transformer (RT), by analyzing both model-side underdetermination and data-side pre-training deficiencies. OpenRFM addresses these by incorporating a dual-stage ICL architecture and a novel pre-training strategy that combines synthetic and real-world data, leading to a significant performance improvement over previous methods. AI
IMPACT Introduces a novel architecture and training methodology that significantly improves performance on relational in-context learning tasks.