New pre-training method boosts jet foundation model performance

Researchers have developed an improved pre-training method for jet foundation models, building upon the initial OmniJet-alpha work. This new approach enhances next token prediction by incorporating continuous feature vectors alongside token IDs and introduces a combined pre-training strategy that merges masked particle modeling with generative learning objectives. These enhancements significantly boost performance in downstream classification tasks without compromising generative capabilities. AI

IMPACT Introduces a novel pre-training strategy that could improve the efficiency and effectiveness of foundation models in scientific domains.

RANK_REASON Academic paper detailing a new method for foundation models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• High Energy Physics - Phenomenology
• Joschka Birk
• OmniJet-alpha

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →