SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
Researchers have developed SPADE, a novel autoregressive transformer model designed for simulating high-granularity calorimeter data in particle physics. Unlike previous methods that embed multiple features jointly, SPADE embeds them independently and introduces a delay between feature streams. This approach allows the standard self-attention mechanism to learn intra-token correlations effectively. SPADE demonstrates competitive performance against existing models for photon shower generation in the ILD detector and offers a new pathway for applying LLM-style pretraining to complex, multi-feature datasets. AI
IMPACT Introduces a new transformer architecture applicable to complex scientific simulation, potentially enabling LLM-style pretraining for high-dimensional data.