VIDRAFT has released Aether-7B-5Attn, a fully open-source foundation model under the Apache 2.0 license. Unlike many "open" models that only provide weights, Aether-7B-5Attn includes its architecture, training data recipe, code, hyperparameters, and logs, enabling full reproducibility. The model features a novel heterogeneous attention mechanism arranged in a Latin square across its 49 layers to control for depth bias. A key operational note is that inference must be performed with a batch size of 1 to avoid output corruption due to specific attention branches not processing padding masks. AI
IMPACT Provides a fully reproducible model for on-premise and sovereign AI development, enabling verification and rebuilding without vendor dependency.
RANK_REASON Release of a new open-source foundation model with reproducible training details. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
