A new open-source project called OpenMythos has been released, aiming to theoretically reconstruct the architecture of Anthropic's Claude Mythos model. This project implements a Recurrent-Depth Transformer (RDT) with a unique structure involving a prelude, a looped recurrent block, and a coda. The RDT design allows for depth-variable reasoning by recycling a subset of layers multiple times within a single forward pass, a concept distinct from chain-of-thought processing. AI
IMPACT Provides an open-source framework for exploring novel transformer architectures and their potential for deeper reasoning.
RANK_REASON Open-source theoretical reconstruction of a proprietary model architecture.
Read on Mastodon — fosstodon.org →
- Anthropic
- Claude Mythos
- CUDA
- FineWeb-Edu
- Flash Attention 2
- OpenMythos
- Recurrent-Depth Transformer
- Transformer
AI-generated summary · Google Gemini · from 5 sources. How we write summaries →