A new open-source project called OpenMythos has been released, aiming to theoretically reconstruct the architecture of Anthropic's Claude Mythos model. This project implements a Recurrent-Depth Transformer (RDT) with a unique structure involving a prelude, a looped recurrent block, and a coda. The RDT design allows for depth-variable reasoning by recycling a subset of layers multiple times within a single forward pass, a concept distinct from chain-of-thought processing. AI
影响 Provides an open-source framework for exploring novel transformer architectures and their potential for deeper reasoning.
排序理由 Open-source theoretical reconstruction of a proprietary model architecture.
在 Mastodon — fosstodon.org 阅读 →
- Anthropic
- Claude Mythos
- CUDA
- FineWeb-Edu
- Flash Attention 2
- OpenMythos
- Recurrent-Depth Transformer
- Transformer
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →