Transformers criticized for limiting AI progress; new architectures emerge

The current transformer architecture, which powers most AI models, lacks persistent memory and continual learning capabilities, hindering progress towards superintelligence. While retrieval-augmented generation (RAG) offers workarounds, it doesn't fundamentally change the models' static nature. Future AI development is shifting towards architectures inspired by the human brain, exploring state-space models, recurrent-style models, and test-time learning to enable true learning from experience. AI

IMPACT Suggests that current AI architectures may not achieve superintelligence, highlighting the need for new models focused on continual learning and memory.

RANK_REASON Article discusses limitations of current AI architectures and potential future directions, rather than a specific release or event.

Read on Forbes — Innovation →

• retrieval-augmented generation
• State Space Models
• Superintelligence
• transformers
• Zuzanna Stamirowska

• model release
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →