Brief

Researchers have developed a "Discrete Transformer" architecture designed to extract interpretable algorithms from trained models. This approach addresses the challenge of representation entanglement in standard Transformers, where overlapping features obscure symbolic expression recovery. By incorporating discreteness through temperature-annealed sampling, the Discrete Transformer facilitates the synthesis of human-readable programs, achieving performance comparable to existing methods on discrete tasks and extending extraction capabilities to tasks with continuous intermediate computations. The architecture also offers fine-grained control over synthesized programs, serving as a platform for algorithm extraction and Transformer interpretability research. AI

Researchers have introduced oMeBench, a new benchmark designed to evaluate the organic mechanism reasoning capabilities of large language models. The benchmark includes over 10,000 annotated mechanistic steps and a dynamic evaluation framework called oMeS for fine-grained scoring. Initial analysis reveals that while current LLMs show some chemical intuition, they struggle with consistent multi-step reasoning, though fine-tuning on the dataset significantly improved performance. AI