Tiny Transformers
PulseAugur coverage of Tiny Transformers — every cluster mentioning Tiny Transformers across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Tiny Transformers achieve higher arithmetic accuracy with new training methods
Researchers have developed a method to improve arithmetic reasoning in small language models by using algorithmic scratchpads and a hierarchical curriculum. They found that proper data loading and linguistic pretraining…
-
Tiny Transformers Show Promise with 'Protoreasoning' for AI Analysis
Researchers have demonstrated that small transformer models can utilize a form of step-by-step reasoning, termed 'protoreasoning.' This technique allows for detailed experimentation and analysis of reasoning processes i…
-
Tiny Transformers: Input Pathways Crucial for Few-Shot Binding, Not Zero-Shot
A new study published on arXiv investigates how different input pathways affect the binding capabilities of small transformer models. The research found that while zero-shot composition performance is limited by inducti…