BERT-tiny
PulseAugur coverage of BERT-tiny — every cluster mentioning BERT-tiny across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
0.1-bit AI models show task-dependent performance limits
Researchers attempted to create a 0.1-bit model, discovering that extreme quantization alone is insufficient for maintaining performance. While a 340 KB distilled model could match a larger teacher on a keyword task, pe…
-
New learned optimizer improves training for BERT-Tiny, GPT-Tiny, and Vision Transformers
Researchers have developed a novel learned optimizer that dynamically recombines gradient history to improve model training efficiency and performance. This optimizer, with only 37,000 parameters, achieved significant i…
-
AI-text detection models achieve strong out-of-distribution performance · 2 sources tracked
A research paper details a method for detecting AI-generated text by fine-tuning BERT-tiny models with Bayesian classification heads and selecting texts from multiple datasets. This approach, tested on the PAN 2026 data…