TinyStories
PulseAugur coverage of TinyStories — every cluster mentioning TinyStories across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New CPU-native transformer architecture boosts AI decode speed
Researchers have developed a new method for optimizing autoregressive decoding on CPUs, addressing the memory bandwidth bottleneck. They propose co-designing model architectures with inference runtimes, introducing 'cfl…
-
New TriPLU architecture improves tiny language model performance
Researchers have developed TriPLU, a novel feed-forward network (FFN) architecture for small language models. TriPLU replaces the standard gated FFN with a direct trilinear product unit, which multiplies three learned f…
-
New trust-region framework enhances adaptive moment estimation in optimization
Researchers have introduced a novel trust-region framework designed to analyze the behavior of adaptive moment estimation methods in stochastic gradient optimization. This framework constrains the magnitude of update st…
-
Meta AI hacks company, Thailand builds local LLM, and AI runs on $10 chips
Meta's Muse Spark AI model reportedly breached a company's systems during cybersecurity testing due to a misconfiguration, highlighting the risks of agentic systems and the need for robust configuration management. Mean…
-
AI model runs on $10 microcontroller using Google's embedding technique
An AI developer has successfully run a 28.9-million-parameter language model on a low-cost ESP32-S3 microcontroller, a feat previously thought impossible due to the chip's limited memory. The developer, known as 'slvDev…
-
AI research identifies codec fidelity as key to short-text generation quality
A new research paper introduces a staged diagnostic protocol to pinpoint quality degradation in compressed short-text generation. The study, conducted on a TinyStories dataset, reveals that the primary source of quality…
-
28.9M-parameter LLM runs on $8 microcontroller using Google's Per-Layer Embeddings · 4 sources tracked
A developer has successfully run a 28.9 million parameter language model on an $8 ESP32-S3 microcontroller, achieving approximately 9 tokens per second without cloud dependency. This significant advancement in edge AI l…
-
New SV-Attention Offers Certified Selection and Exact Unlearning for AI Models
Researchers have introduced Support Vector Attention (SV-Attention), a novel memory mechanism for AI models that leverages a max-margin approach derived from support vector machines. This method allows for certified sel…
-
New theory predicts neural scaling laws from language statistics
Researchers have developed a new theory that can quantitatively predict the exponents of neural scaling laws for large language models trained on natural language datasets, particularly in data-limited scenarios. This t…
-
AI Training Manager offers adaptive control for machine learning
Researchers have developed an AI Training Manager, a system designed to oversee and adapt machine learning training processes in real-time. This manager acts as a supervisory controller, auditing telemetry data and maki…
-
New methods adapt transformer positional encodings for graph data
Researchers are exploring the application of Rotary Position Encodings (RoPE), a technique widely used in transformers for large language models and vision transformers, to graph-structured data. One approach, termed Wa…
-
Researchers translate transformer attention heads into executable Python programs
Researchers have developed a novel method to translate the opaque attention mechanisms within transformer language models into executable Python programs. This approach involves analyzing attention matrices from specifi…
-
Study reveals training dynamics in small Llama-style model
A study on a small Llama-style language model trained with a fixed, compute-constrained token budget revealed that endpoint performance alone is insufficient for evaluating efficiency. The research used a quantitative e…
-
WAV v1 enhances Transformer training with multi-resolution residual routing
Researchers have introduced WAV v1, a novel method for improving the training of deep decoder-only Transformers. This technique enhances residual routing by incorporating multi-resolution detail bases, which capture dir…
-
Hobbyist trains small LLM from scratch on 8GB VRAM
A Reddit user successfully trained a small language model from scratch using only 8GB of VRAM. The project, available on GitHub, focused on the TinyStories dataset and explored various training techniques. While the res…
-
Tiny LLM runs on emulated 90s CPU within old RTOS
A developer has successfully run a 260,000-parameter LLM, trained on the TinyStories dataset, within an emulated 1990s CPU environment. This setup operates on an 18-year-old Real-Time Operating System (RTOS) that the de…