Int4
PulseAugur coverage of Int4 — every cluster mentioning Int4 across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Research: Compressing recursive reasoners for edge AI destroys global reasoning
A new research paper explores the challenges of compressing recursive reasoning models for deployment on edge hardware. The study found that standard compression techniques, such as INT4 pruning and distillation, preser…
-
Quantization of LLMs inflates reasoning token usage, researchers find
A new research paper highlights that while quantization techniques like INT4 and INT3 are effective at reducing the inference costs of large language models, they can unexpectedly inflate reasoning token usage. This phe…
-
Quantization causes 7-point task accuracy drop, bypassing perplexity
A company called Nexus Labs discovered that quantizing a fine-tuned 14B agent model to INT4 using GPTQ resulted in a significant 7-point drop in multi-step task completion accuracy, despite perplexity metrics showing on…
-
New UFP4 recipe tackles shrinkage bias in LLM FP4 pretraining
A new research paper introduces UFP4, a uniform 4-bit training recipe designed to address shrinkage bias in large language model pretraining. The study identifies that current non-uniform FP4 formats, like E2M1 used in …
-
LLM serving latency stems from system queues, not compute
This article discusses how to optimize Large Language Model (LLM) serving performance, emphasizing that latency issues are typically caused by system bottlenecks rather than model compute. It highlights that queueing, n…
-
New methods enhance LLM quantization for efficiency and accuracy
Researchers have developed several new methods to improve the efficiency and accuracy of quantizing large language models (LLMs). These techniques aim to reduce the memory footprint and computational cost of LLMs, makin…
-
The Measure of Deception: An Analysis of Data Forging in Machine Unlearning
Two new research papers explore vulnerabilities and detection methods in machine unlearning, a process designed to remove specific data from trained models for privacy compliance. One paper, "DurableUn," reveals that lo…
-
AI safety research proposes formal framework for computational substrates
This series of posts explores the concept of 'substrates' in AI, which refers to the computational context layers necessary for implementing AI systems. The authors argue that current AI safety research lacks a clear fr…