PulseAugur
实时 09:49:07

LLMs show categorical perception and optimized data selection

Researchers have developed a new framework for optimizing data selection in large language models, adapting data weighting to specific tasks and models using efficient proxies. Another study investigates categorical perception in LLM hidden states, finding geometric warping at digit-count boundaries across various model families. This warping effect, termed "structural CP," appears to be an architectural property independent of explicit semantic knowledge. AI

影响 These studies offer insights into improving LLM training efficiency and understanding their internal representations, potentially leading to more capable and robust models.

排序理由 The cluster contains two academic papers detailing novel research findings in LLM behavior and optimization.

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

LLMs show categorical perception and optimized data selection

报道来源 [2]

  1. arXiv cs.CL TIER_1 English(EN) · Zibin Zheng ·

    Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies

    Data selection is a key component of efficient instruction tuning for large language models, as recent work has shown that data quality often matters more than data quantity. Accordingly, prior studies have introduced various multi-dimensional heuristics to evaluate and filter in…

  2. arXiv cs.CL TIER_1 English(EN) · Jon-Paul Cacioli ·

    Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries

    arXiv:2603.28258v2 Announce Type: replace Abstract: Categorical perception (CP) -- enhanced discriminability at category boundaries -- is among the most studied phenomena in perceptual psychology. This paper reports that analogous geometric warping occurs in the hidden-state repr…