LLM research probes faithfulness, sycophancy, and knowledge updates · 10 sources tracked
ByPulseAugur Editorial·[174 sources]·
Recent research explores various facets of large language model (LLM) behavior and evaluation. One study introduces Normalized Simulatability Gain (NSG) to measure the faithfulness of LLM self-explanations, finding they significantly improve prediction of model behavior across various models like Gemini-3.*, GPT-5.2, and claude-4.5. Another paper investigates "sycophancy" in LLMs, revealing that task difficulty and model reasoning capabilities are more influential than pressure tactics in causing models to abandon correct answers. Further research delves into session-level contamination in LLMs, proposing protocols to assess how false premises are adopted, and introduces Conditional Accuracy Profiling (CAP) to diagnose LLM judge performance under different conditions. Additionally, new methods like EngramEdit aim to enable decoupled knowledge updates in LLMs, while others focus on improving process predictability through step-by-step execution and disentangling model effects from personas in multi-agent simulations. Finally, one study suggests that activation steering for debiasing LLMs may primarily reduce model confidence rather than address underlying fairness issues.
AI
IMPACT
These studies offer new tools and insights for evaluating LLM behavior, improving their reliability, and enabling more effective knowledge updates, crucial for advancing AI development and deployment.
RANK_REASON
The cluster consists of multiple academic papers published on arXiv, detailing novel research methodologies and findings in LLM behavior and evaluation.
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
The cluster consists of multiple academic papers published on arXiv, detailing novel research methodologies and findings in LLM behavior and evaluation.
Source corroboration
174 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
53 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+74 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.
arXiv:2610.23362v1 Announce Type: new Abstract: Modern LLM applications rely on runtime monitors that inspect tool calls, traces, streams, detector outputs, and verifier inputs. This creates an interface-reliability problem: a framework path may remain operational while policy-re…
arXiv cs.CL
TIER_1English(EN)·Yinuo Xu, Jadie Adams, Brian Hu, Shivani Kumar, David Jurgens, Arslan Basharat·
arXiv:2610.23702v1 Announce Type: new Abstract: Personalizing a language model requires knowing what a person values, which stays largely consistent across tasks. Yet most methods either learn task-specific preferences or place the user's profile in the prompt, where models strug…
arXiv:2610.22874v1 Announce Type: new Abstract: LLMs over external data face a per-instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Ex…
arXiv:2505.10597v3 Announce Type: replace-cross Abstract: Reward models (RMs) are central to aligning large language models (LLMs) with human values, yet noisy preference feedback can induce reward misgeneralization through spurious correlations and overfitting. We characterize t…
arXiv:2610.21214v1 Announce Type: new Abstract: Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM verifier. However, judgment certainty varies substant…
arXiv:2610.22139v1 Announce Type: new Abstract: Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible through the score itself. We introduce RADAR (Repair, Adopt, and Diagnose After Recali…
arXiv:2610.21412v1 Announce Type: new Abstract: Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow and costly. Most of these calls …
arXiv cs.CL
TIER_1English(EN)·Sara Rezaeimanesh, Chad Klochko, Kundan Thind, Mohammad Ghassemi·
arXiv:2610.21038v1 Announce Type: cross Abstract: Machine learning models for tabular prediction typically assume that training data are representative of the population on which they will be deployed. In practice, however, data are often collected based on convenience, availabil…
arXiv cs.AI
TIER_1English(EN)·Alexey A. Bokov, Ivan I. Panin, Nikolay B. Nechaev, Mikhail Yu. Valkov, Valeria Yu. Chernina, Natalia Gombolevskaya, Ayyaz Hussain, Vladimir M. Agafonov, Victor A. Gombolevskiy, Aleksandr V. Nesterov, Iaroslav R. Bespalov, Ivan V. Oseledets, Dmitry V. Dy…·
arXiv:2610.22298v1 Announce Type: cross Abstract: Large language models (LLMs) can generate differential diagnoses for complex cases, but a useful second opinion must also be factually grounded, clinically coherent, actionable, and safe. We developed a multidimensional benchmark …
arXiv:2610.21731v1 Announce Type: cross Abstract: Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available. To study this problem, we construct temporal facts whose values …
arXiv cs.AI
TIER_1English(EN)·Jianchang Su, Dawei Xiang, Yifan Zhang, Wei Zhang·
arXiv:2610.21135v1 Announce Type: cross Abstract: LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a …
Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available. To study this problem, we construct temporal facts whose values changed after the models' training cutoffs. We eva…
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow and costly. Most of these calls end in a multiple-choice decision, such as which…
arXiv cs.AI
TIER_1English(EN)·Jianlin Chen, Wenhui Chen, Chi Man Vong·
arXiv:2610.15738v1 Announce Type: new Abstract: LLM optimization loops often show the proposer the candidates it has already tried and rejected, but leave it to the proposer to avoid them. We ask whether avoidance measured in a static probe predicts avoidance in the states an opt…
arXiv:2610.18126v1 Announce Type: new Abstract: Freshness shapes an LLM decision at the moment it is made; the action it commits to is latent, carried out later, in a changing world. We therefore introduce a timestamp theory of decision validity and a decision ledger that realise…
arXiv:2610.17473v1 Announce Type: new Abstract: Generative safety guards must classify requests correctly and return outputs that applications can parse. Reasoning guards can improve classification accuracy while producing outputs that applications cannot parse. When applications…
arXiv:2610.14970v1 Announce Type: new Abstract: With the rapid evolution of large language models (LLMs), there is an increasing demand for effective personalization techniques that can adapt model behavior to individual user preferences. Personalized LLMs must infer a user's que…
arXiv cs.LG
TIER_1English(EN)·Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal, Joseph E. Gonzalez, Matei Zaharia, Kurt Keutzer, Inderjit S Dhillon, Rishabh Agarwal, Devvrit Khatri·
arXiv:2605.12484v3 Announce Type: replace Abstract: Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating model parameters forces them to absorb task-specific information, which can result in catastrophic forg…
arXiv:2510.13928v3 Announce Type: replace Abstract: We propose and test the LLM Brain Rot Hypothesis: continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs). To unveil junk effects, we designed a novel controlled experiment on real T…
arXiv:2508.06149v2 Announce Type: replace Abstract: We present Big5-Scaler, a prompt-based framework for conditioning large language models (LLMs) with controllable Big Five personality traits. By embedding numeric trait values into natural language prompts, our method enables fi…
arXiv cs.CL
TIER_1English(EN)·Minsuh Joo, Youna Kim, Choonghyun Park, Soo Kyung Kim, Hyunsoo Cho·
arXiv:2610.19827v1 Announce Type: cross Abstract: LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques …
arXiv cs.CL
TIER_1English(EN)·Minchan Kim, Sungho Bae, Youngjin Kim, Hyunwoo Park·
arXiv:2610.19349v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) provides no reward-relative learning signal when all rollouts receive identical rewards. In long-horizon tasks that require multiple interactions, rewards are given only upon successful ta…
arXiv:2610.20570v1 Announce Type: new Abstract: Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manac\'a-1B-Instruct-2,…
arXiv:2610.20067v1 Announce Type: new Abstract: Large Language Models (LLMs) frequently encounter conflicts between internal parametric knowledge and external contextual knowledge, yet the mechanisms governing this selection remain unclear. We show that the choice between context…
arXiv:2610.19915v1 Announce Type: new Abstract: Rubric-based evaluation assesses the responses of large language models (LLMs) by aggregating the verification results of explicit criteria, yet not all criteria matter equally. However, existing methods commonly aggregate the verif…
arXiv:2511.00198v2 Announce Type: replace-cross Abstract: Autoregressive language models are usually trained with next-token prediction under a fixed left-to-right target order. This paper studies target-token organization as a training-time design choice across arithmetic reason…
arXiv cs.AI
TIER_1English(EN)·Durga Prasad Maram, Dhruvin Gandhi, Zonghai Yao, Gayathri Akkinapalli, Franck Dernoncourt, Yu Wang, Ryan A. Rossi, Nesreen K. Ahmed·
arXiv:2510.24469v2 Announce Type: replace-cross Abstract: Personalized text generation requires models not only to produce coherent text but also to align with a target user's style, tone, and topical focus. Existing retrieval-augmented approaches such as LaMP and PGraphRAG enric…
arXiv cs.AI
TIER_1English(EN)·Jasper Dekoninck, Maximilian Baader, Martin Vechev·
arXiv:2409.00696v4 Announce Type: replace-cross Abstract: Rating-based human evaluation has become an essential tool to accurately evaluate the impressive performance of large language models (LLMs). However, current rating systems suffer from several important limitations: first…
arXiv:2609.32817v2 Announce Type: replace Abstract: Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names the concepts that actually drove the decision. We introduce three lightweight, d…
arXiv cs.AI
TIER_1English(EN)·Yifan He, Manon Reusens, Sofie Goethals, David Martens·
arXiv:2610.18515v2 Announce Type: cross Abstract: Large language models (LLMs) increasingly act as personal assistants; however, little is known about how they capture national-level preferences on a global scale. Given that "time" is a universal and representative concept, we ex…
arXiv cs.AI
TIER_1English(EN)·Pratuat Amatya, Vinay Setty, V Venktesh, Avishek Anand·
arXiv:2610.18415v1 Announce Type: cross Abstract: Large Language Models (LLMs) are aligned to human preferences, after pre-training and supervised fine-tuning steps. However, standard RL alignment steps such as RLHF do not focus on factuality of LLMs, hence they may generate non-…
arXiv cs.AI
TIER_1English(EN)·Cesare Barbera, Luca Stradiotti, Lorenzo Cascioli, Lorenzo Perini, Niek Tax, Daniel Haimovich, Jesse Davis, Andrea Passerini, Andrea Pugnana·
arXiv:2610.18108v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to predict which of two candidate answers a human evaluator would prefer, providing a scalable alternative when human evaluation is too costly. Unfortunately, LLM judgme…
arXiv cs.AI
TIER_1English(EN)·Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado·
arXiv:2610.18051v1 Announce Type: cross Abstract: This work investigates the application of the Membership Inference Test (MINT) to large language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about…
arXiv cs.AI
TIER_1English(EN)·Vasileios Saketos, William Reveillard, Alexandre Proutiere, Sebastian Dalleiger, Ming Xiao·
arXiv:2610.15128v1 Announce Type: cross Abstract: Modern AI systems increasingly combine large language models (LLMs) with human experts, using the LLM to handle routine queries while deferring uncertain cases for expert review. When expert capacity is limited, however, deferral …
arXiv:2610.14709v1 Announce Type: cross Abstract: Large language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommendation and advertising systems. These applications demand the user dimension of L…
LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity …
A harness that delegates agent tool choices to small decision models via fitted confidence thresholds reduces cost and latency while preserving task success, supported by a new benchmark and a 4B model trained on agent error states.
LLM-generated crisis reports match human performance on some metrics but fail to supply professional provenance and reproduce few official figures, while common automatic scores poorly reflect completeness.
Large Language Models (LLMs) are aligned to human preferences, after pre-training and supervised fine-tuning steps. However, standard RL alignment steps such as RLHF do not focus on factuality of LLMs, hence they may generate non-factual information, which could result in uninten…
Freshness shapes an LLM decision at the moment it is made; the action it commits to is latent, carried out later, in a changing world. We therefore introduce a timestamp theory of decision validity and a decision ledger that realises it. Six timestamps define a decision; a freshn…
arXiv cs.CL
TIER_1English(EN)·Feifei Qian, Xuan Chen, Lixin Cui, Chaoyang Mei, Fanlin Meng, Shuyao Xiao, Ming Cui, Junxi Yin, Lu Bai·
arXiv:2610.13375v1 Announce Type: new Abstract: Rubrics have been widely used to evaluate and optimize language models on open-ended tasks. However, existing work primarily focuses on generating better rubrics, with limited understanding of what makes rubric design effective. In …
arXiv cs.AI
TIER_1English(EN)·Jiwon Jeon, Myungsik Cho, Jeonghye Kim, Youngchul Sung·
arXiv:2610.13672v1 Announce Type: cross Abstract: Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed r…
Large language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommendation and advertising systems. These applications demand the user dimension of LLM intelligence: inferring a person's preferences,…
arXiv:2610.12022v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in sociotechnical systems where social attribution, the reasoning process attributing external events to the causes and reasons of agents' social behaviors, plays a critical r…
arXiv cs.AI
TIER_1English(EN)·Andrea Wynn (Nathan), Harsh Satija (Nathan), Seokhyun (Nathan), Baek, Anqi Liu, Eric Nalisnick, Gillian K. Hadfield·
arXiv:2610.10906v1 Announce Type: new Abstract: Human communities are governed by normative systems: shared standards that produce \textit{norms} dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is…
arXiv:2610.12304v1 Announce Type: new Abstract: Understanding large language model (LLM) reasoning requires looking beyond behavioral performance to examine how reasoning ability is reflected in internal organization. Inspired by neuroscience findings linking higher intelligence …
arXiv:2610.11766v1 Announce Type: cross Abstract: Safety evaluations of large language models commonly summarize harmful-output behavior with attack success rate (ASR). Yet the same non-harmful outcome can arise for very different reasons. A model may recover a harmful task and r…
arXiv:2609.32382v2 Announce Type: replace Abstract: We present our system for the PlurVA-LLM 2026 Shared Task Track-1, which focuses on pluralistic value alignment in the contexts of China, Indonesia, and Sri Lanka. For this resource-constrained track, we fine-tuned Llama 3.1 8B …
arXiv:2610.11978v1 Announce Type: new Abstract: Jev is a "System One" model that returns a choice among given options instead of generating text. We study how such a specialized decision model compares with general-purpose large language models (LLMs). We evaluate Jev on 13 multi…
arXiv:2508.08833v4 Announce Type: replace-cross Abstract: Frontier large language models (LLMs) now reach near-ceiling accuracy on standard mathematical-reasoning benchmarks and gold-medal-level performance at the International Mathematical Olympiad. As these benchmarks saturate …
In this paper, we provide an overview of the NTCIR-19 Automatic Evaluation of LLMs 2 (AEOLLM-2) task. Building on the success of the NTCIR-18 core task AEOLLM, we proposed AEOLLM-2 for NTCIR-19 to further investigate automatic evaluation methods for Large Language Models (LLMs), …
arXiv:2609.35308v2 Announce Type: replace Abstract: Large language models treat conversation history as unverified context, so false premises injected into prior turns can be adopted as fact, a failure mode we term session-level contamination. We introduce five contamination prot…
arXiv:2610.09229v1 Announce Type: new Abstract: LLM-as-judge is now a standard tool for scalable evaluation, but judge performance is still often summarized by a single accuracy number. This aggregate view hides the deployment conditions under which a judge succeeds or fails. We …
arXiv cs.CL
TIER_1English(EN)·Xiaoshu Chen, Sihang Zhou, Ke Liang, Xinwang Liu·
arXiv:2609.33181v2 Announce Type: replace-cross Abstract: Recent advances in online policy self-distillation (OPSD) have demonstrated that large language models (LLMs) can improve their capabilities by leveraging external privileged information (PI), such as manual annotations or…
arXiv cs.CL
TIER_1English(EN)·Hongru Cai, Ran Wei, Wenjie Wang, Chengfa Wu, Ning Song, Yongqi Li, Wenjie Li·
arXiv:2610.10533v1 Announce Type: new Abstract: Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this archite…
arXiv cs.CL
TIER_1English(EN)·Guang Yang, Homa Hosseinmardi, Fengchen Liu, Amir Ghasemian·
arXiv:2610.08840v1 Announce Type: new Abstract: Large language models (LLMs) often abandon a correct answer, or endorse a user's position, once the user pushes back. This behavior, called sycophancy, is usually reported as a single rate per model, which says little about when it …
arXiv cs.LG
TIER_1English(EN)·Harry Mayne, Justin Singh Kang, Dewi Gould, Kannan Ramchandran, Adam Mahdi, Noah Y. Siegel·
arXiv:2602.02639v2 Announce Type: replace-cross Abstract: LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood. Existing faithfulness metrics have critical limitations, ty…
Human communities are governed by normative systems: shared standards that produce \textit{norms} dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by t…
arXiv cs.MA (Multiagent)
TIER_1English(EN)·Gillian K. Hadfield·
Human communities are governed by normative systems: shared standards that produce \textit{norms} dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by t…
arXiv cs.MA (Multiagent)
TIER_1English(EN)·Gillian K. Hadfield·
Human communities are governed by normative systems: shared standards that produce \textit{norms} dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by t…
arXiv cs.CL
TIER_1English(EN)·Sofia Torres, Gabriel Almeida, Carter Adams, Camila Rocha·
arXiv:2610.06861v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become the dominant paradigm for eliciting multi-step reasoning in large language models, and a recent wave of methods (LUFFY, ExPO, PAPO, TAPO) further augments RL with \e…
arXiv cs.CL
TIER_1English(EN)·Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Borui Jiang, Bo Han·
arXiv:2610.07894v1 Announce Type: new Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is insufficient to answer the questions. Existing faithfulness evaluations typically ass…
arXiv:2610.08018v1 Announce Type: new Abstract: Reliable tool use requires more than triggering a mechanism or matching a query to an API description. Before selecting a specific tool, an agent must first infer the capability requirements implied by the user query. In this paper,…
arXiv cs.CL
TIER_1English(EN)·Pierre-Carl Langlais, Pavel Chizhov, Catherine Arnett, Carlos Rosas-Hinostroza, Mattia Nee, Eliot Krzystof Jones, Ir\`ene Girard, David Mach, Anastasia Stasenko, Ivan P. Yamshchikov·
arXiv:2506.01732v4 Announce Type: replace Abstract: Large Language Models (LLMs) are pre-trained on large amounts of data from different sources and domains. Such datasets often contain trillions of tokens, including large portions of copyrighted or proprietary content, which rai…
arXiv:2610.08129v1 Announce Type: new Abstract: Cooperation with unfamiliar partners requires adapting to communication conventions that are not known in advance. We study this problem in a controlled Hanabi-derived environment with scripted hint generation, LLM-controlled receiv…
arXiv cs.LG
TIER_1English(EN)·Chen Chen, Dongjie Wang, Mei Liu, Zijun Yao·
arXiv:2610.07739v1 Announce Type: new Abstract: Post-discharge risk prediction from electronic health records (EHRs) is difficult because many dependencies that link discharge-time observations to downstream complications, such as comorbidity cascades and drug-disease interaction…
arXiv cs.AI
TIER_1English(EN)·Bach Nguyen, Zhaonan Li, Mau Son Nguyen, Sanika Chavan, Nilay Kumar, Hong Anh Nguyen, Khoa Vo, Ben Zhou·
arXiv:2610.07751v1 Announce Type: new Abstract: Real-world reasoning rarely reduces to static question answering: agents must actively gather information from tools and sensors that are often noisy and unreliable. Yet most existing active reasoning benchmarks assume that environm…
arXiv cs.AI
TIER_1English(EN)·Maoqi Liu, Quan Fang, Yufei He·
arXiv:2610.08312v1 Announce Type: new Abstract: Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-L…
arXiv:2610.08563v1 Announce Type: new Abstract: Sequence-level power sampling has recently emerged as a training-free approach to reasoning by sampling from a sharpened output distribution of a base large language model (LLM). Nevertheless, existing methods typically sharpen the …
arXiv:2610.08778v1 Announce Type: new Abstract: Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach it. Existing approaches to training LLMs as teachers rely on demonstrations, pre…
arXiv cs.AI
TIER_1English(EN)·Hashmath Shaik, Gnaneswar Villuri, Alex Doboli·
arXiv:2610.07115v1 Announce Type: cross Abstract: The order in which candidate responses are presented can change an LLM judge's verdict. Detecting such a position flip ordinarily requires judging each pair in both orders, which doubles the number of judgments. We investigate whe…
arXiv cs.AI
TIER_1English(EN)·Wonjun Lee, Kyungsik Yang, Gaeun Ji, Vaidehi Patil, Haon Park, Bumsub Ham, Mohit Bansal, Suhyun Kim·
arXiv:2610.07532v1 Announce Type: cross Abstract: LLMs have advanced rapidly, raising growing concerns about their safety. Recent work has proposed approaches to detect and defend against attacks including defenses at decoding stage that leverage models' hidden states. However, e…
arXiv:2610.07535v1 Announce Type: cross Abstract: Multi-agent simulations with large language models (LLMs) often operate networks of agents with a single base model. This overlooks the inter-model effects which may dominate engagement dynamics in real-world deployments. To show …
arXiv cs.AI
TIER_1English(EN)·Hans Schabert, Christoph Peters·
arXiv:2610.07817v1 Announce Type: cross Abstract: Organizations automating operational processes need more than a correct outcome: they need to predict how a process will run, know which one actually ran, and inspect it step by step. When an agent is the executor that predictabil…
arXiv cs.AI
TIER_1Français(FR)·Stephanie Buttigieg, Maeve Madigan, Parameswaran Kamalaruban, Stuart Burrell·
arXiv:2610.08559v1 Announce Type: cross Abstract: Activation steering has gained popularity as a lightweight inference-time debiasing technique for large language models. However, prior work reports that steering vectors generalise poorly, with unintended effects on model perform…
arXiv cs.CL
TIER_1English(EN)·Yiqi Liu, Joseph James, Yang Wang, Kun Zhao, Chenghao Xiao, Chenghua Lin·
arXiv:2610.07109v1 Announce Type: new Abstract: When an LLM judge scores an output, its score distribution retains uncertainty and disagreement information that is lost after scalar compression. We introduce JudgeMoE, a lightweight aggregator that assigns example-specific weights…
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple…
Large language models can extract useful signals from heterogeneous enterprise data, but high-recall extraction often produces outputs that are duplicated, uneven in granularity, semantically overlapping, or too numerous for downstream systems and human reviewers to use effective…
Multi-agent simulations with large language models (LLMs) often operate networks of agents with a single base model. This overlooks the inter-model effects which may dominate engagement dynamics in real-world deployments. To show this, we simulate a heterogeneous social network p…
Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach it. Existing approaches to training LLMs as teachers rely on demonstrations, preference data, or predefined pedagogical criteria…
arXiv:2605.10247v2 Announce Type: replace Abstract: Applying Large Language Models (LLMs) to graph-structured data usually involves multi-step pipelines in which textual node attributes are compressed into single tokens and further processed by GNNs, discarding most of their sema…
arXiv:2610.03080v1 Announce Type: cross Abstract: Large language models are moving from producing trading signals to writing the code that executes them. The failure mode of the second role is silent: generated code runs, a backtest plots, yet the risk logic that the trader descr…
arXiv:2604.14128v3 Announce Type: replace-cross Abstract: Rhetorical questions are asked not to seek information but to persuade or signal stance. How large language models internally represent them remains unclear. We analyze rhetorical questions in LLM representations using lin…
arXiv:2610.02793v1 Announce Type: new Abstract: Research on LLMs continually uncovers model limitations, their causes, and potential solutions. Yet these human discoveries remain largely disconnected from model evolution: an LLM does not automatically learn from new research abou…
arXiv cs.AI
TIER_1English(EN)·Seoyeon Ye, Gayoung Kim, Jiyoung Hong, Sookyung Kim, Hyunsoo Cho·
arXiv:2610.02902v1 Announce Type: new Abstract: Current analyses of LLMs' parametric knowledge are largely output-centric, drawing conclusions about what a model knows without verifying what it was actually trained on. This leaves fundamental questions, such as whether a correct …
arXiv:2610.02571v1 Announce Type: cross Abstract: As AI-assisted programming becomes increasingly mainstream, the environmental impact of AI-generated software has emerged as an important consideration. This motivates evaluating LLM-generated code beyond functional correctness by…
arXiv cs.AI
TIER_1English(EN)·Katherine Van Koevering, Jon Kleinberg·
arXiv:2406.00092v2 Announce Type: replace Abstract: Randomness is central to human cognition and to many applications in which large language models are deployed, yet probabilistic token generation does not imply that LLMs can produce unbiased random sequences. We study how conte…
arXiv:2603.29112v2 Announce Type: replace Abstract: We introduce GISTBench, a benchmark for evaluating Large Language Models' (LLMs) ability to understand users from their interaction histories in recommendation systems. Unlike traditional RecSys benchmarks that focus on item pre…
arXiv:2602.10273v3 Announce Type: replace-cross Abstract: Reasoning ability in large language models is often attributed to \emph{distribution sharpening}: concentrating output probability on high-likelihood sequences. Recent works show that this sharpening effect can be obtained…
arXiv:2603.13725v2 Announce Type: replace Abstract: Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy efficiency and computational density. However, the…
Jev-style decision models return categorical probability distributions over predefined options without generating free-form text, enabling software systems to act on their outputs directly. In this work, we investigate the extent to which general-purpose LLMs already possess this…
Large language models (LLMs) are increasingly used as interactive agents and simulators, yet it remains unclear whether they can recover latent sequential structure beyond surface action frequencies. This distinction is critical for behavioral simulation, where actions are often …
arXiv:2610.00833v1 Announce Type: cross Abstract: Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, entities, numbers, and claim types. It checks a fixed schema; it does not verify e…
arXiv cs.AI
TIER_1English(EN)·Hongbo Chen, Guohua Lu, Ting Dang, Hong Jia·
arXiv:2609.33149v2 Announce Type: replace Abstract: A common principle of effective learning is to practice material that is neither already mastered nor too difficult to permit progress. We ask how to apply this principle to structured reasoning tasks such as Sudoku and maze sol…
arXiv cs.AI
TIER_1English(EN)·Khawaja Murad ul Hassan, Mehran Ebrahimi·
arXiv:2610.00047v1 Announce Type: new Abstract: Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM) prunes a chain, its high-PRM prefix is extracted and grafted verbatim as an in-c…
arXiv:2610.00331v1 Announce Type: new Abstract: When selecting mathematical training data for LLMs, a natural organizing principle is topic: probability examples for probability targets. An alternative is reasoning approach: worked solutions that share a solution method with the …
arXiv cs.AI
TIER_1English(EN)·Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer, Thomas A. Runkler·
arXiv:2610.00682v1 Announce Type: new Abstract: Large language models (LLMs) increasingly underpin scientific AI applications that reason over structured knowledge, from biomedical question answering to materials informatics. However, their logical reasoning often falls short, pr…
arXiv:2610.00296v1 Announce Type: cross Abstract: Token-level certainty is widely used as a proxy for correctness in LLM training and inference. However, the performance of certainty-based methods depends both on the information in certainty scores and on how those scores are use…
arXiv cs.AI
TIER_1English(EN)·Matthieu Zimmer, Xiaotong Ji, Tu Nguyen, Haitham Bou-Ammar·
arXiv:2610.00332v1 Announce Type: cross Abstract: Distilling the reasoning capabilities of large language models (LLMs) into smaller students is a central challenge for efficient deployment. Current approaches face a fundamental tension: optimizing purely for verifiable task rewa…
arXiv:2610.00562v1 Announce Type: cross Abstract: Longitudinal clinical reasoning requires large language models (LLMs) to identify and integrate relevant evidence distributed across extended patient histories. Although long-context models can process increasingly large amounts o…
arXiv cs.AI
TIER_1English(EN)·Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi·
arXiv:2610.01428v1 Announce Type: cross Abstract: Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed in different ways. Existing work typically evaluates generalization through aggre…
arXiv:2610.01471v1 Announce Type: cross Abstract: Large language models now generate code, documentation, and analyses, and are increasingly used to review such output. We ask when a second review by a different model helps. Building on the author's earlier preprints, which varie…
arXiv:2610.01847v1 Announce Type: cross Abstract: Model specifications define how large language models (LLMs) should behave, guiding alignment training, inference-time behavior, and evaluation. Yet these specifications may themselves contain defects: two individually reasonable …
arXiv:2602.03006v3 Announce Type: replace Abstract: Deploying Large Language Models (LLMs) for discriminative workloads is often limited by inference latency, compute, and API costs at scale. Active distillation reduces these costs by querying an LLM oracle to train small discrim…
arXiv:2608.28421v2 Announce Type: replace Abstract: Post training a language model to reason means updating its weights. Supervised finetuning and reinforcement learning both place the acquired capability inside the model where it cannot be inspected cannot be checked step by ste…
arXiv:2609.39801v1 Announce Type: new Abstract: Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LLMs). However, recent studies reveal that when applied to reasoning models, PTQ n…
arXiv:2609.38829v1 Announce Type: new Abstract: Multi-path reasoning methods such as self-consistency (SC) sample $K$ reasoning paths and choose the most frequent answer. However, their gains quickly plateau as $K$ increases, and existing methods do not predict when this saturati…
arXiv:2609.38260v1 Announce Type: cross Abstract: Values such as honesty, autonomy, and confidentiality are often regarded as general principles underpinning AI alignment. However, what it means to act in accordance with these values can depend on the context in which a decision …
arXiv:2609.39346v1 Announce Type: new Abstract: Large language models (LLMs) offer strong reasoning capabilities but are often costly to access through commercial APIs, while small language models (SLMs) are easier to deploy locally yet remain weaker in reasoning. This capability…
arXiv:2609.39967v1 Announce Type: cross Abstract: Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are …
arXiv:2609.38972v1 Announce Type: cross Abstract: Chain-of-thought (CoT) traces often serve as a proxy for how Large Language Models (LLMs) arrive at their answers. However, growing evidence shows that models' CoT often fails to reflect their internal computations and can be chan…
arXiv:2609.38672v1 Announce Type: cross Abstract: Beam-search-based test-time methods provide an effective way to improve large language model (LLM) performance on long-horizon generation by pruning invalid reasoning paths early, leading to significantly improved reasoning effici…
arXiv:2609.39572v1 Announce Type: new Abstract: We analyze how lexical ambiguity and underspecification affect language model training. We create artificial homonyms and artificial hypernyms as pseudowords and analyze the generative performance of language models as they are trai…
Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed in different ways. Existing work typically evaluates generalization through aggregate accuracy on a single prompt format, task, or …
Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are natural candidates for tools that an LLM can call …
arXiv:2609.37054v1 Announce Type: new Abstract: Ensuring the safety of reasoning large language models (LLMs) across languages is essential for their reliable deployment. However, when exposed to jailbreak attacks in non-high-resource languages, these models may generate unsafe r…
arXiv:2609.38070v1 Announce Type: new Abstract: As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming increasingly important for quantifying the uncertainty of their answers. Current meth…
arXiv cs.AI
TIER_1English(EN)·Gurbir Arora, Toni J. B. Liu, Jiajun Bao, Rapha\"el Sarfati, Christopher J. Earls·
arXiv:2609.35970v1 Announce Type: cross Abstract: Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We s…
arXiv cs.AI
TIER_1English(EN)·Ali Mohammadi Esfahani, Nafiseh Kahani, Samuel A. Ajila·
arXiv:2609.37405v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks that require understanding existing source code, including behavior prediction, function explanation, debugging, and code review. However, aggregate…
arXiv cs.CL
TIER_1English(EN)·Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter·
arXiv:2609.35824v1 Announce Type: new Abstract: Large language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets lab…
arXiv cs.CL
TIER_1English(EN)·Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko·
arXiv:2609.37891v1 Announce Type: new Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--for instance, they contain little explicit reasoning. Thus, many frontier labs have…
arXiv:2609.38137v1 Announce Type: new Abstract: Language-model (LM) harnesses enable LMs to operate effectively over long contexts using additional compute. However, existing long-context evaluations are insufficient for distinguishing modern harnesses, reflected by saturated acc…
arXiv:2609.37915v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies this supervision to unverified student rollouts while conditioning the teacher …
arXiv:2602.11328v2 Announce Type: replace Abstract: As people turn to LLMs for social advice, understanding their behavior in such contexts becomes essential. In this work, we focus on behavioral dispositions: the underlying tendencies that shape responses in social contexts. We …
arXiv cs.LG
TIER_1English(EN)·Shenghong Dai, Shiva Kumar Pentyala, Yingchi Liu, Shubham Mehrotra, Suman Banerjee, James Zhu, Bin Bi, Sitaram Asur, Phil Mui·
arXiv:2609.36115v1 Announce Type: new Abstract: Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-critical applications. Existing prompting and constrained decoding produce verbose, mu…
arXiv cs.LG
TIER_1English(EN)·Qihao Wen, Jiahao Wang, Yang Nan, Pengfei He, Ravi Tandon, Han Xu·
arXiv:2602.02427v3 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved significant breakthroughs across various domains, but they can still produce unreliable or misleading outputs. For responsible LLM applications, uncertainty quantification techniques ar…
arXiv:2609.36159v1 Announce Type: new Abstract: Large language models can reason in continuous space instead of decoded text, by recurring on their own hidden states or by passing those states between agents, while training supervises only the Cross-Entropy (CE) of the final deco…
Chain-of-thought (CoT) traces often serve as a proxy for how Large Language Models (LLMs) arrive at their answers. However, growing evidence shows that models' CoT often fails to reflect their internal computations and can be changed without affecting their final answers. In this…
Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work together on complex problems. However, self-improvement methods typically optimize one system at a time, and multi-agent framewor…
As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming increasingly important for quantifying the uncertainty of their answers. Current methods for estimating the confidence of large langu…
On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies this supervision to unverified student rollouts while conditioning the teacher on privileged context, typically a reference solut…
We study reasoning in Rule-Governed Decision Tasks (RGDTs), where models apply external rules to case facts and justify decisions, as required in policy, contract, and compliance settings. Beyond the deductive capability emphasized by standard mathematical and logical reasoning t…
Large language models can reason in continuous space instead of decoded text, by recurring on their own hidden states or by passing those states between agents, while training supervises only the Cross-Entropy (CE) of the final decoded answer and does not constrain the thought. T…
We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization. Building on this connection, we introduce Least-Square Policy Distillation (LSPD), an RL-ins…
arXiv stat.ML
TIER_1English(EN)·Yang Ba, Michelle V. Mancenido, Rong Pan·
arXiv:2610.14029v1 Announce Type: cross Abstract: LLM-as-a-judge evaluation commonly aggregates multiple judges' decisions into a majority vote, yet this panel decision process does not guarantee that the evaluation of an individual item is reliable. We propose Structured Cross-f…
arXiv:2610.00809v1 Announce Type: new Abstract: Vision-Language Models (VLMs) enable video annotation at scale, but costs accumulate quickly: processing a typical 60-second short-form video at one frame per second requires millions of tokens. To reduce costs, researchers rely on …
<h3>Build a Jev-Style Decision Model from an LLM, Part 4: Letting the Backbone Learn, Then Giving It Eyes</h3><h4><em>Letting the backbone learn with LoRA and then giving the same scoring head images to read.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1…
<h4><em>How four vectors become probabilities: the pointer head that scores the options, how it is trained and how far a frozen backbone can take it.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6Oa07qLP3X1wRCu3" /><figcaption>Photo by <a href="htt…
<h4><em>What is inside a hybrid vision-language model, what that rules out and how to feed it a decision.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lkqz3tuS_wHTFiWZ" /><figcaption>Photo by <a href="https://unsplash.com/@timmossholder?utm_source=…
Medium — fine-tuning tag
TIER_1English(EN)·Darsh Mamtora·
<p>Most engineers treating LLMs as reasoning engines fall into a predictable trap: they ask the model to perform complex arithmetic or heavy-duty decision logic internally. While GPT-4o or Claude 3.5 Sonnet are excellent at structured thinking, relying on their latent ability to …
Medium — MLOps tag
TIER_1English(EN)·Erwin Hermanto·
<p>Most AI-generated strategies suffer from the same terminal flaw: they are forward-only. If you ask an LLM to propose a market entry plan or a scaling architecture, it will present a linear progression of successful steps. It describes the ascent, but it never maps the cliffs.<…
dev.to — MCP tag
TIER_1English(EN)·Renato Marinho·
<p>Large Language Models are notoriously bad at arithmetic. When you ask a model to interpret complex financial oscillators, it isn't performing calculus; it is predicting the most likely next token based on training data. In technical analysis—where a decimal error in a volatili…
Medium — fine-tuning tag
TIER_1English(EN)·SINAPSA Infocomplex·
<h1> How an LLM Evaluation Benchmark Produces Comparable Numbers: Inside the LFORLA Mechanism </h1> <p><strong>TL;DR:</strong> A leaderboard is only useful if you can trust the numbers. LFORLA's LLM evaluation makes scores comparable by fixing the prompts, fixing the judge, scori…
<p>Without evals, every prompt change is a guess. You fix the example in front of you and quietly break three you aren't looking at. An eval set turns that guess into a number you can watch.</p> <h2> Start from real failures </h2> <p>The best eval cases come from real inputs, esp…
<p>LLM agents can select tools, generate arguments, and initiate workflows, but a valid tool call does not guarantee that an action is safe or authorized. Human approval gates solve this problem by introducing a controlled checkpoint between an agent's decision and the execution …
<p>Not every software problem needs a large language model (LLM). If a task can be solved reliably using deterministic rules, SQL queries, standard APIs, search algorithms, or traditional machine learning, adding an LLM may increase cost, latency, security exposure, and maintenan…
<p>Cloud AI APIs are convenient, but they come with recurring costs, latency, and the small matter of sending your data to someone else's servers. Running a language model on your own machine solves all three — and in 2026 the tooling has matured enough to be genuinely practical …
<p>In Switzerland, a letter from a public office can start a clock. A <em>Zahlungsbefehl</em> (payment order) gives you 10 days to object. An <em>Einsprache</em> period is often 30. Miss the date and you lose a right. Many people read these letters in a language that isn't their …
<h1> How to Evaluate LLMs: A Complete Guide to LLM Evaluation </h1> <p>From automated metrics to human evaluation — build a robust evaluation pipeline for your LLM applications.</p> <h2> Why Evaluation Matters </h2> <p>You've built an LLM app. It works on your test cases. But wil…
dev.to — LLM tag
TIER_1English(EN)·Acadify Solution·
<p>A wrong RAG answer does not automatically mean the language model failed. The system may have retrieved the wrong document, omitted the relevant passage, or supplied conflicting evidence.</p> <p>This guide builds a small <strong>retrieval-only evaluation harness</strong>. It s…
<h2> TL;DR </h2> <p>The price per million tokens is the sticker, not the cost. The real cost of an LLM call depends on how many tokens you actually spend, the precision the provider serves at, how much the request waits, and whether the endpoint is even up. This post is a measure…
<h2> TL;DR </h2> <p>LLM evaluation only becomes useful when every model faces the same prompts, the same fixed judge, and the same per-axis rubrics, with a public verbatim trail so anyone can re-read what a model actually answered. This article explains that mechanism step by ste…
<h1> From Auditable LLMs to Auditable Vision Rehab: Why Transparency Wins </h1> <p>A recent dev.to post sparked a necessary conversation about LLM-generated ratings. The author proposed a radical ruleset: fix the evidence window, show the actual quotes, admit limitations, and pro…
<h2> TL;DR </h2> <p>LLM evaluation only produces comparable numbers when every model faces the same prompts, the same fixed judge, per-axis rubrics, and a public verbatim trail. This article walks through how the FreeCAD Fix benchmark on LFORLA does exactly that, and what its lea…
<p><strong>Published:</strong> 2026-10-03</p> <p>LLM evaluation is how you measure whether a model's output is actually correct, rather than just fast or cheap. It is the half of quality that monitoring cannot see. I built a small evaluation script in Python, scored eight model a…
<p>Large Language Models (LLMs) have changed how we interact with computers.</p> <p>Tools such as ChatGPT, Claude, Gemini, and many open-source models can write code, explain complex topics, summarize documents, translate languages, and generate natural-sounding conversations.</p…
<h1> LLM Evaluation: How a Benchmark Turns Raw Answers Into Comparable Numbers </h1> <p><strong>TL;DR</strong> — LLM evaluation only produces comparable numbers when every model faces the same prompts, the same fixed judge, and the same per-axis rubrics, with a public verbatim tr…
<h2> <strong>Mistral Large 4 (“Le Chonk”) Takes the Stage: What the Benchmarks Really Reveal</strong> </h2> <h3> A Sharper Lead </h3> <p>On <strong>October 3 2026</strong>, Mistral released a <strong>1‑trillion‑parameter</strong> model that anyone can download under an open‑weigh…
dev.to — LLM tag
TIER_1English(EN)·Denis Lavrentyev·
<h2> Introduction: The Rise of LLMs and Their Impact </h2> <p>The integration of <strong>Large Language Models (LLMs)</strong> into professional workflows has sparked a critical debate: <em>Is advanced knowledge of local LLMs essential for leveraging AI tools effectively?</em> Th…
dev.to — LLM tag
TIER_1English(EN)·Filipe Martins·
<h1> How LLMs Work: A Journey Through Tokens, Attention, and Transformers </h1> <p>“My favourite rock band is…”</p> <p>How does a large language model take that unfinished sentence and decide what comes next?</p> <p>You’ve probably heard that LLMs “predict the next token”. But th…
<h2> TL;DR </h2> <p>LLM evaluation only becomes useful when every model faces the same prompts, the same fixed judge, per-axis rubrics, and a public verbatim trail. The LFORLA Reverse Engineering benchmark does exactly that: it restores C source from stripped binaries, scores wit…
dev.to — LLM tag
TIER_1English(EN)·Lightning Developer·
<p>Running large language models (LLMs) on microcontrollers has long been considered a "what if" scenario reserved for theoretical discussions. However, the recent emergence of the <a href="https://github.com/Low-Zi-Hong/ESP32s3-LLM-Cluster" rel="noopener noreferrer">ESP32s3-LLM-…
<p>Fine-tuning a large language model on your own data used to require serious GPU budgets and weeks of infrastructure work. LoRA (Low-Rank Adaptation) changed the math: you can adapt a 7B-parameter model to a specific domain in a few hours on a single consumer GPU, without touch…
A mechanism explainer on LLM evaluation using the Team Recruitment Oracle benchmark: same prompts, a fixed judge, per-axis rubrics, and a public verbatim trail. Real scores for Nemotron 3 Ultra, HY3, and our own GLM 5.2. # llm # evaluation # benchmark # ai # software # coding # d…
<!-- SC_OFF --><div class="md"><p>Source: <a href="https://github.com/ying-hui-he/Hi-ToM_dataset">https://github.com/ying-hui-he/Hi-ToM_dataset</a></p> <p>I asked several LLMs to solve this Hi-ToM puzzle:</p> <p>The following story happens in chronological order. You will be give…