Phi-2
PulseAugur coverage of Phi-2 — every cluster mentioning Phi-2 across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New method deciphers Transformer model internals by differencing prompts
Researchers have developed a new method called Contrastive Projection to better understand the internal workings of Transformer models. This technique involves subtracting the hidden states of two closely matched prompt…
-
New ToMoE method converts dense LLMs to Mixture-of-Experts
Researchers have developed a method called ToMoE that converts dense large language models into Mixture-of-Experts (MoE) architectures. This technique uses differentiable dynamic pruning to reduce computational and memo…
-
Guides Explore LLM Fine-Tuning and Cache-Augmented Generation
This cluster provides guides on fine-tuning Large Language Models (LLMs) and explores alternative methods for grounding LLMs with external knowledge. The fine-tuning guides cover local methods using techniques like LoRA…
-
Small Language Models (SLMs) gain traction, challenging large model dominance
Small Language Models (SLMs), typically ranging from 0.5 to 7 billion parameters, are emerging as a significant alternative to large, resource-intensive models. These models are designed for efficiency from the ground u…
-
Gemini CLI: 10-line GEMINI.md matches 100-line performance, saves tokens
A practical test of Gemini CLI's GEMINI.md file revealed that a 10-line version performs identically to a 100-line version in terms of instruction following, while being faster and consuming fewer tokens. The experiment…
-
New RAG research separates context length from semantic competition
A new research paper proposes a method to distinguish between context length and semantic competition as causes for errors in retrieval-augmented generation (RAG) systems. The study introduces a matched-control protocol…
-
Language models' self-verification effectiveness varies by task and model
Researchers have investigated the effectiveness of language models verifying their own answers as a confidence signal. Their study, conducted on ARC-Challenge and TruthfulQA-MC datasets using various models like Phi-2 a…