New MARS LLM architecture uses internal state to override prompts

A researcher has developed a new language model architecture called MARS, which incorporates "proprioceptive channels" to allow the model to perceive its own internal state, such as memory salience or caution level. Initial experiments failed when the channels were used to convey facts, but succeeded when they were used to convey internal state while facts were in the prompt. A key test demonstrated that the model's internal state signals could override explicit text prompts, suggesting a new form of control beyond traditional prompt engineering. AI

IMPACT Introduces a novel architecture that could enable more nuanced and controllable LLM behavior beyond prompt engineering.

RANK_REASON The cluster describes a novel model architecture and experimental results presented in a research context. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Cloud TPU
• LLM
• MARS
• torch.compile
• torch_xla
• transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →