Mohammadali Khodabandehlou

LLMs taught to refuse answers when context is lost to KV-cache compression

Researchers have developed a method to teach Large Language Models (LLMs) to abstain from answering when crucial information is lost due to KV-cache compression. This technique, termed compression-aware abstention, trai…