RadixAttention

New techniques boost small LLM Bash generation and speed up AI inference

Researchers have developed a technique called grammar-constrained decoding to improve the Bash command generation capabilities of small language models. This method enhances accuracy and safety, transforming natural lan…