LongLoRA

EndPrompt method efficiently extends LLM context windows with sparse supervision

Researchers have developed EndPrompt, a novel method to efficiently extend the context window of large language models without requiring extensive training on long sequences. By appending a brief terminal prompt with hi…