HCInfer

HCInfer system enables LLMs on resource-constrained devices with error compensation

Researchers have developed HCInfer, a novel inference system designed to enable large language models (LLMs) to run efficiently on devices with limited memory. This system offloads parts of the model's compensation mech…