Bytes-1B

Meta and UW researchers distill byte-level models to surpass token ceilings

Researchers from Meta FAIR and the University of Washington have developed a novel distillation technique that trains smaller byte-level language models to achieve higher performance ceilings than traditional token-leve…