ENTITY
Unigram LM
Unigram LM
PulseAugur coverage of Unigram LM — every cluster mentioning Unigram LM across labs, papers, and developer communities, ranked by signal.
Total · 30d
0
2 over 90d
Releases · 30d
0
0 over 90d
Papers · 30d
0
2 over 90d
TIER MIX · 90D
TOPICS
RECENT · PAGE 1/1 · 2 TOTAL
-
BPE vs. Unigram-LM: Tokenization algorithms create distinct vocabularies for chemistry SMILES
A new research paper explores the differences between two common tokenization methods, byte-pair encoding (BPE) and Unigram-LM, when applied to chemical SMILES strings. The study found that these algorithms produce sign…
-
New benchmark QuechuaTok highlights tokenization limits for agglutinative languages
A new benchmark called QuechuaTok has been developed to evaluate tokenization strategies for agglutinative, low-resource languages. Standard metrics like fertility rate are insufficient, so QuechuaTok introduces morphol…