Researchers have developed two novel methods for generating semantic IDs (SIDs) for retrieval-oriented systems, aiming to improve the recall capabilities of generative retrieval. The first method, CARE-SID, quantizes in a retrieval-aligned space by training a residual alignment network with a contrastive loss, incorporating anchor and reconstruction losses to prevent data collapse. The second method, CARE-LR, further enhances this by distilling the similarity distribution of original embeddings over a candidate list into the SID reconstructions. Both approaches have demonstrated higher reconstruction-view recall than traditional residual k-means in offline evaluations on industrial video search data, with CARE-LR showing further improvements. CARE-SID has already been implemented in an industrial video search system. AI
IMPACT These methods could enhance the performance of generative retrieval systems, particularly in large-scale search applications.
RANK_REASON Academic paper detailing novel methods for retrieval-oriented semantic IDs. [lever_c_demoted from research: ic=1 ai=1.0]
Read on arXiv cs.IR (Information Retrieval) →
- arXiv
- CARE-LR
- CARE-SID
- CatalyzeX
- Connected Papers
- CORE Recommender
- DagsHub
- Gotit.pub
- Hugging Face
- Litmaps
- RQ-VAE
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →