A new benchmark called H3D has been developed to evaluate unsupervised text hashing methods for fine-grained document deduplication, particularly for scientific documents. The benchmark compares traditional non-learning approaches like MinHash and SimHash with semantic-sensitive methods that utilize BGE embeddings. H3D assesses methods based on ranking quality, efficiency, and robustness, revealing a trade-off between lexical/structural fingerprints for near-duplicates and semantic representations for content rewriting, with the latter incurring higher computational costs. AI
RANK_REASON The item describes a new academic paper introducing a benchmark for evaluating text hashing methods. [lever_c_demoted from research: ic=1 ai=0.7]
Read on arXiv cs.IR (Information Retrieval) →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
