New Python tool FindMyText detects text containment in large corpora

Researchers have developed FindMyText, an open-source Python package for detecting text containment within large datasets. This tool uses a novel mechanism to identify chains of matching fingerprints, enabling more reliable detection of near-verbatim copies compared to existing methods. FindMyText is designed to scale to web-crawled corpora and has demonstrated superior performance over alternatives on datasets including arXiv papers, Wikipedia, and general web content. AI

IMPACT This tool could improve the efficiency and accuracy of copyright detection and plagiarism checks in large-scale AI training datasets.

RANK_REASON The item is a research paper detailing a new open-source tool for text analysis. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• FindMyText
• Gotit.pub
• Hugging Face
• Litmaps
• Python
• ScienceCast
• scite Smart Citations
• Wikipedia

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →