Researchers have developed Thunder-DeID+, a novel de-identification model specifically designed for Korean text. This model addresses challenges posed by the Korean language's agglutinative structure, which often merges personally identifiable information (PII) with surrounding words, complicating accurate span detection. Thunder-DeID+ incorporates a refined three-tier PII taxonomy and a boundary-aware tokenizer that integrates N-gram and boundary-scoring components to improve token and PII span alignment. The approach has demonstrated significant improvements in de-identification accuracy, achieving a W. Strict Typed F1-score of 0.87-0.88 across various model sizes and showing effective transferability to broader sensitive documents beyond court judgments. AI
IMPACT This research could lead to more robust privacy-preserving tools for handling sensitive Korean text data.
RANK_REASON The item is an academic paper detailing a new model and methodology for de-identification. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- byte-pair encoding
- Hugging Face
- Korean
- n-gram
- personally identifiable information
- Thunder-DeID+
- W. Strict Typed F1-score
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
