A new study published on arXiv investigates the impact of pretraining data filtering on protein language models, specifically the ESM-2 family. Researchers found that removing viral proteins from training data significantly reduced the models' performance on viral protein tasks, while largely preserving performance on non-viral proteins. This effect was more pronounced in larger models, but performance could be restored through fine-tuning with modest computational resources and access to the removed data. AI
IMPACT Demonstrates that data filtering in protein language models can be selectively detrimental but is not a durable safeguard against fine-tuning.
RANK_REASON Research paper published on arXiv detailing findings on model training. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →