Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
A new study evaluated 22 different models, ranging from small encoders to large instruction-tuned LLMs, on their ability to process patent data for tasks like retrieval, classification, and clustering. The research found that fine-tuning effectiveness is highly dependent on the specific task and that gains in one area do not always transfer to others. While larger models generally performed better within their families, cross-family comparisons showed noisy results, with smaller models sometimes outperforming larger ones on specific tasks. The study also highlighted that combining abstract and claim information significantly improved retrieval and classification, though all models struggled with out-of-domain queries. AI
IMPACT Provides insights into which models and fine-tuning strategies are most effective for processing specialized data like patents, informing AI operators in legal and R&D sectors.