REAP system extracts knowledge from LLMs for AKBC Shared Task 2026

Researchers have developed a system called REAP for the AKBC Shared Task 2026, which focuses on building knowledge bases from language models without fine-tuning. REAP utilizes structured chain-of-thought reasoning and relation-specific queries to extract knowledge, which is then formatted into JSON. Built on the Mistral-Small-24B-Instruct-2501 model, REAP achieved a macro-F1 score of 0.62 on the test set, demonstrating strong performance on specific relation types like countryLandBordersCountry and companyTradesAtStockExchange. AI

IMPACT This research demonstrates a method for extracting structured knowledge from LLMs, potentially improving the accuracy and efficiency of knowledge base creation for specific domains.

RANK_REASON The cluster describes a research paper detailing a system for knowledge base construction from LLMs, including specific performance metrics and model details. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• AKBC Shared Task 2026
• arXiv
• companyTradesAtStockExchange
• countryLandBordersCountry
• hasArea
• Hugging Face
• JSON
• Mistral-Small-24B-Instruct-2501
• REAP

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →