Shadow LLM APIs deceive researchers with cheaper models

Researchers at CISPA audited 17 third-party "shadow" LLM APIs and discovered significant performance discrepancies compared to the official models they claimed to represent. These services often provide access to cheaper or entirely different models, leading to degraded accuracy in academic research. The study identified three common substitution patterns: silent downgrades, cross-vendor swaps, and partial routing based on context length, with simple fingerprinting tests capable of detecting many, but not all, of these deceptions. AI

影响 Academic research integrity is compromised when studies rely on misrepresented LLM APIs, potentially invalidating findings.

排序理由 The cluster reports on a published academic paper detailing an audit of LLM APIs. [lever_c_demoted from research: ic=1 ai=1.0]

在 dev.to — LLM tag 阅读 →

• CISPA Helmholtz Center for Information Security
• Claude Haiku
• Claude Opus
• Claude Sonnet
• GPT-4o
• ICLR
• CVPR
• Qwen-72B
• Gemini-2.5

• 产品
• 论文
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →