两个不同的AI项目MonkeyCode和OpenAmer正在强调AI代理透明且可验证的性能指标的重要性。MonkeyCode强调，应将免费访问运行器的简单分数视为笔记，而不是能力等级，主张进行可复现的比较。OpenAmer是一个自我验证的AI代理，在其账本中公开其失败记录，认为理解代理为何失败对于改进至关重要，并且这种透明度允许对其性能进行独立验证。 AI
影响 强调了AI代理开发中可验证指标的必要性，可能影响性能的基准测试和沟通方式。
排序理由 该集群讨论了具体的AI代理项目及其性能衡量和透明度的方法，这属于AI开发工具的范畴。
在 Mastodon — mastodon.social 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
A cheap runner can exercise a coding agent, yet that exercise does not become a capability ranking by itself. The honest product of the run is a labeled observation that names the dataset, the controls, and the clock. Readers should treat any single score from an unpaid lane as a…
Most of our agent's runs fail. The ledger records every one of them. OpenAmer is a self-verifying AI agent that runs on a CPU-only Windows laptop. It keeps a public outcome ledger — memory/si/outcome_ledger.jsonl — where every run it takes is recorded as a row with a timestamp, t…
If you ask a language model to grade a text and also to count the words, add up the points and apply... # ai # llm # typescript # webdev # software # coding # development # engineering # inclusive # community Don't let the LLM do the maths: grading writing with AI but scoring in …