PulseAugur
中
实时 09:43:26
English(EN) KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

新的 KaliBench 基准测试用于测试 LLM 生成网络安全 CLI 命令的能力

作者 PulseAugur 编辑部 · [2 个来源] ·

研究人员推出 KaliBench，一个旨在评估大型语言模型（LLM）为网络安全工具生成精确命令行界面（CLI）命令能力的新基准测试。该基准测试包含 1,642 个工具的 8,504 个查询-命令对，重点关注准确的工具选择和参数构建。目前开源模型在此任务上表现不佳，精确命令准确率低于 42%。然而，使用 KaliBench 的可验证奖励进行微调显著提高了 8B 模型的性能。 AI

影响 该基准测试通过为 CLI 命令生成提供标准化评估，有望加速开发更强大的网络安全运营 LLM。

排序理由 该集群包含一篇学术论文，介绍了一个用于评估 LLM 在特定任务上性能的新基准测试。[lever_c_demoted from research: ic=1 ai=1.0]

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

新的 KaliBench 基准测试用于测试 LLM 生成网络安全 CLI 命令的能力

本文如何被排名

Signal score
1 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
该集群包含一篇学术论文，介绍了一个用于评估 LLM 在特定任务上性能的新基准测试。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, product
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
1 days old
Coverage has settled into its steady-state source set.
Coverage growth since scoring
+1 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [2]

  1. arXiv cs.AI TIER_1 English(EN) · Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer ·

    KaliBench：Kali Linux 上网络安全工具使用的细粒度基准测试，具有运行时免费的可验证奖励

    arXiv:2610.02206v1 Announce Type: cross Abstract: LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts' intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks,…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    KaliBench：Kali Linux 上网络安全工具使用的细粒度基准测试，具有运行时免费的可验证奖励

    LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts' intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks, and do not directly measure LLMs' ability to gene…

相关话题