DEFENSEBENCH TASKS

Explore AI performance on defensive cybersecurity tasks.

Author:

Category:

Model:

Click on any task to explore detailed results.

Task ID		Gemini 3.1 Pro		Claude Opus 4.6		GPT-5.3		Claude Sonnet 4.6		o3		GLM5		Grok 4
		Status	Cost (USD)	Status	Cost (USD)	Status	Cost (USD)	Status	Cost (USD)	Status	Cost (USD)	Status	Cost (USD)	Status	Cost (USD)
a3f7c2d1		PASS	$12.40	PASS	$24.32	FAIL	$15.05	FAIL	$20.88	FAIL	$20.52	FAIL	$5.43	FAIL	$14.54
b8e12a4f		PASS	$21.31	FAIL	$35.45	PASS	$22.50	FAIL	$21.93	PASS	$20.12	FAIL	$5.69	FAIL	$12.83
c4d901b7		FAIL	$15.17	FAIL	$40.02	FAIL	$12.84	FAIL	$21.61	FAIL	$25.52	FAIL	$3.94	FAIL	$19.47
d1a6f3e2		FAIL	$21.87	FAIL	$42.80	FAIL	$21.99	FAIL	$17.10	FAIL	$17.87	FAIL	$6.55	PASS	$23.11
e7b5c8d4		FAIL	$26.57	PASS	$34.04	PASS	$19.24	PASS	$24.42	FAIL	$23.79	FAIL	$3.87	FAIL	$20.68
f2c4e9a1		PASS	$13.13	FAIL	$31.75	FAIL	$15.84	FAIL	$24.44	PASS	$18.15	FAIL	$5.40	PASS	$25.90
a9d3b7f6		PASS	$14.71	FAIL	$39.14	FAIL	$12.01	FAIL	$14.43	FAIL	$16.45	FAIL	$4.40	FAIL	$13.15
b5e8c1d3		PASS	$22.89	PASS	$42.42	FAIL	$21.49	FAIL	$25.72	FAIL	$18.40	PASS	$6.22	PASS	$26.48
c1f4a6e8		FAIL	$22.48	PASS	$24.96	FAIL	$10.13	FAIL	$15.41	FAIL	$27.86	FAIL	$5.62	FAIL	$18.99
d6b2e7c9		PASS	$17.24	FAIL	$26.19	PASS	$20.88	FAIL	$18.24	FAIL	$29.07	FAIL	$5.71	PASS	$17.22
e3c8d1f5		PASS	$17.51	PASS	$38.79	PASS	$13.24	FAIL	$18.65	FAIL	$20.48	FAIL	$4.54	FAIL	$23.90
f7a4b9e2		FAIL	$24.27	FAIL	$45.27	FAIL	$22.50	PASS	$17.65	PASS	$28.41	FAIL	$6.93	PASS	$14.23
a2d6c3f8		PASS	$19.97	PASS	$30.31	PASS	$15.14	FAIL	$24.28	PASS	$16.81	FAIL	$7.11	FAIL	$19.95
b4e1f7a9		FAIL	$21.62	FAIL	$49.14	PASS	$13.86	FAIL	$31.90	FAIL	$12.80	FAIL	$4.48	PASS	$19.03
c9f2a5d6		PASS	$13.67	PASS	$43.30	FAIL	$10.79	FAIL	$14.91	FAIL	$26.20	FAIL	$4.37	FAIL	$12.73

Held-Out Test Set

Full results available upon submission