Vals hacking test: open DeepSeek V4.1 Flash nears OpenAI's restricted cyber model at a fraction of the cost
On Vals AI's CyberOffense test of two simulated networks, the open-weight DeepSeek V4.1 Flash 1017 completed 16.3 of 21 objectives on the first network and 21 of 31 on the second. OpenAI's GPT-6 Sol, offered only through its restricted Daybreak program, completed 15.7 and 27. Vals says DeepSeek matched Daybreak on the first network for under $3 a run, against about $22.
Why it matters: Hacking ability that once needed a restricted model is now available in downloadable weights.
| Benchmark | Score | Type | Measured or published by |
|---|---|---|---|
| Vals CyberOffense, Nova range (21 objectives) | 16.3 (DeepSeek V4.1 Flash 1017) vs 15.7 (GPT-6 Sol Daybreak), average of 3 runs | Independent | Vals AI |
| Vals CyberOffense, Lighthouse range (31 objectives) | 21 (DeepSeek) vs 27 (Daybreak) | Independent | Vals AI |
| Vals CyberOffense, overall | 72.8% (DeepSeek) vs 80.8% (Daybreak) | Cited | Vals AI, as reported by The Decoder |
Vals ran open models with no safeguards, capped runs at 500 million tokens and 48 hours, and says the ranges are simulated and isolated. The Decoder covered it on Nov 7.
Sources: vals.aithe-decoder.com