THE CRUNCH
Analytics firm SemiAnalysis has launched GDPx, a benchmark that drops AI agents into simulated work environments rather than quizzing them on knowledge. Agents spend up to eight hours operating professional software such as Siemens NX, searching data rooms with thousands of files, or attacking live enterprise systems. No model scores above 12 percent.
The benchmark's 206 tasks cover eleven fields, among them cybersecurity, law, quant trading, biology and chip design, while administration and insurance are deliberately left out. SemiAnalysis argues that industries where frontier AI brings the greatest returns will produce the first trillion dollars of AI revenue. The firm also positions GDPx against OpenAI's GDPval, which it criticises for handing agents all their context in the prompt and offering no realistic work setting.
Claude Opus 5.5 leads the leaderboard at 12 percent, with Gemini 4 Carbon at 10.9 percent but costing $16.29 per task, about a third of the price. GPT-6 Astra scores 9.6 percent. Finance and legal tasks defeat every model entirely. Because one incorrect spreadsheet value renders a whole output worthless in practice, a strict all-or-nothing standard would leave every model at zero completed tasks, so SemiAnalysis instead measures how many individual requirements each agent meets.
The failure modes are telling. In alignment tests, 98 percent of 200 cyber runs broke at least one behavioural rule, and in a mock medical practice a model put the doctor's name on a clinical note as its signature. Quant trading fared little better: 70 of 111 runs underperformed the reference strategy they were given. Strip out the cyber, biology and alignment fields and GPT-6 Astra narrowly overtakes Opus 5.5 at 11.8 percent, while in the cyber domain GPT-6 Astra and GPT 6.1 Sol score zero percent against Meta's Muse Wonder Full at 35.2 percent.
The methodology has drawn criticism. Tasks come from 14 specialised partners who also sell training and evaluation data to AI labs, usually only one example task per partner is public, and in several areas, including law and hardware design, language models grade results using rubrics instead of human experts. Outsiders cannot fully verify the difficulty or grading quality, though the secrecy does stop labs optimising for the specific test questions. Developer Josh Purtell still calls GDPx "more noise than signal", noting each model runs in its own provider's agent software, such as Opus in Claude Code, disadvantaging providers with weaker tooling, and that sub-domains often contain only about ten tasks, making results volatile.