Why we built it

The economy runs on documents that knowledge workers read through every morning: loan agreements, rating manuals, discharge summaries, wiring schematics, quarterly filings.

GDP.pdf contains 100 tasks drawn from professional workflows across 10 domains. Physicians, attorneys, insurance adjusters, bankers, and other professionals wrote and graded the tasks based on work they do themselves.

One of the hardest numbers on the page

GPT-5.6 Sol performs extremely well across many of OpenAI’s published benchmarks.

It scores 94.6% on GPQA Diamond, 96.7% on internal capture-the-flag challenges, 90.4% on BrowseComp, 88.8% on Terminal-Bench 2.1, and 84.6% on MMMU Pro with tools.

Its score on GDP.pdf is 30.7%. That's one of the lowest capability benchmark scores OpenAI reported for its flagship model.

An example task

Consider a claims handler reviewing a hail-damage claim.

The input file includes policyholder photos and a map showing the storm’s hail track. The claims handler needs to produce a table answering a few questions:

Was there hail damage? Did the May 8 storm cause it? Could the apparent damage come from an old roof, nail pops, missing shingles, or bird droppings?

An experienced adjuster might review several claims like this before lunch.

The models placed a house inside the storm’s hail zone even though the map showed it outside. They missed visible hail damage in the photos. They also missed the bird droppings.

They then presented their conclusions in a clean, confident, and completely wrong table.

Run it yourself

Reliable performance on routine professional work is becoming an important measure of progress.

GDP.pdf is designed to measure that gap.

Full results for every model we've run are on the leaderboard, and the paper covers how we built the tasks and how our experts grade them. The set is public on GitHub and Hugging Face. Reach out to benchmarks@surgehq.ai if you have any questions.

‍