DAYJOB: Tax A benchmark for AI tax agents

Training Lab

Better data. Bigger gains.

We train frontier models on our off-the-shelf data and put the resulting models to the test. Each training run demonstrates what great data makes possible: stronger benchmark scores, gains that generalize to new capabilities, and leaps in model behavior.

Agentic coding
GLM-5.3
NOV 2026

We trained GLM-5.3 on agentic coding tasks. It improved across three SWE evals.

Agentic Coding

Terminal-Bench 4 (mini-swe harness)
pass@1
+21.3pp
34.8% → 56.1%
SWE-Marathon (claude-code harness)
pass@1
+15pp
30% → 45%
sudo L7
pass@1
+10pp
18.9% → 28.9%
DeepSWE (mini-swe harness)
pass@1
+6.5pp
66.90% → 73.45%
Model Deep Dive
Nemotron 3.5 Super
Oct 2026

We trained Nemotron 3.5 Super on four capabilities.
It improved across 12 benchmarks.

We trained Nemotron 3.5 Super on Surge data for agentic coding, long-horizon tool use, professional graphical reasoning, and instruction following.

Agentic Coding

Terminal-Bench 2.1
pass@1
+6.3pp
63.9% → 70.3%
SWE-Bench Pro V2
pass@1
+5.2pp
86.9% → 92.1%
DeepSWE
pass@1
+4.7pp
27.9% → 32.6%

Long Horizon Tool Use

AutomationBench
pass@1
+12.7pp
14.8% → 27.5%

Visual reasoning

Chartography
With tools, pass@1
+11.2pp
39.4% → 50.6%
ChartMuseum
With tools, pass@1
+6.45pp
79.35% → 85.80%
GDP.pdf
With tools, pass@1
+18.6pp
12.4% → 31.0%

Instruction Following

HANDBOOK.md
pass@1
+12.8pp
54.8% → 67.6%
MultiChallenge
pass@1
+8.3pp
58.3% → 66.6%
ComplexConstraints
pass@1
+8.2pp
27.3% → 35.5%
AdvancedIF
pass@1
+7.7pp
50.6% → 58.3%
IFBench
pass@1
+4.5pp
72.3% → 76.8%
Professional work
GLM-5.3
Oct 2026

Professional Work Training Boosts
GLM-5.3 by 54 Elo Points on GDPval-AA

Post-training GLM 5.3 on Frontier Work and GDPval-Verified data raised its GDPval-AA v2.1 Elo rating from 1650 to 1704 (+54 points). On DAYJOB, which remains quite difficult for GLM 5.3, the Healthcare task pass rate rose from 0% to 8%.

GDPval-AA v2.1
elo rating
+54
1650 → 1704
DAYJOB: Healthcare
Task Pass Rate
+8.0pp
0% → 8.0%
Agentic coding
Kimi K2.7
Oct 2026

We trained a model on 2,400 coding tasks.
It improved across every SWE eval, again.

Training on 2,400 expert-built coding tasks improved Kimi K2.7 across every external SWE eval we measured. This builds on an earlier run with 1,700 tasks, with continued performance improvements.

DeepSWE
PASS@1
+17.7pp
31.0% → 48.7%
SWE-Bench Pro
pass@1
+8.0pp
60.1% → 68.1%
Terminal-Bench 4
pass@1
+4.8pp
9.5% → 14.3%
Professional Work
Kimi K2.7
Oct 2026

We trained a model on professional work.
It learned to produce deliverables it couldn't before.

Post-training on tasks from the DAYJOB companion set raised Kimi K2.7's mean rubric score on AA-Briefcase-Lite by +8.4 pp.

AA-Briefcase-Lite
mean rubric score
+8.4pp
14.1% → 22.5%
Agentic Coding
GLM-5.3
Oct 2026

We trained a coding agent to think more like a staff engineer.

Post-training GLM-5.3 on 750 tasks from the sudo L7 companion set improved pass@1 on DeepSWE by +4.8 pp and Terminal-Bench 4 by +6.6 pp.

DeepSWE
pass@1
+4.8pp
66.9% → 71.7%
Terminal-Bench 4 (mini-swe-agent)
pass@1
+6.6pp
38.89% → 45.45%
Goal-Directed Execution
Kimi K2.7
Oct 2026

We trained a model on 1,700 coding tasks.
It got better at office work.

Training on Surge coding data improved GDPval-AA from 30.7% to 37.6%, even though GDPval-AA contains no coding tasks.

GDPval-AA
zero coding tasks in the benchmark
+6.9pp
30.7% → 37.6%
Goal-Directed Execution
Kimi K2.7
Oct 2026

We trained a model on DAYJOB-style office work.
It got better at coding.

Training on the DAYJOB companion set, which contains no coding tasks, nonetheless improved Terminal-Bench 2.1 by +8.4pp.

Terminal-Bench 2.1
pass@1
+8.4pp
67.4% → 75.8%
zero coding tasks in training
Professional Work
Kimi K2.7
Sep 2026

GDPval success doubled after training on 1,400 GDP.pdf tasks

We post-trained Kimi K2.7 on 1,465 GDP.pdf companion tasks. The GDP.pdf holdout full-task pass rate rose +13.5pp and GDPval success more than doubled. The trained model used tools to find more of the relevant sources before creating deliverables, while reading less text overall.

GDP.pdf
full-task pass rate
+13.5pp
11.0% → 24.5%
held-out
GDP.pdf
tasks at 90%+ of rubric points
+9.0pp
27.0% → 36.0%
held-out
GDPval
full-task pass rate
+3.2pp
2.7% → 5.9%
GDPval
tasks at 90%+ of rubric points
+13.1pp
19.6% → 32.7%
Agentic Coding
Kimi K2.7
Aug 2026

We trained a model on 1,700 coding tasks.
It learned skills that generalized across every SWE eval.

Training on 1,700 Surge coding tasks improved Kimi K2.7 across every external SWE eval we measured. The biggest behavioral changes showed up in the complexity of real-world software engineering: carrying messy specifications through, testing beyond the obvious path, preserving existing behavior, and building its own checks when the environment did not provide one.

SWE-Marathon
pass@1
+20.0pp
5.0% → 25.0%
Terminal-Bench 2.1
pass@1
+14.6pp
67.4% → 82.0%
DeepSWE
pass@1
+12.4pp
31.0% → 43.4%
Terminal-Bench 3
pass@1
+10.7pp
1.4% → 12.1%
SWE-Bench Pro
pass@1
+4.7pp
60.1% → 64.8%
Goal-Directed Execution
Qwen3.5-122B-A10B
Jul 2026

We trained a model on office work RL environments with no code. It got better at coding anyways.

We trained on long-horizon office tasks with no coding examples. SWE-Bench Pro still improved +5.7pp. Trajectory analysis pointed to a broader capability we call goal-directed execution: choosing useful subgoals, maintaining task state, staying faithful to the objective, and verifying completion.

SWE-Bench Pro
pass@1
+5.7pp
External transfer
zero coding tasks in training
Complex Instruction Following
QWEN3-4B · NEMOTRON
JUN 2026 · UPDATED OCT 2026

We trained a model on single-turn ComplexConstraints. It also got better at multi-turn instruction following.

Every ComplexConstraints training example was single-turn, yet the largest gains appeared in multi-turn behaviors. Our hypothesis is that the underlying skill is shared: maintaining many interacting constraints without dropping them, whether they arrive in one dense prompt or accumulate over nine turns.

Qwen3-4B

MultiChallenge
overall
+10.1pp
41.1% → 51.2%
AdvancedIF
pass@1
+8.5pp
28.15% → 36.6%
Instruction Retention
multi-turn dimension
+22.1pp
Largest behavioral gain

Nemotron 3.5 Super

MultiChallenge
pass@1
+8.3pp
58.3% → 66.6%
AdvancedIF
pass@1
+7.7pp
50.6% → 58.3%
IFBench
pass@1
+4.5pp
72.3% → 76.8%
Tool Use Agents
Qwen3.5-122B-A10B
May 2026

We trained a model on long-horizon tool-use environments. It learned agentic behaviors that transferred across every tool eval.

We trained Qwen3.5-122B-A10B on realistic, multi-tool agent environments. The gains transferred to Toolathlon, τ²-Bench, and BFCL-V4. Trajectory analysis showed better task closure and more efficient tool use across unfamiliar evaluation environments.

Toolathlon
pass@1
+9.6pp
24.2% → 33.8%
τ²-Bench
pass@1
+5.3pp
54.8% → 60.1%
BFCL-V4
pass@1
+3.5pp
55.7% → 59.2%
Expert STEM
GLM-4.7
May 2026

We trained a model on expert STEM problems. The gains transferred across every scientific reasoning benchmark.

We trained GLM-4.7 on expert-authored STEM problems. It nearly doubled performance on held-out Expert STEM tasks, with gains also appearing on HLE, FrontierScience, PubMedQA, and DeepDive. The result suggests that difficult expert problem solving can teach reasoning patterns that remain useful across different scientific domains.

Expert STEM
pass@1
+24.9pp
22.7% → 47.6%
held-out
HLE
pass@1
+6.8pp
26.8% → 33.6%
text prompts
FrontierScience
pass@1
+9.3pp
29.1% → 38.4%
PubMedQA
pass@1
+13.7pp
66.7% → 80.4%
DeepDive
pass@1
+9.1pp
57.4% → 66.5%
Enterprise Agents
GLM-4.6
May 2026

We trained a model on customer-support RL environments. It learned tool-use skills that generalized beyond.

We trained GLM-4.6 inside CoreCraft, a high-fidelity customer-support environment. The gains transferred beyond CoreCraft to BFCL, τ²-Bench Retail, and Toolathlon. The result suggests that realistic enterprise workflows can teach general agent skills that survive changes in tools, tasks, and evaluation environments.

CoreCraft
task pass rate
+11.4pp
25.4% → 36.8%
held-out
BFCL Parallel
task pass rate
+4.5pp
91.0% → 95.5%
BFCL Simple
task pass rate
+2.0pp
91.5% → 93.5%
τ²-Bench Retail
task pass rate
+7.4pp
68.7% → 76.1%
Toolathlon
task pass rate
+6.8pp
18.8% → 25.6%

Surgio Model Family

Meet our family of post-trained models. Each starts with an open foundation model and is trained on Surge AI data and environments.

Surgio-SWE Kimi K2.7 Edition

Specialized intelligence for real-world software engineering.

Surgio-SWE GLM-5.3 Edition

Trained to code with the taste and judgment of staff engineers.

Surgio Custom Models

Post-trained on your code, data, and workflows

Talk to us

Train on the data behind these runs

The same off-the-shelf Surge AI datasets powering these runs are available to you. Train on the full dataset first, and pay only if it improves your metrics.