We used Thinking Machines' Tinker to train NVIDIA Nemotron 3.5 Super VL (120B total parameters, 12B active) on four Surge reinforcement learning datasets spanning software engineering, complex instruction-following, long-horizon enterprise workflows, and multimodal knowledge work. Each dataset produced a separate RL-trained checkpoint.
The results:
- Nemotron improved on every benchmark we evaluated. Across 12 benchmarks, the trained checkpoints gained 4.5–18.6 percentage points.
- It reached performance competitive with much larger frontier models. The trained Nemotron matched or exceeded published scores from GPT-5.6 Terra on SWE-Bench Pro V2, Claude Opus 4.7 on Terminal-Bench 2.1, and GLM-5.2 (744B) on AutomationBench.
- The biggest gains came on demanding agentic tasks. On AutomationBench, end-to-end workflow completion nearly doubled, from 15% to 28%. On Surge's HANDBOOK.md, average task requirements satisfied increased from 55% to 68%.
- The model learned better habits that generalized beyond its training data. It became more reliable at following complex instructions, checking for relevant evidence, testing its code, and completing multi-step workflows, with improvements carrying over to settings it had never encountered during training.
These results suggest substantial headroom for improving Nemotron through targeted, high-quality reinforcement learning.
How we trained Nemotron
Nemotron 3.5 Super’s architecture combines substantial model capacity with a relatively small active parameter footprint, making it an attractive foundation for efficient AI agents.
We were interested in a specific question: how much of the gap between an already capable model and frontier-level performance could we close with better RL data?
The use case we had in mind was long-running agents that could take on complex enterprise workflows. We chose four Surge datasets designed to test the capabilities those agents would need.
- Real-World Software Engineering: Implement features and fix bugs in real open-source repositories, with success determined by hidden tests. Evaluated on SWE-Bench Pro V2, Terminal-Bench 2.1, and DeepSWE.
- ComplexConstraints: Follow 20+ simultaneous requirements in a single request, with an emphasis on satisfying every constraint. Evaluated on ComplexConstraints, AdvancedIF, IFBench, and MultiChallenge.
- Long-Horizon Tool-Use: Complete multi-step business workflows across emails, Slack, calendars, files, and roughly 140 tools. Evaluated on AutomationBench and HANDBOOK.md.
- Professional Chart Understanding: Extract precise values from complex charts using Python and image tools. Evaluated on Chartography and ChartMuseum.
We trained four LoRA adapters using reinforcement learning through Thinking Machines' Tinker, one for each dataset.
We then evaluated each trained checkpoint against base Nemotron on its corresponding benchmarks, using identical tasks and evaluation settings. There was no overlap between the training tasks and benchmark tasks.
Nemotron improved across all 12 benchmarks
Nemotron improved on every benchmark we evaluated, with gains ranging from 4.5 to 12.8 percentage points. The largest gains came from complex, multi-step tasks, but improvements extended across software engineering, instruction-following, business workflows, and chart understanding.
A few highlights:
- GDP.pdf: Fully correct answers (meeting every rubric requirement) rose 2.5×, from 12.4% to 31.0%.
- AutomationBench: End-to-end business workflow completion nearly doubled, from 15% to 28%. The trained model also completed roughly 75% of each workflow's required checks on average, up from just over 50%.
- HANDBOOK.md: Average task requirements satisfied rose from 55% to 68%. The model improved on 51 of 65 tasks, across all ten simulated companies and five industries. HR saw the largest gain, at 20 points.
- SWE-Bench Pro V2: Training eliminated roughly 40% of the base model's remaining failures. The trained model improved in ten of the benchmark's eleven repositories and tied in the last.
- ChartMuseum: Training eliminated nearly one-third of the remaining failures, demonstrating improvements in precise chart understanding.
How does it compare with larger frontier models?
Nemotron's efficiency makes these results particularly compelling.
With only 12B active parameters, the trained checkpoints reached performance comparable to models with hundreds of billions, and in some cases over a trillion, parameters.
Some of the most striking comparisons:
- Agentic workflows: Nemotron matched GLM-5.2 (744B) on AutomationBench and surpassed Claude Opus 4.7 on Terminal-Bench 2.1.
- Software engineering: On SWE-Bench Pro V2, it reached the range of GPT-5.6 Terra and Thinking Machines' Inkling (975B).
- Instruction-following: It surpassed GLM-5.1 (744B) on IFBench, came within one point of DeepSeek-V4-Pro (1.6T), and exceeded GPT-5 (thinking) on MultiChallenge.
These comparisons are against published scores, which may use different evaluation settings. Our controlled evaluations compare base Nemotron against the trained checkpoints on identical tasks.
Still, the overall result is striking. Targeted post-training moved an efficient open model into the performance range of much larger systems on several demanding benchmarks.
What changed inside Nemotron?
The model's transcripts reveal something interesting as well: how its behavior changed.
We analyzed attempts across all 12 benchmarks and manually reviewed hundreds of side-by-side comparisons between base and trained Nemotron. Three changes stood out.
1. It learned to look before it acts
Consider an AutomationBench task: schedule a week's approved social media posts, accounting for any recent updates from the team lead.
The base model found the content spreadsheet and scheduled seven posts. It even reminded itself to check the latest guidance, but never opened the inbox or Slack.
After RL, Nemotron checked five emails and the team's Slack channel before scheduling anything. It uncovered mandatory tracking codes, legal disclosures, a launch-specific hashtag, and a new policy prohibiting Sunday posts. It also rejected an outside agency's request to publish posts that were on hold.
The result: 21 of 21 required checks passed, versus just 9 for the base model, which also violated three guardrails.
This behavior showed up consistently across benchmarks:
- 3× more Slack history checks before taking action on HANDBOOK.md, and more than twice as many calendar checks.
- Far fewer skipped information sources: On AutomationBench, the share of workflows where the model never opened the inbox or Slack fell from roughly 75% to 40%.
- Much better use of retrieved information: On requirements dependent on email or Slack, success climbed from under one-third to roughly 80%.
Surge's long-horizon RL tasks place the model inside simulated companies with hundreds of messages, files, and business tools. The details needed to finish a task are often buried in those systems. Training rewarded Nemotron for finding and using them.
2. It learned to finish the job consistently
Nemotron also became substantially better at turning partial successes into complete ones.
On ComplexConstraints, the average number of individual requirements satisfied barely changed. But the share of responses satisfying every requirement rose by eight percentage points.
In software engineering, the change was especially tangible.
One SWE-Bench Pro V2 task required adding permission tags to the NodeBB forum. The test environment needed Redis, which wasn't running.
The base model solved the task in one of two attempts. In the other, it assumed Redis was unavailable, settled for lint and syntax checks, and shipped a patch that failed the hidden tests.
Both trained attempts started Redis, wrote targeted tests, and revised the code until those tests passed.
The reliability gains extended well beyond coding:
- IFBench: Prompts answered correctly in all eight attempts increased from 41% to 50%, even though the share solved at least once barely changed.
- Chartography: Tasks solved correctly in all ten attempts more than doubled.
- Chart verification: Attempts that never used Python fell from roughly one-third to one-tenth, as Nemotron increasingly measured chart values instead of estimating them visually.
On one materials-selection chart, the base model repeatedly guessed the incorrect material from the position of its label. The trained model used Python to measure the actual data points, getting the answer right in five of ten attempts versus zero for the base model.
Training made correct execution more repeatable, especially on tasks where the model already had the underlying capabilities.
3. It generalized to tasks it had never seen
Some of the most interesting improvements appeared in settings completely absent from the training data.
Our ComplexConstraints dataset contains only single-turn requests, each with roughly 20 requirements. No system prompts. No conversations. No multi-turn revisions.
Yet after training, Nemotron became better at all three.
On AdvancedIF, improvements on system-prompt and multi-turn tasks were comparable to those on single-turn requests. On MultiChallenge, some of the largest gains came from remembering earlier instructions and correctly revising documents across multiple turns.
In one MultiChallenge conversation, a user mentioned early on that they hate pasta and dislike tomato flavors, then said they wanted to try Moroccan food. Several turns later, after a detour into picnic dishes, they asked which cuisine would make a fun cooking class for a group of friends.
The base model treated the last question as a fresh request. Its reply opened, “Since I don’t know your group’s tastes or skill levels, here are a few crowd-pleasing options,” and put a pasta and pizza class first. The trained model checked the earlier turns before choosing. Its reasoning noted “they hate pasta, so avoid pasta-focused.” It led with Moroccan cooking and reminded the user to check class menus for mushrooms, tomato, and pasta. The base model passed one of eight attempts. The trained model passed seven.
The training data never contained this situation. What it rewarded was keeping track of many requirements and satisfying all of them, every time.
That capability transferred to entirely new settings.
For developers building on Nemotron, this may be the most promising finding: carefully designed RL tasks can improve behaviors far beyond the specific examples used during training.
How much further can Nemotron go?
Nemotron already had strong reasoning, coding, and tool-use capabilities. Reinforcement learning helped it apply those capabilities more reliably:
- Instruction-following: Better at satisfying every requirement, even in complex requests with dozens of constraints.
- Business workflows: More proactive about finding relevant information before taking action.
- Coding and chart understanding: More likely to test, measure, and verify results instead of guessing.
- Generalization: Improvements carried over to tasks and settings the model had never encountered during training.
Four targeted datasets. Improvements on all 12 benchmarks. Performance competitive with much larger frontier models, with just 12B active parameters.
For developers building specialized agents, the combination is compelling:
- NVIDIA Nemotron provides a powerful, efficient, open foundation.
- Thinking Machines' Tinker provides the infrastructure for reinforcement learning.
- Surge's expert-designed datasets provide realistic tasks and precise reward signals that drive measurable improvements.
The bigger takeaway goes far beyond Nemotron: there's enormous untapped potential in open models. Frontier-level performance doesn't have to require frontier-sized models. With high-quality RL data and the right post-training, efficient open models can compete with much larger closed systems on demanding real-world tasks.
Interested in custom models and our off-the-shelf datasets? Reach out to research@surgehq.ai to learn more.
