Benchmarks
Benchmarks built on the real work businesses do, from coding and agents to professional work, STEM, and multimodal reasoning.
License them to see where you stand against public leaderboards.
Frontier Data
on Fireworks
For years, frontier labs have come to Surge AI to find where their models fail and to build the data that fixes it. Now our evals, training data, and benchmarks are available on Fireworks.
Off-the-Shelf Data
Training data, evals, and RL environments from the experts behind our frontier-lab work. Available today.
Each ships with samples, documentation, and frontier-model pass rates, so you can judge fit first.
Custom Evals
Most of what your model needs to get right isn’t in a public benchmark. We define great with your team, then experts in your field grade it.
See where it fails on your work, not someone else’s.
Custom Training Data
Expert demonstrations, tasks, environments, rubrics, and preference data for fine-tuning, reinforcement learning, or preference optimization.
Train on Fireworks, then rerun the same evals to measure the gain.
Evaluate. Train.
Measure the gain.
We work with your team to evaluate your AI capabilities, uncover what is limiting performance, and identify the solutions that will move your business forward. Then we build it with you.
Find the gaps
Run Surge AI evals and benchmarks against the model you serve on Fireworks. See where it fails on your work, not on someone else’s.
Train on Fireworks
Fine-tune on Fireworks with Surge AI training data, environments, and rubrics. Supervised fine-tuning, reinforcement learning, or preference optimization.
Measure the gain
Rerun the same evals on the tuned model and measure the gain. Same tasks, same graders, same experts. Whatever it still misses becomes the next round.
Why Surge AI
We’ve spent years finding where frontier models fail and building the data that fixes them. Frontier labs taught us what moves a model: rigorous evaluations, difficult training examples, high-quality feedback, and clear signals about where models fail.
We bring those methods to teams building on Fireworks.
Expert judgment at scale
High-stakes work requires more than generic annotation.
Our expert workforce includes doctors, lawyers, engineers, researchers, finance professionals, and other specialists who evaluate AI against professional standards and real-world requirements.
Meet our experts
Pilot results in a week
Frontier AI moves in days, not quarters, and we’re built to match.
Share your pilot tasks and our experts start right away. Within a week, you’ll have results on your own work and a clear view of where your model needs to improve.
Start a pilot
Published research, proven results
We test our own data. We study whether post-training methods actually produce meaningful gains, not just theoretical capability. We publish the methods and the measured improvements across coding, reasoning, agentic tasks, and many other domains.
Explore our research
Bring us the problem you’re trying to solve
Tell us what you’re running on Fireworks and where it falls short. We’ll scope the evals or data that move it, and have pilot results back within a week.
You’re serving a model on Fireworks and need to know where it fails.
You want to fine-tune on Fireworks but don’t have the data.
You’re choosing between open models with no benchmark you trust.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.