New Frontier data and RL environments, off the shelf

Frontier Research Mathematics

Riemann-bench

We evaluate AI models on advanced mathematical problems requiring deep reasoning and novel synthesis. Our benchmark features cutting-edge problems sourced from leading mathematicians, including Ivy League professors, PhD IMO medalists, and graduate students at the top of their field.

RL Environments and the Hierarchy of Agentic Capabilities
Our RL environment run on 9 models revealed the core capabilities all agents need to master: tool use, planning, adaptability, groundedness, and common sense.
Leaderboard
1
GPT
5.6 Sol (Max reasoning)
74.4
%
1
Claude
Opus 5 (Adaptive/Max)
68
%
1
Claude
Fable 5 (Adaptive/Max)
60
%
1
GPT
5.5 (xHigh reasoning)
55.2
%
1
Claude
Opus 4.8 (Adaptive/Max)
47.2
%
1
GPT
5.4 (xHigh reasoning)
41.6
%
1
Grok
4.5 (High reasoning)
38.4
%
1
Kimi
K3 (Max reasoning)
37.6
%
1
GPT
5.2 (xHigh reasoning)
37.6
%
1
Kimi
K3
37.6
%
1
Gemini
3.5 Flash (High reasoning)
36.8
%
1
Gemini
3.1 Pro
33.6
%
1
Claude
Opus 4.7 (Adaptive/Max)
32.8
%
1
Gemini
3.6 Flash (High reasoning)
30.4
%
1
Claude
Opus 4.6 (Adaptive/Max)
27.2
%
1
Muse
Spark 1.2 (xHigh reasoning)
23.2
%
1
Muse
Spark 1.1 (xHigh reasoning)
20.8
%
1
Qwen
3.8 Max
15.2
%
1
Qwen
3.7 Max
15.2
%
1
Inkling
Inkling
15.2
%
1
Kimi
K2.5
12
%
1
Claude
Opus 4.5 (Adaptive/Max)
11.2
%
1
Gemini
3.5 Flash-Lite (High reasoning)
11.2
%
1
DeepSeek
V4 Flash
10.4
%
1
GLM
5.2
10.4
%
1
DeepSeek
V3.2 (Thinking)
8
%
1
Kimi
K2.6
7.2
%
1
DeepSeek
V4 Pro
5.6
%
Examples

Extreme Difficulty,
Rigorous Verification

Robust Maximal Independent Sets
Problem

A robust maximal independent set in a graph $G$ is a maximal independent set that remains maximal in all connected spanning subgraphs of $G$. How many connected graphs on $12$ vertices have the property that every maximal independent set is a robust maximal independent set, up to isomorphism?

Hahn Series and Multibasic Modules
Problem

Notation and definitions for background context:

Let $F$ be the field of order 2. Let $K$ be the field of Hahn series in indeterminate $t$ with value group $\mathbb{R}$ and residue field $F$. Let $A$ be the subring of $K$ consisting of those $a \in K$ with non-negative valuation. Consider $K$ as an $A$-module. For $q \in \mathbb{R}$, let $I_q = t^q A$ and $I_{>q} = \bigcup_{r>q} I_r$. Write $A/I_{>0}$ as $F$, since they are identical both as $A$-modules and as fields. Let $\Theta = K/I_{>0}$ and $\Phi = K/A$. We say that an $A$-module $M$ is 'basic' if it is isomorphic to $L/N$ for some $N < L \le K$, and that it is 'multibasic' if it is a direct sum of a (possibly empty) finite list of basic modules. For $A$-submodules $U, V$ of $K$, let $U + V = \{a + b : a \in U, b \in V\}$, $UV = \{ab : a \in U, b \in V\}$, and $c(V) = \{a \in K : aV \le I_{>0}\}$.

You may assume the following facts:

Fact 1: The decomposition of a multibasic $A$-module into basic submodules is unique up to the order of the summands.

Fact 2: If $M_i = L_i / N_i$ and $N_i < L_i \le K$ for $i = 0, 1$, then $M_0 \otimes M_1 = \frac{L_0 L_1}{L_0 N_1 + N_0 L_1}$ and $D(M_0) = c(N_0) / c(L_0)$.

Find the number of distinct isomorphism classes of multibasic $A$-modules $M$ satisfying the following conditions:

(i) $K \otimes \text{End}(M) = K$.

(ii) $F \otimes \text{End}(M) = F$.

(iii) Let $e_r = \dim_F(F \otimes I_r \text{Hom}(I_{>0}, M))$ for all real $r \ge 0$. Then $\lim_{p \to q^-} e_p = e_q$ for all real $q > 0$ except for integers $q$ with $29 \le q \le 328$.

If your answer is infinite, write -1.

Eynard-Orantin Topological Recursion
Problem

Consider the Eynard Orantin Topological Recursion Formalism for the spectral curve $(\mathbb{C}\mathbb{P}^1, x, y, \omega_{0,2}(x, y))$, where $x = t + 1/t$ and $y = t^3 / 3$, and the fundamental bidifferential is given by $\omega_{0,2}(x_1, x_2) = \frac{dz_1 dz_2}{(z_1 - z_2)^2}$, with $z_1, z_2 \in \mathbb{C}\mathbb{P}^1$. Note that $x$ has two simple ramification points at $\pm 1$ of order $2$ with deck transformation $\theta(t) = 1/t$.

Please calculate the Free energies $F_2$ and return it as a rational fraction in the format $a/b$ for $a$ and $b$ coprime. Recall that the free energies $F_g$ can be computed as the following integral $F_g = \frac{1}{2g-2} \sum_{a \in \Delta} \text{Res}_{q=a} \Phi(q)\omega_{g,1}(q)$, where $\Phi(q) = \int_{o}^{q} y(t)dx(t)$, for an arbitrary base point $o$.

Stay Posted
on New Benchmarks