AI meets Cryptography 2: What AI Found in OpenVM's zkVM

This is the second post in the series. In case you have not read the first one on Cloudflare's CIRCL, it has more context on why we run these experiments and how our pipeline is set up. In this post, we pointed zkao, our AI auditor, at OpenVM's zkVM, and it found a critical soundness bug in its guest library openvm-pairing that lets a malicious prover forge any pairing equality. Note that this is not a soundness bug in the zkVM's proving system itself; it only affects code that uses the vulnerable library.

The bug in this post was assigned CVE-2026-46669 and fixed in OpenVM 1.6.0. As far as we know, all partners building on OpenVM have since upgraded to that version.

How it happened

Four months ago, we scanned OpenVM as part of our AI experiment, the same way we scan everything at first: an LLM with a simple prompt, and then an LLM with our expert-maintained skills. We ran it with Opus 4.6 and Codex 5.3. As soon as Opus 4.7 and Codex 5.4 came out, we ran them again. The candidate findings were all valid observations, and the models confidently labeled several of them as Critical or High, but none of them were actually exploitable.

Our hypothesis was that a zkVM is simply too complex for a naive LLM setup to handle with 300K tokens or even 1M tokens of context. The dependencies between modules are far denser than in a typical library. A cryptography library can often be audited in parallel by simply handing each subagent a folder that maps to a single cryptographic primitive. Each subagent reads a small number of lines, applies only the relevant skills, writes its findings to a markdown file, and the main agent stitches those files together. All of this happens out of the box with popular agentic coding tools such as Claude Code and Codex, with little human steering.

That approach does not transfer to a more complex codebase, like OpenVM. There, except for the low-hanging fruit, a subagent's useful output is not a list of bugs. You can have a provably secure module A and a provably secure module B whose composition is still not secure. So, hunting for bugs in that "isolated" mode cannot catch meaningful bugs. Instead, a subagent's output should be knowledge about a module: what it assumes, what it delegates to its callers, and what invariant it is silently relying on. However, representing that kind of output well is the hard part. Too short and it skips the implementation detail that the bug actually sits on. Too long and it overflows the main agent's context before it can be combined with anything else. From what we have seen, at least at the time of writing, this problem is not solved efficiently by the agentic coding tools mentioned above.

With that hypothesis in mind, we decided to run zkao on OpenVM, even though our original rule for these experiments was to run zkao only after the plain LLMs had already found a real bug. We have spent a lot of time on context engineering for zkao, and we have encoded the working methods of our own experts into it as reusable flows for finding vulnerabilities, so it seemed like the right tool for exactly this situation. After more than nine and a half hours of scanning, it returned many findings. Similar to the prior experiment, we did not have time to go over every finding in depth. After a quick pass, one stood out immediately: a critical soundness bug in the pairing check in one of the guest libraries. Our hypothesis held up, and months of effort had paid off!

Although there is only one bug to share, to stay consistent with the first post, here is the bug at a glance.

Severities and fixes at a glance

This time, the AI severity and the maintainer severity agree.

Bug 1: openvm-pairing pairing check missing proper subfield check on scaling factor

Background

Pairings are the engine under Groth16, PLONK with KZG, and BLS signatures. In all of these protocols, the verifier is usually not asking for one pairing value. It is asking whether a product of pairings is one:

From that one yes-or-no answer, a verifier concludes that a SNARK proof is valid, that a KZG opening is correct, or that a signature verifies. So if a prover can make a false pairing product appear to be one, everything built on top of it is no longer sound.

A pairing is a bilinear map

where $G_1, G_2, G_T$ are abelian groups. In our case, $G_1$ and $G_2$ are elliptic-curve groups and $G_T$ is a multiplicative subgroup of $\mathbb{F}_{p^{12}}^{*}$.

The most important property of pairing is bilinearity:

This is why pairings are useful, but we will not actually need this property to understand the bug. So you can just ignore it.

Computing a pairing $e(P, Q)$ has two main steps (except for Weil pairing).

The first step is the Miller loop. It evaluates a Miller function $f_{r, Q}(P)$, which we can just view as a black box for simplicity. This stage outputs an element in $\mathbb{F}_{p^{12}}^{*}$. For a product of pairings, the circuit can run all the Miller loops and multiply their outputs together. Let us call that combined output $f$, i.e. $f = \prod_i f_{r, Q_i}(P_i)$.

The catch is that $f$ is not the pairing product yet. It is only one representative of the equivalence class $\mathbb{F}_{p^{12}}^{*} / (\mathbb{F}_{p^{12}}^{*})^r$. This is one of the main observations from Novakovic and Eagen's paper: Miller-loop outputs are unique only up to multiplication by an $r$-th power. In other words, $f_1$ and $f_2$ stand for the same pairing whenever there is a non-zero $c$ with $f_1 = f_2 \cdot c^r$. That undetermined factor $c$ is what makes a direct equality check tricky.

That is why the second step exists. The final exponentiation raises $f$ to

This removes the ambiguity, because for every non-zero $c$, we have:

The last term disappears because every non-zero element of $\mathbb{F}_{p^{12}}$ satisfies $x^{p^{12}-1} = 1$. After this exponentiation, the result lands in $G_T$, the group of $r$-th roots of unity. So the real pairing-product check is:

The problem is that the exponent $h$ is far too expensive to raise to inside the circuit, while it is fine to let the prover compute $c$ outside the circuit and pass it in as a hint. Checking $f = c^r$ is much cheaper than computing the $f^h$ exponentiation. So to prove $\prod_i e(P_i, Q_i) = f^h = 1$, instead of computing $f^h$ directly, the prover only needs to provide a non-zero $c$ with $f = c^r$, which holds exactly when $f^h = 1$.

This is the core idea behind the optimization. OpenVM implements it with the residue-witness trick from Novakovic and Eagen's paper: the prover supplies a few extra values and the circuit checks a cheap equation instead of running the full exponentiation.

The actual optimized equation is slightly different from $f = c^r$:

Here $\lambda = m \cdot r$ is a curve-specific exponent whose structure lets the circuit evaluate $c^\lambda$ cheaply through the Frobenius map, and $u$ is called the scaling factor. In the OpenVM code this scaling factor is called u for BN254 and s for BLS12-381. The remaining symbols are $d = \gcd(m, h)$ and $i = v_d(h)$.

The scaling factor is needed because $\lambda$ does not line up perfectly with the original $r$-residue check. Theorem 3 of the paper closes that gap by requiring the scaling factor to satisfy a small root-of-unity relation, which is the $u^{d^i} = 1$ condition above.

For these particular curves, the paper gives an even cheaper route: instead of checking $u^{d^i} = 1$ directly, it is enough to restrict the scaling factor to the proper subfield $\mathbb{F}_{p^6} \subset \mathbb{F}_{p^{12}}$. And this check is very cheap. OpenVM stores an $\mathbb{F}_{p^{12}}$ element as six $\mathbb{F}_{p^2}$ coefficients:

Being in the $\mathbb{F}_{p^6}$ subfield means the odd-indexed coefficients are zero:

So the whole security-critical subfield test is just three equality checks.

The bug

Recall the shape of the check. The circuit has a combined Miller-loop output $f$, and it should accept only when the pairing product would become one after final exponentiation. OpenVM avoided that expensive exponentiation by checking the optimized relation:

But that relation is sound only if $u$ is constrained to the right subfield. OpenVM only checked that the hint $c$ was non-zero, and stopped there. It did not check that the scaling factor was in $\mathbb{F}_{p^6}$. Here is the BLS12-381 code path before the fix:

// guest-libs/pairing/src/bls12_381/pairing.rs
let (c, s) = Self::pairing_check_hint(P, Q);
// ... no check that s lies in Fp6 ...
let c_conj = c.conjugate();
if c_conj == Fp12::ZERO {   // only rejects c == 0
    return None;
}

BN254 had the same pattern, just with if c == Fp12::ZERO { return None; }. So the circuit rejected a zero $c$, but accepted any scaling factor.

At that point the optimized equation no longer proves the real pairing check. For any Miller output $f$, even one coming from a false pairing equation, the prover can set:

Then $c^\lambda = 1$, and the checked relation becomes:

The check passes. The same substitution works on both curves: $c = 1, u = f^{-1}$ for BN254 and $c = 1, s = f^{-1}$ for BLS12-381.

Normally $f^{-1}$ is a full $\mathbb{F}_{p^{12}}$ element, not an element of the $\mathbb{F}_{p^6}$ subfield. That is exactly why the subfield check would have rejected the forgery.

There is also a subtle control-flow detail. Because the optimized routine returned success, the slower fallback that would have performed the full final exponentiation was never reached.

Impact

Forging an arbitrary pairing check breaks the cryptographic floor that a lot of things stand on:

• On BLS12-381, a prover can forge KZG opening proofs, which breaks the polynomial commitment schemes behind data availability, blob verification, and PLONK or KZG verifiers.
• On BN254, the same forgery breaks Groth16 SNARK verifiers, BLS signature checks, and any bridge or protocol that relies on a pairing equation.
• Any zkVM guest emulating the Ethereum ecPairing precompile at address 0x08 through OpenVM's pairing check would produce forged results, leading to incorrect EVM execution.

So any L2 rollup, bridge, or privacy protocol that verifies a pairing inside an OpenVM guest program inherits the problem. zkao rated it Critical, and the OpenVM maintainers confirmed it as Critical.

The fix

The fix was to add the missing subfield membership test. It asserts that the odd-indexed coefficients of the scaling factor are zero:

// the scaling factor is an honest hint only if it lies in the subfield Fp6
for i in [1, 3, 5] {
    if s.c[i] != Fp2::ZERO {
        return None;
    }
}

A scaling factor like $f^{-1}$ usually has non-zero odd coefficients, so it is rejected and the forgery disappears. The fix landed in commit a720e2c and shipped in OpenVM 1.6.0.

A few things we learned

A naive LLM still hits a wall on a complex codebase. This is the observation we opened with, and the experiment confirmed it: across two model generations, the plain LLM passes produced findings that all triaged down to Informative on OpenVM. The reason is the one described above. A zkVM does not decompose into independent primitives the way a library does, so the unit of work an agent has to pass upward is knowledge about a module, and that is genuinely hard to represent.

Triage cannot just ask an LLM to produce a working PoC. One of our first attempts at automated triage was to ask whether the LLM could turn its own report into a working PoC. The idea was simple: if the PoC runs, the bug is real. In practice, that signal was much weaker than it looked. The model was very good at producing PoCs that passed, even when the reported issue was not real. Enough nonsensical comments, hidden assumptions, patched helper functions, disabled checks, mocked state, and suspicious run flags can make almost any exploit look successful. Validating those PoCs ended up taking more time than triaging the original report. This is why we have spent much time since then on zkao and have improved its triaging process to reduce false positives. In another post, we will discuss how we achieved that and how zkao also learns from users who triage bugs, so that after the first few triages in a project, false positives are reduced even further.

What's next

Our thanks to the OpenVM team, who triaged and fixed this quickly and shipped the fix in 1.6.0. We thank Yi Sun, co-founder of Axiom, for valuable feedback on this blog post.

This is the second post in the series. We will keep publishing confirmed bugs from other projects as they are resolved, especially when they teach us something useful where AI auditing works, where it still fails, and what kind of human review is needed around it.

If you maintain a zkVM or a cryptography project and this is interesting to you, we would love to look at it alongside you, so that serious bugs get found and fixed before they ship. Continuous AI coverage of this kind is exactly what zkao is built for. Reach out at zksecurity.xyz/contact.