Testnet 8 NEAR AI TEE runs, all AUDITED

Audit a private benchmark score by opening 20 of 100 entries.

The owner commits a Merkle root of a private test set, then each graded run, to a NEAR contract. Any account can then call the draw, and random_seed picks 20 of 100 entries. Each one must match its commitment, carry a pinned NEAR AI TEE signature and re-grade the same, or the run fails.

Open the explorer Run the checks yourself
Real runs audited
–
Drawn entries checked on-chain
–
NEAR AI signers pinned
–
Reveal transactions
–
sealedbench-v2-hk26.testnet replay of contract state
  1. 01CommitSalted Merkle root and grade vector
  2. 02DrawAny caller; random_seed picks 20
  3. 03RevealDrawn entries with proofs
  4. 04Verifyecrecover and re-grade
Replay of a real audit on the contract: 100 entries are committed, the draw picks 20, the owner reveals them, and the contract checks each signature and settles the run as AUDITED.

Problem

Published test sets can end up in training data.

A

Public sets leak

Published benchmarks get scraped into training corpora. A high score can then come from memorized answers.

B

Private scores rest on trust

An unpublished set limits exposure. Outsiders then have only the reported number.

C

Sealed Bench

The set stays private. After the runs are committed, the owner must reveal a random 20 of 100 entries, and the contract checks each one.

Protocol

How a run is audited

Explorer

Runs on the testnet contract

Status, score, grades and draw are contract views read over public RPC. Question and response text come from the published reveal files.

Result

Public problems vs. sealed variants

Each sealed suite holds 100 numeric variants of 25 public problems, 4 per problem, with answers computed in code. Temperature 0. The hypothesis and analysis were pre-registered before inference. On the AIME-style set no gap is detected for either model: Qwen scored higher on the sealed variants, and GLM's +11 points has a 95% interval that includes 0. Most sealed misses are answers cut off at the 4,096-token limit, which grade as wrong (operator's count; only drawn responses are public). The GSM8K control sits at the ceiling.

Tamper lab

Edit a signed entry and submit it

Each submission calls the contract's free check_reveal view, which returns the first failed check.

Detection odds

How often a misreport is caught

With 10 bad entries out of 100, the draw catches the run 90.5% of the time. With 2, 36%. Across 10 suites at 10% misreporting, at least one gets through 63% of the time. These figures assume an unbiased draw and one suite per test set.

Hypergeometric, per run, 20 distinct entries of 100. This is not the probability that the score is correct.

90.5%chance the draw includes a bad entry

NEAR

What runs on NEAR

TEE-signed inference

NEAR AI Cloud runs models in Intel TDX and NVIDIA confidential VMs and signs each response with an enclave key. The signature covers the gateway's model id. It does not identify the weights.

GET /v1/signature/{chat_id}
// EIP-191 personal_sign over:
model_id + ":" + sha256(request) + ":" + sha256(response)

On-chain ecrecover

The contract rebuilds the request from the committed template and recovers the signer.

// contract/src/verify.rs
let pk = env::ecrecover(&hash, &sig[..64], v, false).ok_or("ecrecover failed")?;
let h = env::keccak256_array(&pk);
addr.copy_from_slice(&h[12..]);
// contract/src/lib.rs
if !run.signers.iter().any(|p| *p == addr) {
    fail!("signature not from a signer pinned for {} when this run started", run.model);
}

random_seed

Sets each run's nonce at start_run and picks the sample at the draw. Reveals must land in a later block than the draw.

Pinned signers

The admin pins a key after checking its TDX quote off-chain with dcap-qvl. The contract does not verify the quote. Each run copies the pins at start_run, so a key pinned later does not count for it.

check_reveal

Runs the reveal checks as a free view. The explorer and tamper lab use it.

Threat model

Trust assumptions and limits

You trust

  • NEAR AI Cloud to keep each signing key inside its enclave. The provider sees every sealed entry inside the TEE.
  • The operator. Owner, admin and drawer are separate accounts run by one operator. The admin pins keys after an off-chain check of the DCAP quote, report data, MRCONFIGID and RTMR3.
  • The gateway's model id. The signature does not identify the weights.
  • Shared keys. All instances of a model use one signer, so a match does not identify the serving instance.
  • The upgrade key. The contract account still has a full-access key. npm run verify compares the deployed code hash with the published wasm. Until the key is deleted, the operator can deploy code that changes stored results.

A passing audit shows

  • The revealed questions and answer keys were committed before the run.
  • Each revealed response used this run's nonce and was signed by a key in the signer set fixed at start_run.
  • For each revealed entry, the committed grade matches a deterministic re-grade of the raw response.
  • The score equals the sum of the committed grade vector.

It does not show

  • Unsampled entries. They stay unchecked. The odds above are the only coverage.
  • Re-rolls. The nonce binds signed responses to the run. Best-of-N within the run is still possible.
  • Small misreports. 2 bad entries are caught 36% of the time.
  • Suite shopping. v2 rejects duplicate roots and lists every run, including FAILED and Abandoned ones, in owner_record. An owner can still re-salt a set under a new root and try again.
  • Secrecy from the owner. The owner sees and grades the set.
  • An independent draw. Block producers can bias random_seed slightly. Every draw here was called by sealedbench-drawer-hk26.testnet, which is not the owner but is run by the same operator.

Revealed entries are public and must be retired from the private set. The 0.1 NEAR bond returns to the owner only if every run is AUDITED. Otherwise it goes to the account that drew or expired the suite. The mock signer's key is public; mock runs are labeled mock/DEV-ONLY-not-a-TEE and no claim cites them.

Reproduce

Check it from a clean clone

Needs Node 22.18+ and testnet RPC. No account or key. About two minutes.

  1. npm run verify Reads roots, grades, nonces and signers from the chain at one block, runs check_reveal on all 20 drawn entries per run, replays them offline and checks the deployed code hash.
  2. npm run tamper Applies 13 edits to a signed entry and labels each result as live contract view, offline what-if or unit-tested rule. It also lists 3 attacks that still succeed.
~/sealed-bench 
$ git clone https://github.com/jazulay/sealed-bench
$ cd sealed-bench && npm ci

$ npm run verify
# expect "ok": true, 8 results, every check true

$ npm run tamper
# expect "13/13 lies rejected (...); honest reveal accepted; 3 known limits"

The README covers the attestation check.

The contract, CLI and fixtures are on GitHub under MIT.

Source Contract on NearBlocks