Consortium forVerified Computing

K-0008

Numerical nondeterminism

Differences between runs, or between machines, in the results of the same AI computation, because floating-point rounding depends on the order of operations.

Source reviewed 2026-09-25

01 / The mechanism and its boundary

What is being described

Numerical nondeterminism is variation in the results of the same computation on the same inputs, across repeated runs or across hardware and software setups, that comes from floating-point arithmetic rather than from intended randomness such as sampling S-1010 S-0016.

Floating-point addition is not associative, so a rounded sum depends on the order in which its terms are accumulated S-1009 S-1010. GPUs leave that order, their rounding strategy and their handling of subnormal numbers unspecified, and the same matrix multiplication can give different results on different GPU architectures S-1010. On a single machine, a Thinking Machines post finds the LLM forward pass run-to-run deterministic for a fixed batch, and traces the variation users see to kernels whose results change with batch size, which depends on server load S-1009. Exact replay can therefore need a record of the original hardware model, quantization, parallelism layout, kernels and batch size S-0018. For a verifier, this makes legitimate variation hard to tell from real problems S-0016. Verification designs either tolerate it or remove it:

Connections in the research map

Related research

mechanism / R3

Deterministic and bit-exact inference

Making model inference reproducible bit for bit, so that a verifier's re-run must match the provider's output exactly rather than approximately.

mechanism / R3

Model identity attestation

Establishes that responses come from a specific, committed set of model weights, using enclave measurements or recomputation of sampled outputs.

mechanism / R1

Network taps and certifiers

Devices on a cluster's network links that copy and hash all traffic, so a verifier can later check sampled records against declared work.

mechanism / R3

Sampled inference recomputation

A verifier re-runs a random sample of an AI provider's logged queries on a trusted copy of the declared model and checks the outputs match.

implementation / R2

DiFR (Divergence From Reference)

DiFR checks that an inference provider ran its declared model by comparing output tokens or activations with a trusted re-run using the same random seed.

implementation / R3

Verde and RepOps (Gensyn)

Gensyn's system for checking delegated machine-learning jobs, which settles disagreements between providers by re-running a single operation with bitwise-reproducible operators.

implementation / R3

Pearl proof-of-useful-work blockchain

A blockchain whose mining is designed to be a by-product of GPU matrix multiplications in AI workloads, with public node and miner code.

implementation / R3

TOPLOC

TOPLOC is a hashing scheme from Prime Intellect that lets a verifier check whether an inference provider ran the model, prompt and precision it claims.

Sources and provenance

  1. S-1010 / Tier A

    Hawkeye: Reproducing GPU-Level Non-Determinism ↗

    E. Badash, D. Boneh, I. Komargodski, M. Srivastava · 2026 · Proceedings of Machine Learning and Systems 8 (MLSys 2026)

    Supports: GPU non-determinism arises from unspecified details including rounding strategy, subnormal numbers and accumulation order, since floating-point arithmetic is not associative; results differ between GPU architectures

    Locator: abstract; §1

    Version and catalogue details
  2. S-0016 / Tier B

    DiFR: Inference Verification Despite Nondeterminism ↗

    A. Karvonen, D. Reuter, R. Rinberg, L. Marks, A. Garriga-Alonso, K. Warr · 2025 · ICML 2026 Workshop on Technical AI Governance Research

    Supports: re-running the same inference often gives different results due to benign numerical noise; comparison against a trusted reference conditioned on the same sampling seed

    Locator: abstract

    Version and catalogue details
  3. S-1009 / Tier C

    Defeating Nondeterminism in LLM Inference ↗

    H. He, Thinking Machines Lab · 2025 · Thinking Machines Lab: Connectionism

    Supports: floating-point non-associativity; LLM forward pass run-to-run deterministic; lack of batch invariance with load-dependent batch size as the main cause of nondeterminism in LLM inference endpoints; batch-invariant kernels

    Locator: sections on non-associativity, the concurrency hypothesis and batch invariance

    Version and catalogue details
  4. S-0018 / Tier B

    A System Overview for Near-Term, Low-Trust AI Compute Verification ↗

    N. Cankaya · 2026 · Machine Intelligence Research Institute

    Supports: metadata needed for bit-exact replay (hardware SKU, quantization, parallelism, kernels, batch size)

    Locator: §5.2.2

    Version and catalogue details
  5. S-0029 / Tier B

    What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring ↗

    Y. Shavit · 2023 · arXiv

    Supports: accept a recomputed checkpoint within a small distance of the reported one

    Locator: §5.1

    Version and catalogue details
  6. S-0020 / Tier B

    Bit-Exact AI Inference Verification Without Performance Tradeoffs ↗

    N. Cankaya · 2026 · ICML 2026 Workshop on Technical AI Governance Research

    Supports: software emulation predicting every bit of transformer forward passes across NVIDIA GPU architectures, validated on dense blocks on A100, L40, L40S and H100

    Locator: abstract; evaluation

    Version and catalogue details
Source review date
2026-09-25
Drafted by (source map)
ai
Review handles (source map)
codex-review