Consortium forVerified Computing

M-0006 / Cryptography & computation

Training-transcript verification (proof-of-learning)

A trainer logs checkpoints, data order and settings, so a verifier can re-run sampled training segments and check that the claimed training happened.

R2 DemonstratedSource reviewed 2026-09-25

01 / The mechanism and its boundary

What the technique establishes

Proof-of-learning asks a model trainer to keep a transcript of training: weight checkpoints, the data used at each step and the settings. A verifier re-runs a few segments and checks that they reproduce the logged checkpoints. It was proposed to show that compute was really spent on training, and later to check that training runs obeyed rules on compute and data. Later data-focused tests were run on language models of up to 1 billion parameters. A 2024 variant makes small training runs exactly reproducible across GPU types, so an auditor can replicate them. The biggest known weakness is that published attacks produced invalid proofs that passed verification, and the original authors later argued that no provably robust version exists without a better theory of deep-learning optimisation. The biggest obstacle for verification use is that the verifier must see the training data and weights and be able to re-run training.

Threat model
Adversarial prover
Adversarial evaluation
Independent red-team
Hardware needed
None
Prover cooperation
Required
Confidentiality
Revealing
Category
Cryptography & computation

Claims and scope

A direct link identifies the intended claim. A supporting link supplies part of the evidence. Neither establishes that a complete verification system has been demonstrated.

Readiness for a stated use

R2 Demonstrated

Assessed use: checking from its transcript that a training run followed declared rules

medium confidence · current · assessed 2026-09-25 · rubric 1.1

This is the source map’s editorial assessment. Production use is not evidence of resistance to every adversary.

R2 through later peer-reviewed work by Choi et al. and Srivastava et al. The original verification rule is broken, and no independent attack on the later defences has been published.

  • R1 met: Jia et al. define proof-of-learning and its security goal S-0028, and Shavit sets out training-transcript verification for rules on large training runs S-0029.
  • R2 met through later work, not through the original scheme. The original authors demonstrated spoofs that pass verification across configurations at a fraction of the cost of earlier spoofing strategies S-0027. An earlier independent attack reported spoofs cheaper than honest training S-1109, which the original authors dispute S-0027. These breaks invalidate the original demonstration, including the public code, which implements that rule S-1111. Choi et al. publish peer-reviewed end-to-end experiments against a stated set of spoofing attacks, training GPT-2 (124M) and testing published Pythia checkpoints up to 1B on GPUs S-0030. The paper links no code; the rubric does not require code for results described in enough detail to repeat. Srivastava et al. replicate ResNet-50 training and GPT-2 fine-tuning bit for bit on three NVIDIA GPU types, against a trainer that may poison data, and release their code S-3081. The open critical flaws concern the original rule, not the evidence the level rests on.
  • R3 not met: Verde realises this mechanism for delegated training jobs, but Gensyn states that its public runtime is limited to LLM inference S-1812, so its production use does not count for training verification. As of September 2026 no third party is known to rely on training-transcript checks, and no production-grade implementation has been published.
  • R4 not met: no independent attack on the post-2023 defences has been published.

Evidence needed for the next level

  • Use by a party other than the developer, or a production-grade implementation, at realistic training scale.

  • Verification rules with formal robustness arguments, as Fang et al. argue are needed, or an independent red-team of the post-2023 defences.

Limitations, flaws, and blockers

These are attributed assessments from the source map. Absence of a listed flaw is not a security guarantee.

significant / disputed / demonstrated attack

Adversarial-example spoofs pass verification at lower cost than training

Zhang et al. construct proofs that pass the original verification "with significantly less cost than generating a proof by the prover". Their attack uses adversarial-example-style perturbations, and they demonstrate it on CIFAR-10, CIFAR-100 and an ImageNet subset S-1109. They suggest dynamic thresholds, or proofs built on verifiable computation, as countermeasures S-1109.

S-1109S-0027
Response recorded by the source map

Fang et al., who include the original PoL authors, state that the attack assumes the adversary sets the checkpoint interval k, which the verifier should set, and that the verifier can prevent it by using a small k S-0027.

critical / open / demonstrated attack

Structurally correct spoofs exploit tolerance thresholds and sampled checks

Fang et al., including the original proposers, present spoofing strategies that work across PoL configurations at "a fraction of the cost of previous spoofing strategies" S-0027. The strategies exploit the tolerance that verification must allow for hardware noise, and the choice of which updates the verifier re-runs S-0027.

S-0027

significant / open / theoretical argument

No provably robust verification without better optimisation theory

Fang et al. conclude that "one cannot develop a provably robust PoL verification mechanism without further understanding of optimization in deep learning" S-0027. Shavit notes that the PoL literature "has been heuristic-based" S-0029.

S-0027S-0029

significant / open / theoretical argument

Rule-compliance transcripts face a stronger adversary

Shavit argues that proving a training transcript "appears to be strictly harder than PoL". The adversary knows the true transcript and may spend extra compute to build a compliant-looking one S-0029.

S-0029

significant / open / open question

Small data changes and masked hyperparameters may go undetected

Choi et al. state that their protocol cannot yet detect modest data additions, such as inserted backdoors. They note that attacks could be hidden with "cleverly chosen hyperparameters", such as a temporarily lower learning rate than reported, and that the protocol does not apply to online or reinforcement learning S-0030.

S-0030

What still blocks use or stronger assurance

  1. The verifier must receive the training data, weights and code S-0028 S-0030.

    S-0028S-0030

  2. Transcripts are large: weight checkpoints may each require terabytes S-0029.

    S-0029

  3. The verifier must reproduce training segments, which may be infeasible if the prover uses specialised or proprietary hardware S-0030.

    S-0030

  4. The noise tolerance needed for honest reproduction is what structurally correct spoofs exploit S-0027.

    S-0027

  5. Tying transcripts to real chips needs on-chip weight-snapshot logging, chip inspections and a trusted chip-owner directory S-0029.

    S-0029

Connections in the research map

Alternative approaches

Concepts used

Implementations

implementation / R3

Verde and RepOps (Gensyn)

Gensyn's system for checking delegated machine-learning jobs, which settles disagreements between providers by re-running a single operation with bitwise-reproducible operators.

The Consortium’s case files

Related editorial reviews use the Consortium’s own descriptive scores and review dates. Their scores are separate from the atlas readiness rubric.

PT-07 / AttestationProof that training happened properlyRead case file ↗

Sources and provenance

  1. S-0028 / Tier A

    Proof-of-Learning: Definitions and Practice ↗

    H. Jia, M. Yaghini, C. A. Choquette-Choo, N. Dullerud, A. Thudi, V. Chandrasekaran, N. Papernot · 2021 · 2021 IEEE Symposium on Security and Privacy (SP), pp. 1039-1056

    Supports: PoL definition, transcript contents, verification, security analysis, limits

    Locator: Definition 1; Algorithm 2; §IV–VI; Table I

    Version and catalogue details
  2. S-1111 / Tier B

    Proof-of-Learning: code for Proof-of-Learning: Definitions and Practice ↗

    CleverHans Lab · 2021 · GitHub

    Supports: public PoL implementation (train and verify scripts); rationale for the cost of forging

    Locator: README

    Version and catalogue details
  3. S-1109 / Tier A

    "Adversarial Examples" for Proof-of-Learning ↗

    R. Zhang, J. Liu, Y. Ding, Z. Wang, Q. Wu, K. Ren · 2022 · 2022 IEEE Symposium on Security and Privacy (SP), pp. 1408-1422

    Supports: independent demonstrated spoofing attack; countermeasures

    Locator: abstract; §III–V

    Version and catalogue details
  4. S-0027 / Tier A

    Proof-of-Learning is Currently More Broken Than You Think ↗

    C. Fang, H. Jia, A. Thudi, M. Yaghini, C. A. Choquette-Choo, N. Dullerud, V. Chandrasekaran, N. Papernot · 2023 · 8th IEEE European Symposium on Security and Privacy (EuroS&P 2023)

    Supports: structurally correct and stochastic spoofing; robustness conclusion; response to Zhang et al.

    Locator: abstract; §2.3; attack sections; conclusion

    Version and catalogue details
  5. S-0029 / Tier B

    What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring ↗

    Y. Shavit · 2023 · arXiv

    Supports: training-transcript verification for rules; chip logging; open problems; costs

    Locator: §4; §5; §5.1; §5.2; §6.1; Table 1

    Version and catalogue details
  6. S-0030 / Tier A

    Tools for Verifying Neural Models' Training Data ↗

    D. Choi, Y. Shavit, D. K. Duvenaud · 2023 · Advances in Neural Information Processing Systems 36 (NeurIPS 2023)

    Supports: proof-of-training-data protocol, experiments, defended attacks, costs, limits

    Locator: §3; §4.1–4.4; §6; §7; App. A

    Version and catalogue details
  7. S-3081 / Tier A

    Optimistic Verifiable Training by Controlling Hardware Nondeterminism ↗

    M. Srivastava, S. Arora, D. Boneh · 2024 · Advances in Neural Information Processing Systems 37 (NeurIPS 2024)

    Supports: exact-replication verifiable training: rounding logs, Merkle-tree dispute search, experiments, overheads, 1-of-n auditor assumption, limitations

    Locator: abstract; §3–§5; limitations

    Version and catalogue details
  8. S-1809 / Tier B

    Verde: Verification via Refereed Delegation for Machine Learning Programs ↗

    A. Arun, A. St. Arnaud, A. Titov, B. Wilcox, V. Kolobaric, M. Brinkmann, O. Ersoy, B. Fielding, J. Bonneau · 2025 · arXiv

    Supports: Verde dispute narrowing for training jobs; RepOps LoRA fine-tuning overhead

    Locator: §3; Table 2

    Version and catalogue details
  9. S-1812 / Tier B

    gensyn-ai/ree: Gensyn Reproducible Execution Environment (GitHub repository) ↗

    Gensyn · 2026 · GitHub

    Supports: Gensyn's REE release limited to reproducible LLM inference (provider-reported)

    Locator: README

    Version and catalogue details
  10. S-0022 / Tier A

    Trustless Audits without Revealing Data or Models ↗

    S. Waiwitlikhit, I. Stoica, Y. Sun, T. Hashimoto, D. Kang · 2024 · 41st International Conference on Machine Learning (ICML 2024)

    Supports: cryptographic alternative (ZK proofs of SGD)

    Locator: abstract

    Version and catalogue details
Source review date
2026-09-25
Drafted by (source map)
ai
Review handles (source map)
codex-review