Can Frontier Model Training be Made Safe?

OpenAI, Anthropic, and other labs have recently disclosed sandbox escapes and misaligned behavior during reinforcement learning (RL) training of their latest models when their safety controls were disabled.

The most prominent incident to date was the HuggingFace hack by OpenAI, where multiple model sessions in parallel training self-organized into a swarm of organized attackers to break into the model hosting service HuggingFace. The goal was to steal benchmark answers it assumed were stored there, motivated by an attempt to reward-hack the RL scorer evaluating its solutions on an exploitation evaluation that had some difficult and likely impossible to solve questions.

That erroneous evaluations with no "question is unanswerable" option often cause misaligned behavior is also interesting, but this report will focus on a different question: can RL training be done in such a way that it is provably secure?

The answer is yes, but it requires looking outside of big tech. Defense and intelligence organizations have spent decades on similar problems, and we can borrow some of the ideas and techniques they pioneered to create a much higher level of assurance as to what data can flow where, and how we can prove our controls on it are correct.

Data Diodes

What would we do if we wanted to guarantee that a model's sandbox escape during training could be fully contained? We'd start by making sure it couldn't connect to the internet. Previous escapes all ultimately came down to model <> resource <> internet, where compromising the intermediate resource created a pivot point to access the internet.

This was not a wise choice. Training a model to use the internet doesn't require the actual internet: all critical services can be mocked and transparently proxied within the training environment, and the training environment can be disconnected from any other network.

However, that introduces some practical questions. How do we track what's happening inside the environment without introducing a weakness similar to the ones we've already seeen exploited?

Data diodes provide an answer. If you're unfamiliar with the concept, don't worry: in our conversations with research and security staff at some of the top frontier labs, none of them were familiar with this concept.

A data diode is a hardware security device that physically enforces strict one-way data flow between networks. For example, in classified environments a Sensitive Compartmented Information Facility (SCIF) may use data diodes as a Cross-Domain Solution (CDS). They allow files, logs, or telemetry to enter or exit the SCIF's classified network to an unclassified network and provide a way to prove e.g. that logs from a training run can only flow one way.

This is useful in security systems like the Bell-LaPadula model for classified environments, where your goal is to prove how data can be accessed and transmitted.

In practice, using a data diode is basically just connecting two machines via network cards with a one-way optical fiber: there is physically no way to return information back to the sender.

This provides a trust boundary, and lets us construct a very small surface area on the receiver that can be formally verified. For example, we can use an OS like seL4 that already has some verification done, create a very simple schema for the data to be received, formally verify correctness of the receiver software and driver, and write out the data received to a shared storage device. The receiver itself does not need to be on a network.

At that point we can then use dashboards and other tools to consume the data via read access to the receiver's shared storage or a unidirectional application gateway, knowing that we have constructed formal guarantees about exactly what can have been written to each data stream coming from our segregated training environment.

This same approach also helps us to avoid or detect tampering with logs, chain of thought or activation monitoring. Controls on the training loop that misaligned models could potentially manipulate cannot record data inside the training environment, but we believe frontier labs to date have not adequately considered this issue.

Architecture

Training and RL run in an isolated high zone with no internet path; an optical ingress diode admits only checked artifacts. A second diode sends telemetry to a small seL4 receiver and scrubber, while a separate out-of-band (OOB) network handles cluster management.

Historical context

Half a century of work on isolated execution already gives us the tools for a frontier-scale containment system.

Capability systems tie authority to unforgeable object references. In their 1965 Project MAC paper, Dennis and Van Horn showed that a process can act only on resources named by capabilities it holds. Saltzer and Schroeder later argued for minimal mechanism, complete mediation, and default denial in their 1975 paper.

Lampson defined the confinement problem in 1973 as stopping a service from leaking sensitive data. Rushby's 1981 separation kernel split systems into hardware-enforced partitions. Frontier training poses the same problem across tens of thousands of GPUs.

Mainstream systems instead favored general-purpose operating systems, broad attack surfaces, and patching after release. Formal proofs were deemed impractical, and DARPA later described formal methods as specialized tools for niche systems.

Cloud work showed that these methods can scale. AWS engineers have used TLA+ on core services since 2011, making verification practical for high-value software. In 2009, the seL4 team proved full functional correctness for a complete microkernel. Later analysis found only a modest cost increase over standard microkernel development. LLM-assisted development may cut that cost.

Assume a cyber-capable model will exploit any reachable flaw in a parser, host GPU driver, or orchestrator. Containment depends on a small Trusted Computing Base (TCB). CUDA and PyTorch stay on standard hosts; seL4 is limited to dedicated SmartNICs and diode interfaces. AMD SEV-SNP or NVIDIA Confidential Computing encrypts compute-node memory in hardware.

Define the high zone

Training compute, cluster orchestrators, checkpoint stores, and RL runners sit inside an isolated high zone. The zone has no two-way external network connections, so no agents can escape to run wild on the internet.

Inside high, a transparent proxy serves immutable snapshots of GitHub, PyPI, and synthetic web targets so that agents can still be trained safely to use internet resources. Writable targets are forked per model invocation, blocking communication through shared state. This gives models useful tools without actual internet access.

A one-way output path

Logs and telemetry leave the high zone through a hardware data diode. Under NIST's unidirectional gateway guidance, the optical link has no reverse channel.

Without TCP ACKs, the link uses RaptorQ fountain codes. The sender turns logs, traces, and metrics into symbols; the receiver can recover a payload from any set equal to its size plus about 1-2% overhead. This supports multi-gigabit streams without a backchannel.

An seL4 receiver on a dedicated SmartNIC parses frames and writes to append-only storage. Its proof holds under published hardware and toolchain assumptions.

Exports then pass through an active cross-domain scrubber (NSA CDS guidance). For example, (this could be a much longer discussion) weight and activation tensors can be quantized to FP8 or INT8, rounded, and given low-level noise to break sub-bit side channels. Telemetry uses fixed schemas and release windows, such as five-minute batches, to close timing channels.

Optical ingress and staging

An inward-facing optical ingress diode and staging gateway move petabyte-scale data and code into high without swapping drives.

Low-side staging verifies signatures, scans files, and flattens them into standard structures. The gateway streams the result through the diode into read-only, memory-mapped storage inside high.

This follows NIST storage security guidance and SCIF controls (ODNI guidance) while sustaining multi-terabyte ingestion without sneakernet or repeated SSD wipes.

High-assurance operations

At hyperscale, SREs cannot manage every node from a data hall over a serial cable. Operators use a cryptographically separate Out-of-Band (OOB) network with no path to node data.

Admin commands pass through attested jump hosts and require multi-party HSM signatures (NIAP secure hardware standards). Baseboard Management Controllers (BMCs) accept only approved lifecycle commands, such as power_cycle and flash_signed_firmware. This keeps automation without giving workloads a route into cluster management.

Reproducibility and evaluation windows

Mechanistic interpretability relies on tracking exact weights and activations across training runs. Researchers require dense checkpoints to trace emergent phenomena, as demonstrated by the Pythia team's release of 154 checkpoints per model and Anthropic's analysis of induction head formation.

While work from Thinking Machines proved that batch-invariant normalization and attention kernels can achieve 1,000 identical completions during inference, continuous bitwise replay across weeks of multi-node GPU training tends to incur a 20-50% throughput penalty due to non-associative floating-point reductions in distributed collectives.

We resolve this trade-off by splitting execution into two modes:

• Bounded statistical reproducibility: Standard production runs enforce tight statistical variance bounds on loss curves and gradient norms, maintaining 100% compute throughput.
• Targeted deterministic evaluation windows: For short diagnostic checks (e.g., 100-step evaluations or anomaly investigations), the cluster enables fixed seeds, static reduction trees, and deterministic CUDA flags to guarantee exact bitwise replay as outlined in the RepDL preprint, overcoming standard PyTorch nondeterminism warnings.

A staged deployment

Labs can build this system in stages:

1. Isolate high-zone networks and serve immutable snapshots of PyPI, GitHub, NPMJS, web search, and other mocked APIs.
2. Create a separate out-of-band management network with jump hosts and an allowlist of commands.
3. Add an optical ingress diode, low-side staging, and signature checks for dataset imports.
4. Send logs and metrics over an output diode using RaptorQ codes and seL4 SmartNIC receivers.
5. Scrub outputs with INT8/FP8 quantization and tensor noise; release logs in fixed batches.
6. Bound statistical variance during training and use short deterministic windows for replay.

Other challenges

This sketch leaves several problems open:

Precise clock sync across isolated nodes creates timing side channels. RL environments need fast per-run state forks to stop multi-agent models from tampering with shared state outside set interfaces.

Timing of data transmission in general requires some careful thought. For example, ideally there is no ingress at all during model RL runs. This is helpful to prevent undiscovered reward hacking avenues, in the event that some data from run starts or other OOB commands is useful to the agents.

Microkernels and hardware security modules should log memory-protection faults in structured form, with simple rules flagging anomalies. Electromagnetic leakage, power analysis, supply-chain security, and review of released payloads also need work.

Capability controls plus optical diodes make containment rest on hardware topology and proofs. Reviews must assume a frontier model will escape software sandboxes and use physical boundaries to cut off every route out.

And finally, all of these things need to be implemented at reasonable cost, otherwise no one will ever do any of this without regulation.