Closing the Operator Access Gap That Encryption at Rest and in Transit Leave Open

TEE-backed AI assistants use hardware-enforced isolation to protect prompts during inference, when encryption at rest and in transit no longer shields them from the host. Closing that operator-access gap requires both confidential computing and independently verifiable attestation, so users can independently establish what processes their data.

Why the standard two layers of encryption leave a gap during computation

“Your data is encrypted at rest and in transit, but what protects it while it is being processed?” Encryption at rest keeps data safe if a drive is stolen or a backup is leaked, but once the operating system mounts the volume, privileged processes can read plaintext. Encryption in transit protects against a network interceptor, but TLS protection ends where the connection terminates. Neither layer excludes the operator running the application.

Computation introduces a third state for the data. The application must decrypt a value to act on it, leaving that value in process memory accessible to anyone with sufficient host privileges. At that point, both protections have been bypassed without either failing at its intended job. Stronger disk encryption or another protected network connection cannot make those protections reach into RAM. The gap belongs to their threat model.

What "operator access" actually means and why it cannot be solved by trusting harder

Who, precisely, has access inside that gap? The operator has the privileges that come with running the infrastructure, including control over the host operating system or hypervisor. Root or hypervisor access to the physical host can, in principle, permit inspection of a workload’s decrypted memory. Administrators can do this as part of their normal work, without an exploit. An attacker who compromises those layers can inherit the same privileged position.

Shared infrastructure adds another route to exposure. Virtualization separates tenants through software, but hardware side-channel attacks such as Meltdown, Spectre, and L1TF have shown why software isolation alone is insufficient for high-assurance workloads. A co-tenant exploiting a side channel belongs in the threat model because separation between workloads may not prevent it from learning protected information. The architecture must account for every party that can reach plaintext, regardless of job title.

IAM policies, access logs, and contracts can reduce the likelihood of abuse, but a contract cannot revoke a kernel’s memory-read capability. That distinction separates an assurance about how access will be used from a design that removes access. For AI inference, the distinction is especially consequential because prompts, model weights, and outputs pass through memory during computation.

How a Trusted Execution Environment moves the trust boundary inward at the silicon level

A Trusted Execution Environment, or TEE, relocates decryption inside a hardware-enforced boundary. Think of a locked room inside a data center whose owner has a key to every corridor but not to that room. The room can also produce evidence of what is happening inside without opening its door. The analogy has limits, but it captures the shift. Ownership of the surrounding infrastructure no longer implies permission to inspect the protected computation.

Three properties turn that separation into a technical protection. Hardware-enforced memory isolation blocks the host operating system, hypervisor, and other workloads from reading the protected region. Memory encryption protects allocated RAM with keys held by the CPU, so an external memory dump returns ciphertext. Decryption takes place inside the protected boundary, where the data is processed. This is not computation on ciphertext, and it is not simply more careful encryption before computation begins.

Remote attestation supplies the third property. The hardware gives a remote party signed cryptographic evidence of the code and configuration inside the protected environment, so they can check them without taking the operator’s word for it. That evidence supports a workflow in which secrets are released only after the configuration passes verification. The protected code, the hardware guarantees, the attestation process, and secret provisioning still have to be trusted. It no longer has to encompass the entire operator stack.

The scope of protected code depends on the design. An application enclave can isolate a narrow component, while a confidential virtual machine, or CVM, protects an entire VM. Sabanic et al.’s Confidential Serverless Computing describes CVMs based on AMD SEV-SNP, Intel TDX, and ARM CCA as providing “hardware-based security properties” for confidentiality and integrity without significant application modifications. Hardware isolation can keep the host out, but the guest’s own software still has to be trusted.

The hardware implementations in production and what each one protects

Intel SGX uses an application-level enclave model. Its fine-grained isolation protects only the chosen code and data within a virtual machine, which makes it a fit for small components that handle highly sensitive data. The trade-off is a more restrictive programming model and tighter memory constraints. The engineering choice begins with how much of the application needs protection.

AMD SEV-SNP instead encrypts the memory of an entire virtual machine. Its Reverse Map Table adds hardware-enforced memory integrity so the hypervisor cannot read or silently modify guest RAM. Existing VM-based applications therefore have a path closer to lift-and-shift deployment, although the protected boundary encompasses the entire VM. Intel TDX takes the same whole-VM approach with its own threat model and attestation chain. Its encrypted Trust Domain deliberately excludes the host hypervisor from the trust boundary.

AWS Nitro Enclaves provides AWS-native isolated compute environments. AWS states that even root users or administrators on the parent instance cannot access or SSH into the enclave. Each enclave has CPU and memory set aside for it but no persistent storage or external network interface, and it talks to its parent through a secure local channel called vsock. That restricted interface is part of its design. ARM CCA and Realms provide another approach for cloud, server, and edge environments within the broader CVM landscape.

AI inference also needs protection beyond the CPU. NVIDIA H100/H200 Confidential Computing extends the protected boundary to the GPU, where hardware access-control firewalls isolate model weights in VRAM, which remain unencrypted at runtime. Data crossing the PCIe bus is encrypted, and a hardware session key protects the CPU-enclave-to-GPU-enclave path. Before GPU confidential computing reached production, CPU-only enclaves and severe memory limits made protection of large model weights impractical.

Why attestation is the property that actually closes the operator-access gap rather than just narrows it

An encrypted memory region leaves a crucial question unanswered. How does the data owner know which program is inside it? Without attestation, users still have to take the operator’s word that the intended code is in the protected environment. Memory encryption supplies the protection mechanism, but attestation makes the deployment’s claim verifiable.

At boot, the hardware measures the components loaded into the protected environment using cryptographic hashes. Depending on the TEE, those measurements cover enclave code, data pages, associated metadata, or the firmware and kernel image of a protected VM. The measurements are stored in tamper-evident registers. A single changed byte in a measured component changes its measurement, making a mismatch detectable. The hardware then produces a signed document, called a quote, which a remote party can verify against a hardware root of trust.

Before sharing secrets, the data owner can check whether the expected binary is running on genuine TEE-enabled hardware with the required configuration and current microcode. Independent verification is essential to this reasoning. If the provider both operates the environment and supplies the only verdict on its attestation, the user still depends on the provider’s assurance. Intel Tiber Trust Services, formerly Intel Trust Authority, illustrates a separate SaaS service for independent attestation verification.

For inference, this evidence can reach beyond a generic claim that encryption is enabled. Attestation can support verification that the model receiving a prompt is the expected model and that it runs inside an environment the operator cannot inspect. The claim depends on what the measured configuration establishes. An attestation label alone is not a substitute for that connection. Properly tied to the expected workload and secret-release process, attestation makes “the operator cannot read this” a verifiable fact within the hardware’s trust model.

Where the TEE trust model still has open edges and what honest adoption looks like

Chip-vendor public key infrastructure, firmware, and attestation chains become trust roots, including Intel’s attestation certificates, AMD’s key distribution service, and AWS’s Nitro signing. Compromised vendor signing keys or a closed-source firmware backdoor could undermine the evidence without being detectable from outside the enclave. A narrower trust boundary is valuable, but it is not a boundary without dependencies.

The existence of hardware side channels is one reason to distinguish a verified configuration from a universal proof of security. The measured code and the platform’s guarantees remain part of the assessment. The Confidential Computing Consortium, hosted by the Linux Foundation, has pushed for transparency reports and reproducible firmware builds to address vendor trust. Current production systems nevertheless remain vendor-anchored.

Attestation integration, key management, and secret provisioning take skills that go beyond those needed for standard cloud deployment. Hardware isolation alone cannot supply the workflow that releases secrets only to an accepted configuration. Honest adoption therefore evaluates the verification and provisioning design as well as the chip. This is a different burden from administering ordinary access policies, because the system must connect cryptographic evidence to actual control over sensitive data.

Fully homomorphic encryption computes on ciphertext and removes the hardware trust root, but its performance overhead makes it impractical for general AI inference workloads. Secure multi-party computation distributes trust, while network bandwidth constrains it at scale. TEEs run at near-native performance, but they still rely on a hardware trust root.

How can an AI assistant keep prompts private from the company hosting it?

For an AI assistant, architectural privacy-by-design requires hardware-enforced confidentiality, independent attestation, and a design that excludes the operator from plaintext during inference. Protecting only the surrounding storage and connections does not meet that requirement. Putting key management inside an enclave also falls short of that requirement if inference stays outside the protected boundary. The workload itself needs protection, including the GPU path when large model inference depends on it. The H100/H200 approach makes that scope tractable through GPU isolation and a hardware-secured CPU-to-GPU data path.

The evaluation criteria follow from those requirements. What is inside the protected boundary, what does the attestation measure, and who verifies the evidence? A system that publishes verifiable attestation evidence and supports independent verification gives the user something different from an assertion of privacy. Its claim can be assessed against a configuration. Evaluation must also establish whether secrets are released only after that configuration has been accepted, regardless of whether the provider advertises a TEE somewhere in its infrastructure.

These criteria also explain why a vendor ranking alone would be inadequate. Application enclaves, whole-VM protection, and GPU isolation protect different scopes, while the surrounding verification workflow determines what a user can establish about a deployed assistant. A privacy policy can describe commitments, but it cannot demonstrate those properties. For organizations evaluating providers, the practical question is therefore not simply whether their data is encrypted. It is whether they can verify, cryptographically and independently, that the operator cannot read their inputs or outputs during inference.