Vendor-Blind AI Infrastructure Where Prompt Confidentiality Is Architecturally Enforced

Vendor-blind AI infrastructure is a zero-trust AI architecture for enterprise in which hardware, rather than vendor policy, enforces prompt confidentiality. It requires protected inference, attested routing, zero-retention execution, and a small, auditable trusted code base. Together, these properties block ordinary operator access to plaintext inputs, but hardware vulnerabilities and implementation flaws still need to be checked.

The structural problem with policy-based prompt confidentiality

A prompt containing client records travels to a cloud inference endpoint, where a hosted model processes it as plaintext in server memory. Transport encryption protects the journey, but not the data from the endpoint’s privileged operators. The router may also hold a readable copy. A promise not to train on that prompt therefore answers a different question from whether the provider can read it. The trust gap is between what a provider is allowed to read and what it can read.

A California law firm’s IT department stopped partners from using public generative AI because sharing client information could put privilege at risk and break bar rules. Talaria’s authors document restrictions at JPMorgan, Citi, and Goldman Sachs, as well as healthcare institutions’ HIPAA concerns. These cases illustrate the limits of contractual reassurance. Data processing agreements and retention schedules govern what happens after exposure becomes possible.

What a TEE actually does to a prompt in memory

A Trusted Execution Environment, or TEE, moves the confidentiality boundary below the operating system and hypervisor. Its hardware protects code and data inside an enclave, with memory encryption preventing a root-privileged operator from reading the protected contents. The data is still used for computation inside that boundary. Prompts, intermediate activations, and optionally model weights can remain within the protected environment.

GPU confidential computing extends this boundary to the accelerator. NVIDIA H100 hardware uses an on-die engine to encrypt writes to GPU memory with a key generated inside the chip that never leaves it. The CPU-to-GPU link is encrypted as well. Protecting only CPU memory would leave that transfer unprotected, so the accelerator and its connection belong in the same architectural account. Confidentiality depends on the whole computation path, including its entry point.

How can a client know which environment receives its prompt? Remote attestation makes that question answerable before plaintext is released. Along with memory encryption and tamper-evidence, the hardware lets a remote party use cryptography to check that the hardware is genuine and runs the exact software image that was approved. The enclave’s measurement is the fingerprint of that image. A client can refuse to proceed when it does not match, turning an asserted configuration into an inspectable condition of access.

Why the API router is the overlooked gap in end-to-end confidentiality

An encrypted inference endpoint does not protect a prompt already exposed upstream. An API router terminates the client’s TLS session and opens another session to the provider, leaving the full request and response readable inside the router. The router can read tool definitions, credentials in traffic, and tool-call payloads. The router is a man-in-the-middle by design.

AEGIS’s authors identify four attack classes that share this plaintext access. A router can rewrite tool calls, substitute typosquatted packages in install commands, trigger alterations only under audit-evading conditions, or passively extract credentials. Client-side policy gates and response logs cannot close that common opening because their visibility is limited to delivered responses, leaving secrets on the request path unprotected. The enabling capability is the same throughout. Removing it addresses the family of attacks collectively.

The proposal confines plaintext handling to a hardware-enclave component while leaving authentication, scheduling, and billing on the untrusted host. The client verifies the enclave before releasing plaintext, and the attested image fixes the provider and endpoint so the operator cannot redirect traffic. Its trusted data path has 851 lines of code and passes provider-native APIs without converting them, which adds roughly six milliseconds of local relay overhead to each request. Attested routing is consequently necessary wherever a router would otherwise gain plaintext access.

Zero-retention execution and what it takes to make "nothing is stored" structurally true

Zero-data-retention distinguishes a system incapable of retaining interaction content from one that promises not to retain it. Protected memory alone does not establish that property. The approved software must also keep prompts, context windows, and outputs out of persistent storage. Otherwise, even a hardware-protected computation could leave a record that lasts.

The mechanism has three complementary pillars, namely dynamic PII masking before transmission, a stateless AI gateway that never writes to disk, and metadata-only audit logging. Those logs can keep timestamps, token counts, policy outcomes, and anomaly flags without saving the content of a prompt or response. Auditability need not require a transcript of the interaction. Masking reduces what enters the pipeline, while stateless execution and content-free logs govern what can remain afterward. None is a substitute for the others.

A legal agent retrieving contract precedents or a healthcare agent pulling patient history into a RAG pipeline exposes that context at inference, before storage-layer controls can act. Zero retention is therefore stricter than minimisation followed by scheduled deletion. It supports the design goals of GDPR Article 25, HIPAA Technical Safeguards, and SOC 2 CC6.7 by removing retained content from the system. That content no longer needs retention schedules or deletion audits, though the architecture alone cannot set out every compliance obligation or settle whether BAA coverage is needed.

How the "impossible trinity" shaped research into partitioned confidential inference

Confidential inference must keep client data safe without exposing the provider’s model, keep output fidelity, and use computing resources well. Talaria formalizes the conflict as the “impossible trinity” of model privacy, model performance, and model efficiency. Each requirement constrains the available protection mechanisms. Changing inputs through differential privacy or anonymization can change outputs, while homomorphic encryption adds a large amount of computation and needs access to model parameters. CVM-based approaches have also protected prompts while leaving responses exposed.

The proposed resolution partitions the inference pipeline so that sensitive, weight-independent operations run inside a Confidential Virtual Machine that the client controls, while weight-dependent computations run on cloud GPUs. Reversible Masked Outsourcing, or ReMO, masks intermediate data before it crosses that boundary and reverses the masking on the client side. The output remains identical to the unprotected model’s output. The authors report an average token reconstruction accuracy of 1.34%, down from a near-perfect baseline, while maintaining output invariance.

That result is a reported research finding whose scope is limited to the system studied. Its architectural significance is the explicit treatment of what crosses each boundary and what remains protected on either side. A separate project, πCreds, extends attested inference to another purpose. Researchers from Cornell Tech, MIT, and UCSD use TEE-protected LLM inference over authenticated bank statements, medical records, and email to generate privacy-preserving verifiable credentials. Both approaches place protected execution within a larger pipeline whose transfers and outputs also require deliberate design.

Where TEE-based guarantees are genuinely incomplete and what that means for practitioners

A TEE changes the attack surface while leaving attacks possible. SGX-class enclaves have faced controlled-channel attacks using page-fault patterns visible to an untrusted operating system, Foreshadow’s recovery of enclave keys through transient out-of-order execution, and SgxPectre’s leakage through speculative execution. These are not ordinary administrator reads of protected memory. Attestation verifies an approved image, which may still contain vulnerabilities or run on vulnerable hardware.

LLM inference introduces further questions about observable behavior. Autoregressive token generation creates sequential memory-access patterns that may expose information to a co-tenant attacker, while transformer attention, KV-cache access, and speculative decoding have not been systematically studied in TEE contexts. Protocol design can create weaknesses as well. Saini et al.’s February 2026 demonstration identifies the “static secret basis” pattern in performance-driven, partially shielded inference as a critical vulnerability caused by reused secret keying material. Encryption surrounding a flawed protocol does not repair its assumptions.

For practitioners, the response is to make the trusted boundary narrow enough to examine. Minimizing trusted code, verifying the exact software image before releasing plaintext, and treating the enclave as a constrained execution surface all follow from this risk model. Policy-only protection remains weaker against an operator who chooses to read accessible data, but the stronger hardware model should not be described as requiring a physical attack to defeat.

How production deployments are making architectural confidentiality a realistic default

Anthropic’s October 2026 acquisition of Opaque Systems is the clearest market signal in this shift. Opaque built technology that used hardware to protect sensitive data in AI systems, and its website says its team is joining Anthropic as it winds down its operations. The acquisition brings people who know confidential-computing into a frontier model lab. It signals investment in architectural confidentiality, though an acquisition alone does not prove that every service from the buyer provides it.

Anthropic and Pattern Labs had already described confidential inference through trusted VMs in June 2025. Their design places a small model loader and invoker inside an isolated virtual machine, with the rest of the inference server treated as untrusted. Only that protected component sees decrypted data. The loader accepts code signed after multi-engineer review, and a key server releases decryption keys only after attestation establishes the correct configuration. This connects software approval, runtime evidence, and access to secrets in one enforceable sequence.

Apple Private Cloud Compute shows how the same principles can work on standard data-center hardware. Apple’s design rests on a small, checkable set of components and excludes operator access to user data, including access by staff handling an incident. PCC now runs on Google Cloud using Nvidia Confidential Computing on Nvidia GPUs, Intel CPUs with TDX, and Google’s Titan security chip. Hardware performance is also reducing the deployment barrier. NVIDIA’s Dion Harris states that Blackwell and newer GPU architecture allow confidential computing with minimal performance impact, while Blackwell’s inline encryption addresses the remaining encryption bottleneck.

What vendor-blind architecture actually requires, and where Confidant fits

How can an enterprise prevent its AI vendor from seeing internal prompts and data? A useful evaluation begins with four architectural properties. First, inference must execute inside a hardware TEE that excludes ordinary operator, cloud-provider, and hypervisor access to plaintext. Second, if a routing layer handles plaintext, that data path must run inside a verified enclave. TLS alone does not satisfy this requirement because it ends at the router.

Third, zero-retention execution must prohibit persistent writes of prompts, context windows, and outputs. Fourth, the trusted code base must be small enough for independent review, with attestation binding the client’s approval to the software actually running. These requirements address different points of failure. Protected inference cannot compensate for a readable router, and neither prevents an approved application from retaining content unless its design forbids doing so. The evaluation concerns the complete interaction path.

Confidant is positioned around this combined architectural commitment, with privacy built into infrastructure. Its stated design excludes operator, router, and cloud-layer access to plaintext inputs even when those parties would prefer otherwise. That claim belongs against the same four criteria. For healthcare, legal, and financial-services teams, the decisive question for Confidant or any provider is not simply what the data policy promises, but what the architecture makes structurally impossible and what evidence allows the client to verify it.