TEE-Backed AI Assistants on the Procurement Shortlist — Audit Criteria for Regulated Buyers

TEE-backed AI assistants process prompts inside hardware-isolated environments designed to keep inference data unreadable to the hosting provider. Regulated buyers should shortlist them on independently verifiable execution evidence. The audit framework below distinguishes genuine protection from contractual non-retention through five tests covering attestation, attributable logs, log integrity, model identity, and human control.

Why regulated buyers cannot rely on contractual privacy promises alone

A vendor comparison matrix can place zero-data-retention, or ZDR, beside TEE-backed inference as though they were different strengths of the same privacy feature. They are different kinds of assurance. ZDR is a contractual commitment not to retain prompt data after a session ends, while the data still exists in cleartext in vendor-controlled memory during inference. No independently verifiable artifact proves that the promise was kept.

Sensitive data is processed inside a hardware-protected enclave whose configuration is measured by the CPU and can be checked cryptographically against a public root of trust. The distinction is between a promise about handling data and evidence about the environment that processed it. An infrastructure certification does not supply that request-specific proof. Without it, the comparison sheet obscures the central audit question.

How do TEE-backed assistants keep prompts private from the hosting provider, and what remains exposed?

Traditional encryption protects data that is stored or moves between systems, but inference needs cleartext while it processes the data. TEEs use hardware isolation to close this data-in-use gap. Their protected regions keep code and data hidden even from a host operating system, hypervisor, or cloud provider with root access. The boundary is enforced at a level below those privileged software layers.

NVIDIA Confidential Computing protects GPU execution, memory, and register states, including model weights and prompts. It operates on Hopper, Blackwell, and Rubin GPUs and supports moving workloads into a protected enclave without code changes. Rubin is its third generation. NVIDIA’s Vera Rubin NVL72 extends the protected domain across NVLink, spanning 72 Rubin GPUs, 36 Vera CPUs, and interconnects. Its hardware-rooted device attestation evaluates compute assets regardless of where the workload runs.

Isolation does not eliminate every leakage channel. Timing, cache and bus contention, page-fault or controlled-channel attacks, speculative-execution effects, GPU residual-state leakage, and traffic-analysis signals remain residual risks. Hardware trust also remains a separate dependency. CPU attestation chains to a manufacturer such as Intel or AMD, which can be an unresolved constraint for national-security buyers. These limits define where additional controls are needed rather than negating the value of protection against direct memory disclosure.

Which regulatory obligations now require verifiable data-in-use protection, and what changed in 2026

Regulatory obligations make evidence essential, but they should not be collapsed into a universal mandate for TEEs. The EU AI Act places data governance in Article 10 and technical robustness in Article 15, with risk and quality management in Articles 9 and 17. DORA Articles 5–10 address ICT governance and risk management for financial entities. NYDFS guidance in its October 2024 and May 2026 industry letters brings AI into the cybersecurity programs already in place without adding requirements to Part 500 itself.

Healthcare raises a distinct attribution problem. HIPAA’s technical safeguards apply to AI access to PHI, and its unique user identification rule requires tracking the user behind access to ePHI. A shared service account cannot show who that user is. For federal procurement, the Unbiased AI Principles contractual inclusion requirement applied to LLM solicitations issued after December 11, 2025. March 11, 2026 was the deadline for updating agency procurement policies.

The 2026 European timeline changed while the substance of the obligations remained intact. The European Parliament formally approved the Digital Omnibus on AI on 16 June 2026, and the Council of the EU adopted it on 29 June 2026. The date when the rules apply to Annex III high-risk systems moved from 2 August 2026 to 2 December 2027, while Article 50 transparency obligations still take effect in August 2026. Buyers in high-risk categories should therefore treat documentation, oversight, and governance artifacts as required now. BSI C5:2025’s community draft adds focus areas for confidential computing, data sovereignty, and AI security that must be used from January 2027, while regulated healthcare sectors have had to use C5:2020 Type 2 attestation since July 2025.

Criterion 1 — Cryptographic attestation receipts that the buyer can verify without vendor mediation

Can the buyer establish that a particular inference ran inside a genuine, unmodified TEE without asking the vendor to confirm it? This is the foundational test. When an enclave loads, the CPU measures its code and configuration. On request, it produces a quote containing the measurement, a nonce, and a signature from a key derived from a CPU-fused master secret.

Intel’s Provisioning Certification Service issues PCK certificates chaining to an Intel root of trust. A verifier checks the signature against Intel’s public PKI and compares the measurement with an expected value before deciding whether to release a secret. If validation fails, the keys needed to unlock the data should remain withheld.

For procurement, the evidence is a sample quote for a test inference, with the relevant certificate chain for independent verification. The buyer also needs documentation showing when validation occurs and whether it gates key release. Confirmation that the endpoint is the manufacturer’s public service, rather than a vendor proxy, completes this test. Unlike ZDR, this supplies a cryptographic basis for evaluating the execution environment.

Criterion 2 — Individual-attributed audit logs retained to the governing framework's minimum period

Attestation establishes where processing occurred, but not who directed it. Many AI platforms are built to send requests through a shared API key or service account. HIPAA’s unique user identification rule, GDPR’s accountability principle, and SOX’s audit-trail requirements make that gap consequential when regulated data is involved.

The practical test belongs before deployment. Buyers should request a test-prompt log showing the individual user, timestamp, exact model version and commit, input hash, and output hash. Storage documentation should identify where records are written, whether they are write-once, and which retention controls belong to the platform and which to the buyer’s storage layer. The format must also support export to the buyer’s SIEM without vendor mediation.

Records must be kept for the period that the governing framework sets for their record category. The review should account for at least a year of operational logs and 7 years of audit work papers for SOX-relevant systems, and 6 years under HIPAA. PCI DSS v4.0 specifies 12 months, with 3 months immediately available. The high-risk AI logging requirement tied to EU AI Act Article 12 calls for logs to be kept for at least 6 months.

Criterion 3 — Tamper-evident log integrity provable to an external auditor

Correct fields are insufficient if records can be altered afterward. Log integrity is a separate evidentiary test. The EU AI Act and SOX cover tamper-evident integrity, while PCAOB AS 1105’s 2024 changes cover technology-assisted analysis and the reliability of electronic evidence. That standard does not explicitly require cryptographic or equivalent tamper-evidence for AI-generated records.

Buyers should request the hash or signature scheme applied at write time and the procedure an external auditor can use to recompute and verify it. The signing-key boundary deserves particular scrutiny. If the inference system both generates records and holds their signing key, the proposed assurance is circular. Evidence that the key sits outside the inference infrastructure should accompany a walkthrough of producing verifiable extracts for a regulatory examination.

Criterion 4 — Model identity pinned to a specific version, not a product name

COSO 2026, the EU AI Act, and FFIEC require model identity and version documentation more specific than a product name such as “GPT-4”. Model behavior can change without a visible signal to the user. When an update is silent, it breaks the link between the model checked during validation and the outputs kept for audit.

Buyers should require an exact version field for every request that matches an entry in the vendor’s model registry. Documentation alone cannot establish per-request identity. The evidence should include a change-notification policy and whether buyers can pin a prior version during a compliance review. As an advanced verification step, TEE buyers should also establish whether attestation covers the loaded model weights, because an attested enclave can otherwise leave the identity of weights loaded at runtime unresolved.

Criterion 5 — Human oversight gates that can stop or redirect an agent before it acts

The EU AI Act’s “effective oversight” requirement for high-risk systems turns attention to the ability to stop or redirect an agent. A notification is not an approval gate. An assistant that browses, writes files, calls APIs, or triggers workflows can propagate a permission error or reasoning failure before a person reviews its output.

The vendor should show a gate that buyers can configure for a chosen high-risk action, with enforcement in the execution path beyond a UI advisory. What happens if the gate becomes unavailable? Buyers need confirmation of whether the agent halts or proceeds and whether they can configure that behavior. They should also request the action categories gated by default and those requiring configuration, since theoretical capability does not establish the deployed posture.

What genuine TEE-backed inference looks like in production today

NVIDIA supports confidential bare-metal deployment, confidential containers on Kubernetes, and confidential virtual machines. Bare-metal deployment gives AI labs and model owners infrastructure set aside for them that they can verify themselves, with the infrastructure provider kept outside the trusted boundary. Confidential containers keep each pod safe in its own hardware-protected environment without changes to application code. Buyers should match those deployment properties to their infrastructure posture.

Apple’s Private Cloud Compute shows how server-side Apple Intelligence inference runs in production. Originally built exclusively on Apple Silicon, it now also uses NVIDIA GPUs on Google Cloud. Dell Technologies and SpaceX AI show how enterprises use Blackwell GPUs within the Dell AI Factory to protect proprietary model weights and enterprise data while they are in use. These cases establish that confidential inference is deployed rather than merely proposed. They do not, by themselves, establish that an assistant built on similar hardware passes every procurement criterion.

The trade-off that procurement teams must document explicitly — model catalog breadth versus cryptographic verifiability

TEE-backed inference has a narrower model catalog than ZDR deployments because the weights must be available to load into a verified enclave. ZDR instead provides contractual access to frontier models on vendor infrastructure. This is a real procurement constraint. The rationale should document whether the applicable framework requires cryptographically verifiable execution evidence or whether contractual non-retention satisfies the relevant requirement.

If independently verifiable execution is required, catalog breadth cannot replace the missing proof. Conversely, where contractual non-retention is sufficient, the broader ZDR catalog changes the evaluation. PCAOB AS 1105’s reliability requirements should not be described as an explicit TEE mandate. The distinction between an evidence obligation and a prescribed architecture must remain visible.

Performance belongs in the same documented trade-off. NVIDIA’s benchmarking reports up to 98% of unsecured performance, with overhead narrowing as hardware generations advance. The shortlist rationale still needs to explain the choice between available models, execution evidence, and the governing audit requirements.

Assembling the shortlist review — the evidence artifacts to request at each stage

At the RFI or initial-qualification stage, buyers should eliminate vendors unable to identify the TEE hardware, CPU generation, GPU architecture, and attestation root of trust. Buyers should also check whether verification uses the manufacturer’s public PKI or a service that goes through the vendor. Individual attribution belongs in this first screen too. A platform that cannot identify the person behind each request leaves a foundational evidence gap.

At the RFP and technical-due-diligence stage, claims must become inspectable artifacts. Buyers should get a test-inference quote they can verify independently and a sample log that has individual attribution, model version, input and output hashes, and a tamper-evidence signature. The documentation for the signing architecture must show that the key is outside the inference infrastructure. The model-pinning policy and sample registry entry should accompany a live demonstration of a high-risk approval gate.

The agreement should include the right to request attestation quotes for any inference session within the retention period, advance model-change notification with a defined minimum notice period, and buyer consent for changes to the manufacturer-rooted public PKI. It should also preserve vendor-independent SIEM export with tamper-evidence signatures intact. The operational output is not a privacy score, but a set of artifacts the buyer either possesses or lacks.