SOC 2 Type II Reports for AI Services — What They Miss

SureCloud reports that most enterprise buyers, especially in technology, financial services, and healthcare, now require SOC 2 Type II before commercial discussions even open, with arriving lacking one meaning exclusion from the shortlist entirely, putting the credential close to near-universal gatekeeping status. That's a lot of weight resting on one document, and the weight is deserved in one sense: a Type II report reflects an outside CPA firm's opinion that controls existed and worked over a defined stretch of time, which is a meaningfully higher bar than a Type I snapshot. A Type II evaluation typically covers a defined period of six to twelve months. The trouble is that buyers treat a narrow, well-scoped attestation as if it were full AI due diligence, a job the framework was never built to do. Auditors themselves are starting to push harder, asking pointed questions about model vendors, agentic behavior, and shadow AI tools, but they're doing it without any standardized AI criteria to work from, so the scrutiny varies wildly from one audit to the next. Enterprise procurement has effectively made the report non-negotiable, and that non-negotiable status is precisely what makes its blind spots invisible to the people depending on it most.

What SOC 2 was built to certify

To see where the gaps come from, it helps to look at what SOC 2 was designed to do in the first place. Blaxel's guide describes the framework as resting on five Trust Services Criteria, Security, which every audit must include, plus four optional categories, Availability, Processing Integrity, Confidentiality, and Privacy, built on three core assumptions.

It assumes controls can be documented once, tested at a point in time, and trusted to stay effective until someone deliberately changes them. It assumes systems process information exactly as their documented logic says they will. And it assumes a human sits behind every meaningful access decision. These assumptions were designed for environments where systems behave deterministically, controls are static, and humans mediate every significant access decision.

AI systems violate all three assumptions at once: they produce non-deterministic outputs, exhibit emergent behaviors unpredictable by traditional change management, and, in agentic configurations, make access decisions themselves. SureCloud notes the system operates within the five TSC (Security, Availability, Processing Integrity, Confidentiality, and Privacy) defined by the AICPA's 2017 criteria, revised October 2022, which remain the operative standard. A SOC 2 report generates an auditor's opinion on whether controls are designed and functioning, not on whether the outputs those controls surround are accurate, fair, or safe. The report is built to show that the scaffolding around the AI is controlled. Every gap explored in the rest of this piece traces back to that one distinction: control versus outcome.

Agentic behavior and SOC 2's change management and access control logic

Traditional change management inside SOC 2 assumes a sequence: someone proposes a change, someone designs it, someone documents it, someone tests it, someone approves it, and only then does it go live. Autonomous agents skip every step in that sequence. They generate code and execute it at runtime, with no human authorizing each individual instance, which makes the entire change-management apparatus structurally unable to apply to what the agent is doing. SOC 2's security criteria are built around governing who can deploy code; agentic systems generate and run their own code, bypassing that pipeline stage by stage.

Access control runs into the same wall. SOC 2 expects that any privileged action can be traced back to an accountable person, and when an autonomous agent takes a privileged action on its own, auditors flag the absence of a human request as a serious accountability gap. If the logs show nothing more than a tool name or a shared service account rather than a specific human initiator, an auditor has grounds to ask who signed off on the action, and the TSC framework offers no criterion for agent-level attribution to answer that question. Static, role-based access control can't watch how an agent behaves in real time or apply context-aware policy when the agent is combining permissions in ways nobody designed for in advance. Teleport reports that availability evidence degrades the same way in agentic environments: auto-scaling creates short-lived instances that aren't consistently instrumented, producing gaps in the log chain that weaken auditors' confidence in monitoring coverage.

This isn't a hypothetical concern dressed up for the sake of argument. CVE-2025-34291, a confirmed vulnerability in the Langflow AI platform, let attackers take over accounts through CORS and CSRF exploitation and then execute arbitrary code, an agentic-architecture weakness that SOC 2's change management and access controls had no way to catch or prevent. Anthropic's threat report goes further, documenting state-sponsored actors and financially motivated criminal groups using Claude as an execution layer inside multi-agent frameworks, building what amount to AI-orchestrated kill chains where a model becomes one node in a larger autonomous pipeline, issuing instructions to other agents and taking actions with consequences in the real world. SOC 2's change management and access criteria simply have no category for that threat.

Model drift and output quality outside SOC 2 certification

A Type II report is a backward-looking attestation over a fixed observation window, unable to certify what a model will produce after that window closes, or even during it, if the model was silently updated mid-period. That's the mechanism behind model drift: a model's outputs diverge from what's expected over time, either because it keeps processing new data or because a provider quietly swaps in an updated hosted model or fine-tuned checkpoint without announcing it.

What makes this failure mode dangerous is how invisible it stays at the infrastructure layer. The endpoint keeps returning a success response. Latency doesn't budge. The answer coming back can look polished and confident even as the model has drifted away from the behavior that passed evaluation months earlier, producing hallucinations, weaker refusals, or tool calls that miss their mark. Nothing in the dashboards trips an alarm, because nothing in the dashboards was built to measure this.

Processing Integrity, criterion PI1.4, is supposed to cover whether system processing stays complete, accurate, and timely, but the TSC sets no standardized threshold for model output quality or behavioral consistency. The evidence gap widens further when automated pipelines promote outputs based on internal thresholds without generating the approval and test artifacts that a human-driven change ticket would leave behind.

Moss Adams and Baker Tilly have each argued that organizations can embed AI-specific controls, including drift monitoring, inside the existing TSC framework simply by tailoring controls to the complexity of the AI system in question. That's a fair point as far as it goes, but voluntary scoping doesn't fix the underlying absence of a standardized threshold. Two vendors can both claim PI1.4 covers their drift monitoring while defining "acceptable drift" in entirely different terms, so the report handed to a buyer will not surface that difference. One vendor's drift monitoring might flag a five-point accuracy drop; another's might only flag total failure. Both pass audit. Neither buyer can tell which is which from the report alone. The Moss Adams and Baker Tilly guidance cited above is not two independent publications but a single article jointly produced under a disclosed alternative practice structure between the two firms.

Bias, fairness, and explainability outside SOC 2 scope

No criterion inside the TSC framework asks a vendor to test for bias, demonstrate fairness, or explain how a model reached a decision, and no amount of supplemental control language changes that: bias, fairness, and explainability simply aren't defined properties an auditor can render an opinion on. Left without systematic oversight, AI systems can absorb and amplify bias baked into flawed training data or buried inside opaque decision logic, and SOC 2 carries no requirement that a vendor show it even looked for that risk. An auditor might ask, at their own discretion, whether a model is explainable or whether its decision process is transparent, but that request is a courtesy, not a mandate, and for something like a deep neural network the decision path may genuinely resist interpretation no matter how hard anyone looks.

A 2025 comparative audit published in the Defensive Publications Series put this to the test directly, examining Anthropic, Google DeepMind, and OpenAI against ISO 27001 and NIST frameworks.

That absence lands hardest on regulated-industry buyers: healthcare systems, financial institutions, and legal practices that carry downstream liability for whatever a model outputs on their behalf. A SOC 2 report sits next to a vendor's AI product and, through simple proximity, gets read as an assurance of quality it never actually offers. A report that never assessed bias says nothing about whether bias is present. And no AICPA-branded "AI SOC 2 certification" exists. Any vendor claiming SOC 2 alone satisfies EU AI Act obligations or comparable AI management standards is making a claim that isn't true.

Two SOC 2 privacy controls that become data-leakage risks in AI deployments

Two ordinary implementation habits turn SOC 2's own audit-trail controls into sources of data leakage rather than protections against it: logging raw prompts without redacting them, and leaving the LLM providers a product actually calls, its subprocessors, invisible inside the vendor's report.

Many AI startups log raw prompts and completions with no redaction policy applied before the log is written, and those logs routinely contain customer PII or PHI, turning an audit-trail control into a confidentiality violation. Without a redaction policy applied before the log gets written, a control meant to preserve an audit trail becomes the confidentiality violation itself. Auditors may look for proof that logs preserve enough metadata to reconstruct what happened without leaking sensitive content, but the TSC gives no specific guidance on how to strike that balance for LLM prompt logs specifically, so vendors are left improvising. A lineage problem underlies the redaction and subprocessor gaps: if a vendor can't trace where personal data moved across training, fine-tuning, inference, and logging, it has no way to prove it's handling and disposing of that data the way it claims to.

Subprocessor disclosure has the same gap: any application built on top of a hosted LLM provider is routing customer data through that provider's infrastructure, which makes the LLM provider a subprocessor under the SOC 2 reporting structure. But a SOC 2 report scoped to the customer-facing vendor doesn't automatically pull the underlying model provider's own security posture into view. Buyers need written confirmation that their data isn't being used to train the provider's models unless they've explicitly opted in, because confidentiality and privacy commitments extend to whatever the LLM provider does with that data behind the scenes. SOC 2 has no standard way of requiring that subprocessors even get disclosed, let alone evaluated. OpenAI's public security materials pair SOC 2 with ISO 27001-family certifications and ISO 42001, an implicit acknowledgment that SOC 2 alone is insufficient for AI trust claims, including subprocessor governance. OpenAI's security page specifically states it maintains ISO/IEC 27001:2022 and ISO/IEC 27701:2019 certifications alongside SOC 2 Type 2, and separately maintains an ISO/IEC 42001:2023 AI Management System covering its consumer and business AI products and models.

Linford & Co. reports that shadow AI, engineers piping company data into unapproved AI tools, introduces unvetted subprocessors that bypass change-management and vendor-review controls, and auditors are increasingly probing for it.

ISO 42001, the NIST AI RMF, and the EU AI Act beyond SOC 2

ISO/IEC 42001, published in December 2023, is an AI management system standard that extends conventional information security governance into territory built specifically for AI: model governance, bias, explainability, all the ground SOC 2 and even ISO 27001 leave uncovered. The NIST AI RMF Generative AI Profile, formally NIST-AI-600-1, followed in July 2024 and narrows in on model governance and bias in generative AI systems specifically, a level of specificity that SOC 2's broad Processing Integrity criterion was never built to match. The Defensive Publications Series audit referenced earlier makes the case empirically rather than theoretically: evaluating Anthropic, Google DeepMind, and OpenAI beyond perimeter security controls required pulling in ISO 27001, NIST, and AI-specific risk models, because SOC 2 by itself couldn't get the job done.

The EU AI Act, which entered into force in August 2024, adds a regulatory dimension on top of these voluntary standards, and its timeline matters for any vendor selling into European markets. For enterprise buyers assembling a real AI vendor vetting checklist, the practical shape of this is straightforward: SOC 2 Type II still belongs on that checklist, since it answers real questions about security and operational controls that nothing else answers as well, while ISO/IEC 42001, the NIST AI RMF Generative AI Profile, and the EU AI Act each address specific gaps that SOC 2 leaves open, together forming the emerging fill layer that enterprise AI buyers should require alongside SOC 2, not instead of it. It just can't be the whole checklist.