Data Residency Isn't Enough: Why Regulated Workflows Need On-Prem (or Air-Gapped) Deployment

Summary

Data residency pins data to a geographic region, but data sovereignty governs who can access or compel it, and the US CLOUD Act (enacted 23 March 2018) reaches data "located within or outside of the United States."
SOC 2 attests that controls were designed and operated effectively over a defined period; it does not guarantee data location, so verify scope, reporting period, and disclosed subprocessors.
On-premise is not the same as air-gapped: an air-gapped AI workflow requires an in-boundary model endpoint, in-boundary identity and policy, controlled updates, local telemetry, and in-boundary audit evidence.
Regulated workflows need human approval as a gate before execution, plus knowledge retrieval that is traceable to certified sources rather than probabilistic RAG.
To close every external path, Jinba Flow supports on-premise and private cloud deployment with private model hosting, RBAC, SSO, and audit logging.

Selecting a cloud region is treated as a compliance decision. It is a location decision. For banks and insurers running AI workflows on sensitive customer and transaction data, the operative question is whose infrastructure the data sits on, and whether any external party can reach it.

The distinction matters because regulators, auditors, and courts treat it as fundamental. A vendor's data residency offering pins the data to a geography. It does not transfer custody. The distinction between data residency and an on-premise AI workflow separates location from control.

Region-pinning leaves data on the vendor's hardware

Mattermost's analysis makes the distinction: data residency describes physical location, while data sovereignty describes legal jurisdiction and control. Location alone does not deliver control.

Cloud providers sell residency as a compliance feature. The customer selects a region and the data stays there. What that framing omits is that the hardware, the management plane, and the operating staff all belong to the vendor. The data is isolated from other tenants, but it is not inside the customer's boundary.

The US CLOUD Act makes this concrete. Enacted on 23 March 2018, the Act amended the Stored Communications Act to apply to data "located within or outside of the United States," as O'Melveny and Myers document. A US-headquartered cloud vendor pinning the data to a European or Asian region does not remove US law enforcement reach over that data. The Microsoft v. United States case, which had directly tested this question, was dismissed as moot once the CLOUD Act resolved it.

The practical implication for financial institutions: selecting an EU region on AWS, Azure, or Google Cloud is a data residency decision, not a data sovereignty decision.

What "stateless" AI calls store

The data persistence problem extends into AI services. Microsoft's own Azure AI documentation confirms that standard Azure OpenAI API calls are stateless and that prompts and completions are not retained to train base models. However, optional features do persist customer data at rest: fine-tuning uploads, vector store contents, Assistants API threads, and Stored Completions are saved within the customer's Azure tenant.

Abuse monitoring, which is enabled by default, captures and may store prompts and completions for human review when misuse is flagged. Disabling it requires a separate approval process.

The data remains physically on Microsoft-operated infrastructure. It is tenant-isolated, but it is not inside the customer's own boundary. For a bank running credit decisions or an insurer running claims workflows through an AI layer, that distinction is not academic.

SOC 2 compliance is not a location guarantee

The second common shortcut is treating a vendor's SOC 2 report as a proxy for data safety. It does not fulfill that purpose.

As CyberVantage360 describes, SOC 2 is an assertion-based examination of a service organization's system description and relevant controls. It attests that specified controls were designed and, in the case of a Type II report, operated effectively over a defined period. It is not a certification that data resides within a particular geographic or organizational boundary.

The scope boundary of a SOC 2 engagement is the determining factor. A well-executed examination of the wrong service scope does not answer the customer's question. When a financial institution reads "SOC 2 compliant" on a vendor's trust page, the operative questions are: which systems are in scope, which period does the report cover, and which subprocessors does it disclose?

SOC 2 reports are required to make outsourced dependencies visible. What that disclosure typically reveals is a chain of subprocessors, each with their own hosting arrangements, jurisdictions, and access patterns. A single AI workflow platform may rely on a model endpoint hosted by a third party, a telemetry service operated in a different region, and identity infrastructure with its own dependency chain. The SOC 2 attestation covers the primary vendor's controls. It does not collapse that dependency graph into a single boundary.

Trunnion.ai's analysis of deployment models states the point: deployment labels are shortcuts, not architectures. Buyers of AI infrastructure for regulated workflows need to inspect the full dependency list, not the marketing summary. The dependencies that matter include model endpoints, embeddings services, vector stores, identity providers, policy services, secrets management, package repositories, telemetry channels, update feeds, and support access paths.

Any of these can carry data outside the intended boundary.

What on-prem and air-gapped require

On-premise and air-gapped are not synonyms. The distinction determines what the deployment controls.

On-premise means infrastructure runs inside the customer's data center or private cloud. It eliminates the shared-hardware problem. It does not, by itself, eliminate external dependencies. An on-premise deployment that calls a cloud model endpoint, sends telemetry to a vendor SaaS service, or pulls updates from a public package repository still has internet-connected paths that can carry data outward.

Air-gapped deployments close those paths by design. Internet connectivity is prohibited across the security boundary. That constraint applies to every component in the stack.

For AI workflows, operationalizing a true no-egress environment requires the following controls to be verified before deployment:

Model endpoint: The inference model must run inside the boundary. A call to an external cloud model API is not a valid runtime dependency in an air-gapped environment.
Identity and policy services: Authentication, authorization, and policy enforcement must be available entirely within the approved environment.
Update process: Software updates cannot be pulled automatically from external repositories. A controlled transfer process is required, including provenance checks, digest verification, content inspection, malware scanning, compatibility testing, and formal approval before installation.
Telemetry: Operational data must be collected locally. Any authorized export must go through a governed, logged transfer path, not an automatic outbound channel.
Audit evidence: Logs and compliance evidence must be stored inside the boundary by default, not in a vendor-hosted evidence store.

Each of these is an explicit architectural control, not a configuration option.

Building a compliant AI workflow inside the boundary

The controls described above are architecture requirements. The platform delivering the AI workflow must satisfy them, not just the infrastructure layer.

Jinba Flow is purpose-built for regulated deployment environments. It supports on-premise and private cloud deployment, allowing banking and insurance operations teams to run AI workflows entirely within their own network boundary. Private model hosting is available via AWS Bedrock, Azure AI, or self-hosted options, which addresses the model endpoint requirement directly: inference does not leave the customer's environment.

Jinba Flow includes RBAC, SSO, audit logging, version control, and feature flags as standard governance controls. Jinba is SOC 2 compliant. That attestation confirms the control design. It does not replace the on-premise deployment as the mechanism that keeps data inside the customer's boundary. Both matter, and they perform different functions.

Human-in-the-loop as a governance requirement

Regulated workflows cannot run fully automated. Credit limit changes, claims approvals, and policy exceptions all require a documented human decision in the approval chain.

Jinba App Neo includes a WAIT_FOR_APPROVAL step that pauses a workflow run until a designated reviewer approves or rejects the action via an in-chat approval card or the run console. The workflow does not proceed until that decision is recorded. This is the human-in-the-loop control that audit trails require: a gate before execution rather than a notification after the fact. AI operations in Jinba App Neo are logged, and credentials are never passed to connected systems.

Jinba App Neo also enforces role-based access control, with administrative operations restricted to owners. This is the governance layer for shared AI agents operating across a team, distinguishing it from single-user AI tools that provide no shared access control or audit record.

Auditable knowledge, not probabilistic retrieval

The knowledge layer of an AI workflow carries its own compliance risk in regulated industries.

Standard retrieval-augmented generation, or RAG, retrieves vector chunks based on cosine similarity. The same question can return different chunks on different runs, producing variable answers. More critically, RAG provides no auditability of the knowledge itself. There is no mechanism to inspect which facts the system is working from, trace them to their certified source, or gate updates through a review process.

Jinba Brain does not use RAG. It uses ontology-validated linked records rather than vector search. The base-layer ontologies are FIBO (the EDM Council Financial Industry Business Ontology) for banking and finance workflows, and the ACORD Information Model for insurance. Every record in Jinba Brain stores the source path and a SHA-256 hash, making every fact auditable back to its source document.

The raw data store is kept on-premises and is never altered by the pipeline. The system reads from it but does not write to it. Proposed knowledge updates go through an Evaluator that checks schema conformance, referenced IDs, duplicate detection, and source hash before any record is saved. No update goes live without human review. This gate exists because in a regulated environment, incorrect knowledge propagating to all agents simultaneously is a compliance event, not an edge case.

The knowledge layer encodes company-specific definitions, calculation rules, and business logic. It makes data answerable, not merely findable.

The boundary is the control

Data residency, as sold by cloud vendors, is a location commitment. It indicates which region data sits in. It does not establish who can compel access to it, which subprocessors handle parts of the workflow, or whether a telemetry channel carries inference data to an external service.

For banks and insurers, the audit question is whether any path exists by which this data leaves the boundary without the institution's approval. Region-pinning cannot answer that question. On-premise and air-gapped deployment can, when the full dependency graph is verified and every external path is closed.

SOC 2 compliance confirms that controls exist. On-premise deployment determines whether those controls operate inside the customer's boundary or inside a vendor's.

For regulated AI workflows, the deciding factor is the physical and logical boundary of the institution's own infrastructure, rather than a vendor's trust page. Heads of AI and operations at financial institutions building production workflows should evaluate every component of the stack against the air-gap requirements above, verify that model endpoints, identity services, audit logs, and knowledge stores all operate within that boundary, and confirm that human approval gates are present before any automated action executes.

If any component in the stack calls home, the boundary is not closed.

Frequently Asked Questions

What is the difference between data residency and data sovereignty?

Data residency is where data is physically stored. Data sovereignty is who holds legal control over it and who can be compelled to hand it over. Region-pinning on AWS, Azure, or Google Cloud is a residency decision: the data sits in the chosen geography but remains on vendor-operated hardware, under vendor management planes and vendor staff. Sovereignty requires that custody, access control, and legal jurisdiction sit with the institution. Both can be true at once, but only the second answers the audit question.

Is data residency enough for regulatory compliance?

No. Data residency proves location, not custody or access control. The questions an auditor, regulator, or court will ask are who can compel access to the data, which subprocessors handle parts of the workflow, and whether any telemetry channel carries inference data to an external service. A region selection screen on a cloud console answers none of them. Residency is a useful control. It is not a compliance outcome on its own.

Does the US CLOUD Act apply to data stored outside the United States?

Yes. The CLOUD Act, enacted on 23 March 2018, amended the Stored Communications Act so that it applies to data "located within or outside of the United States." A US-headquartered cloud provider pinning the data to a European or Asian region does not remove US law enforcement reach over that data. The Microsoft v. United States case had directly tested this question and was dismissed as moot once the Act resolved it. Selecting an EU region is a residency decision, not a sovereignty decision.

What is the difference between on-premise and air-gapped deployment?

On-premise means infrastructure runs inside the organization's own data center or private cloud. Air-gapped means internet connectivity is prohibited across the security boundary. The gap between them determines whether data can leave. An on-premise deployment that calls a cloud model endpoint, ships telemetry to a vendor SaaS service, or pulls updates from a public package repository still has internet-connected paths that can carry data outward. Air-gapped deployment closes those paths by design, across every component in the stack.

What does an air-gapped AI workflow require?

An air-gapped AI workflow requires five controls, each verified before deployment. The inference model must run inside the boundary, with no calls to an external cloud model API. Identity, authorization, and policy enforcement must be available entirely within the approved environment. Software updates must arrive through a controlled transfer process with provenance checks, digest verification, content inspection, malware scanning, compatibility testing, and formal approval. Telemetry must be collected locally, with any export running through a governed, logged path. Audit logs and compliance evidence must be stored inside the boundary by default.

Does SOC 2 certification mean my data stays in my jurisdiction?

No. SOC 2 is an assertion-based examination of a service organization's system description and relevant controls. A Type II report attests that specified controls were designed and operated effectively over a defined period. It says nothing about where data resides or who can reach it. The scope boundary of the engagement is what matters, so ask which systems are in scope, which period the report covers, and which subprocessors it discloses. That disclosure typically reveals a chain of dependencies, each with its own hosting arrangement and jurisdiction.

Can AI workflows run entirely inside a bank's or insurer's own network?

Yes, provided every component of the stack operates inside that boundary. In practice this means private model hosting through AWS Bedrock, Azure AI, or a self-hosted runtime so inference never leaves the environment, plus in-boundary identity and policy services, locally collected telemetry, and audit evidence stored on the organization's own infrastructure. Jinba Flow supports on-premise and private cloud deployment for exactly this reason. The test is whether any path exists by which data leaves without the institution's approval, not the deployment label.

Why is human-in-the-loop approval a compliance requirement for AI in financial services?

Because consequential actions require a documented human decision in the approval chain. Credit limit changes, claims approvals, and policy exceptions cannot run fully automated in a regulated workflow. The control must be a gate before execution, recorded before the action runs. Jinba App Neo implements this as a WAIT_FOR_APPROVAL step that pauses a workflow run until a designated reviewer approves or rejects the action through an in-chat approval card or the run console, with the decision recorded in the audit trail.

Why is retrieval-augmented generation a compliance risk in regulated industries?

Because standard RAG is probabilistic and provides no auditability of the knowledge itself. It retrieves vector chunks by cosine similarity, so the same question can return different chunks on different runs and produce variable answers. More critically, there is no mechanism to inspect which facts the system is working from, trace them to a certified source, or gate updates through review. Jinba Brain replaces vector search with ontology-validated linked records built on FIBO and the ACORD Information Model, storing a source path and SHA-256 hash for every record.

How do I evaluate a vendor's deployment claims?

Inspect the full dependency list rather than the deployment label. Deployment names are shortcuts, not architectures. Ask specifically about model endpoints, embeddings services, vector stores, identity providers, policy services, secrets management, package repositories, telemetry channels, update feeds, and support access paths, then confirm which of those operate inside the organization's boundary and which do not. SOC 2 confirms that controls exist. On-premise and air-gapped deployment determines whether those controls operate inside the organization's boundary or inside a vendor's.