Private LLM Hosting: Dedicated vs Shared Infrastructure for Sensitive Workloads

We choose between shared and dedicated private LLM hosting by reading the rule that makes a workload sensitive, before comparing prices or GPU counts.

Shared infrastructure means a model endpoint that a provider runs for many customers. Dedicated infrastructure means GPUs set aside for one customer, inside the customer's own network.

A shared endpoint can meet a data-location rule through region pinning, a setting that keeps requests in one region. Dedicated GPUs come in when a rule keeps prompts inside the buyer's network.

Rules on provider staff access depend on hardware that protects data in use.

Which Mandates Call for Shared or Dedicated LLM Hosting?

A mandate is the law, contract clause or internal policy that governs a sensitive workload's data. We sort mandates into three types, each pointing to a different kind of hosting:

1. Residency and retention: the mandate sets the location of inference and storage, and the retention period for records.
2. Network boundary: the mandate sets the networks a prompt may cross between the application and the model.
3. Provider-staff access: the mandate names the provider staff who may see prompts, outputs and model weights in use.

Can a Shared LLM Endpoint Meet a Data Residency Rule?

A shared endpoint can meet a residency rule when the provider pins inference to a region and the contract names that region.

Vercel's guide to self-hosted LLMs gives an example from its AI Gateway, a routing layer that passes each request to a model provider. The gateway rejects a request with an HTTP 400 error when the pinned region can't serve the request.

Each response also reports the region that served it, so an auditor can check the serving region for each request.

The General Data Protection Regulation (GDPR) governs personal data in the EU. It counts sending that data as processing, so a prompt carrying personal data falls under its limits on transfers outside the EU.

Region pinning has two limits:

1. Network route: pinning sets the region where the provider runs the request. The setting leaves out the path a prompt takes through other networks to reach that region.
2. Safety review: Vercel notes that a provider's abuse and safety review can happen outside the pinned region. A mandate that limits who reads prompts needs that review covered in the contract.

What Does a Network-Boundary Mandate Require From LLM Hosting?

A network-boundary mandate means running the model on dedicated GPUs inside the buyer's network, with its weights stored there too. A private link to a provider's model hands the prompt to the provider's systems.

Cloud consulting firm ASCENDING's private LLM buyer guide says private connectivity leaves the model service on the provider's infrastructure. An application hosted in the buyer's cloud can also send its prompts to a model hosted elsewhere.

AI engineering firm Prodinit built a fintech platform on private Amazon Web Services (AWS) subnets in four weeks. Its air-gapped fintech case study reports zero internet egress, meaning the subnets kept traffic off the public internet.

The platform's inference ran on Amazon Bedrock, AWS's managed model service, through a private connection called a VPC endpoint. The model stayed on AWS systems, so the build meets a mandate written as zero internet egress.

Prodinit's self-hosting playbook describes the zero-egress version that keeps the model inside the buyer's network, whether on-premises or in its own cloud account. Weights load from a private storage bucket, container images come from a private registry, and the serving software stays off the internet.

A local model can still call outside services for telemetry, package updates or logins. Check outbound traffic for each component before signing off on the network boundary.

Does GPU Encryption Keep Provider Staff Away From LLM Data?

GPU encryption for data in use isolates prompts and model weights while the GPU processes them, which can keep that data from provider staff. NVIDIA sells it as NVIDIA Confidential Computing on Hopper, Blackwell and Rubin GPUs.

NVIDIA puts the speed at up to 98% of an unprotected deployment, a best case that suggests the protection costs a buyer little performance.

Ask the provider for two answers in writing:

1. GPU coverage: which GPUs in the deployment run with that encryption switched on, and how the buyer receives attestation reports, the hardware's signed record of its setup.
2. Support access: diagnostic access that expires after a set period and leaves an audit log the buyer can read.

Encrypted health data still falls under HIPAA, the US health privacy law. HHS's cloud computing guidance says a cloud provider storing encrypted health data without the key still needs a business associate agreement (BAA) with its customer.

Our sovereign AI cloud architecture guide checks administrator access to the control plane, the software that manages the cluster, as one layer of a control audit.

What Belongs in a Zero Data Retention Clause for LLM Hosting?

Write four things into a zero data retention (ZDR) clause, the contract promise that a provider won't store prompts or outputs:

1. Logging scope: the events and fields the provider records.
2. Retention period: how long each kind of record stays, including backups.
3. Inspection rights: the provider staff who can read records, and the buyer's right to audit that access.
4. Safety review: whether the provider's abuse and safety checks fall inside the clause, and the location of that review.

Vercel's guide adds that model location covers one part of complying with that law. Logging, key management and outbound traffic also have to stay inside the regulated environment.

Consumer AI endpoints without a BAA fall short of HIPAA, wherever the provider hosts the model.

What Does Dedicated LLM Hosting Cost Against an API?

Dedicated LLM hosting starts to cost less than a frontier-model API, one serving the most capable commercial models, at roughly 100M to 500M+ tokens per month. Prodinit's playbook draws that range from published cost analyses.

Model tier, GPU pricing and GPU use decide each project's break-even point. Self-hosting often doesn't win on cost alone against a low-priced API for open-weight models (models with published weights), so the mandate usually decides the hosting.

Vercel's guide cites a benchmark where the same NVIDIA H100 deployment cost 17.5 times as much per output token at one request per second as at maximum load. A dedicated build pays off when steady traffic keeps its GPUs busy.

Changing models also adds work on dedicated GPUs. Vercel notes that a model swap on a private cluster means re-provisioning the GPUs, re-validating the model and redeploying the service. On a shared endpoint, the same swap is a setting change.

Seven Contract Questions for a Private LLM Hosting Provider

Send these seven questions from ASCENDING's deployment and ownership checklist to each provider, shared or dedicated:

1. Model and license: the model and version that will run, and the rights covering use, changes and redistribution of the resulting files.
2. Inference: where each request runs, and the endpoints that receive prompts, retrieved documents and responses.
3. Training and retrieval data: where datasets, embeddings, search indexes and backups are stored, and how the provider handles access and deletion.
4. Logs and support: what the provider records, how long it keeps each record, who can read it, and whether support access expires with an audit trail.
5. External calls: whether the workflow contacts model APIs, embedding services, telemetry, package repositories or business tools outside the agreed network boundary.
6. Release and recovery: who approves a new model version, how a failed release rolls back, and when the provider last tested recovery.
7. Handoff: the code, configurations, model files and runbooks the buyer receives, and the team that handles incidents and later changes.

Put the answers in the contract. A provider that won't answer in writing leaves each question to its own defaults.

Our GPU cloud, zCLOUD, sells capacity on demand, as Reserved Clusters and as Private Cloud, a custom cluster scoped around a buyer's security review and region limits. We hold zCLOUD to the same seven questions.