Services: private LLM deployment
Open-weights LLMs on hardware you own.
We handle procurement and the deploy stack, finetune where it helps, and give your apps and engineers an OpenAI-compatible internal endpoint. You control the latency, the audit trail and the bill. It suits teams with PDPA-class data, steady high-volume workloads, or long-context costs that per-token pricing makes expensive.
Hardware and deploy stack
GPU server procurement, the serving runtime, and the open-weights model that fits the workload.
Hardware
GPU server procurement and on-prem or colo install: H100, L40S, A100 or RTX 6000 Ada, sized against the actual workload. NVMe for fast model load, redundant power and sensible cooling. We'll tell you the smallest box that does the job, whatever a reseller wants to quote.
Serving runtime
vLLM for high-throughput production, TGI where it fits the model, and Ollama or llama.cpp for small workloads where a full inference server is overkill. We quantise only where the eval still passes, and never to squeeze a large model onto a small box.
Open-weights model
Llama, Qwen, DeepSeek, GLM or Mistral, chosen for your workload, your language mix and a licence you can live with. We read the licences and test against your task before recommending anything. The right model is the smallest one that passes your eval.
Open-weights models we deploy
- Llama
- Qwen
- DeepSeek
- GLM
- Mistral
We test against your task before recommending anything. The right model is the smallest one that passes your eval, open-weights or otherwise.
Finetuning and domain adaptation
LoRA for almost everything, and continued pretraining when the data and budget justify it.
LoRA or QLoRA first
The shortest path to a model that knows your domain, your phrasing and your refusal policy. It takes days, is cheap to redo when the source data shifts, and sits cleanly on a base model you can rotate later.
Continued pretraining when the corpus justifies it
A substantial private corpus, such as bilingual technical writing, decades of regulator filings or a domain-specific code base, can justify it. We'll tell you honestly whether yours does. Most don't, and that saves you money.
We don't pretrain from scratch
That takes an eight-figure budget and a research team. If it's what you need, you need NVIDIA's account team more than you need us, and we'll point you to the right people.
Eval set first, weights second
Every finetune has a paired eval with golden cases, adversarial cases and a regression bar from the base model. If the finetuned weights don't beat the base model, the finetune doesn't ship.
Engagement shape
A four-to-six-week LoRA build covering data curation, training, eval and deploy, with an optional retainer for the next two refresh cycles. We scope continued-pretraining engagements against the corpus during the diagnostic.
Internal API and end-user access
An OpenAI-compatible endpoint your apps already know how to call, behind your SSO, audit log and perimeter.
Drop-in API
OpenAI-compatible chat-completions and embeddings endpoints. Your app code, agent framework, RAG stack and CLI clients keep working once the base URL changes.
SSO-fronted access
Engineers and internal tools authenticate through your identity provider: Entra ID, Okta, Google Workspace or your own AD. RBAC by group, rate limits per user and per app key, and no long-lived plaintext tokens left in repos.
Audit log on the wire
Every prompt, response, model and caller is captured, indexed and queryable, with retention by class. It's the discipline behind our infrastructure and security work, applied to model traffic.
When the box pays for itself
In our experience, self-hosting beats per-token vendor pricing on three workloads: steady high volume, where the model is busy most of the day; long context, where vendors charge for every input token of a 100k-context system prompt; and data residency, where the workload isn't allowed to leave the building. Outside those, vendor APIs are cheaper and we'll tell you so.
How it pairs with our other services
See infrastructure and security for hardening the host fleet, and consulting and training for the eval harness, context engineering and memory discipline that make the deployed model useful.
Common questions
What people ask before they brief us.
When does self-hosting an LLM beat per-token vendor pricing?
In our experience, on three workloads: steady high volume, where the model is busy most of the day; long context, where vendors charge for every input token of a 100k-context system prompt; and data residency, where the workload isn't allowed to leave the building. Outside those, vendor APIs are usually cheaper and we'll tell you so.
Which open-weights models do you deploy?
Llama, Qwen, DeepSeek, GLM and Mistral, chosen for your workload, your language mix and a licence you can live with. We test against your task before recommending anything. The right model is the smallest one that passes your eval.
Do you offer LoRA finetuning, or only deploy?
Both. LoRA or QLoRA is the default: it takes days and is cheap to redo when source data shifts. We use continued pretraining when you have a substantial private corpus. Pretraining from scratch needs an eight-figure budget and a research team, so we don't offer it.
What does the engagement look like?
A paid diagnostic week sizes the workload, picks the model and the box, and defines the engagement: hardware only, deploy only, finetune only or end to end. You keep a written plan and a number whether or not we run it.
Brief us
Tell us about
your workload.
The diagnostic week sizes the workload, picks the model and the box, and defines the engagement: hardware only, deploy only, finetune only or end to end. You keep a written plan and a number whether or not we run it.
Brief us