Local LLMs for First-Pass Medical PII Redaction

Medical groups want GPT-4 to summarize unstructured clinical referral notes. Compliance officers block the API key. Stop trying to negotiate. The compliance team is right.

An unredacted patient referral can contain a full name, an NRIC number, a date of birth, a residential address, and a clinical history. Sending that payload to a managed cloud API — even one backed by a Data Processing Agreement — introduces data-transfer and sub-processor obligations that a clinic may not be able to clear under its audit requirements. Patient information in Singapore is classified by sensitivity, and the controls applied to it must match that classification. If the clinic’s audit requirements follow that classification, routing raw clinical notes through a US-hosted inference endpoint will not meet the compliance bar.

The conflict here is real, not bureaucratic. Clinical reasoning capability sits in the cloud. Patient data sits on-premise. The conventional response — negotiate an enterprise agreement, sign the DPA, invoke the BAA equivalent — may not satisfy local audits. The architecture described here sidesteps that negotiation entirely. A local, quantized language model runs inside the clinic network as a data scrubber; its only job is to locate and replace PII before any text leaves the building. The architecture is designed so identifiable data does not cross the network boundary; in production, the local pass still needs fallback handling for low-confidence spans and OCR-corrupted text. The scrubbed text then proceeds to the cloud model for the clinical work it is actually built for.

Why Data Residency Expectations Shape the Design

A primary care clinic group that wants to digitize patient intake, reduce waiting time, and generate structured records from triage nurses’ freeform notes has to take a hard look at healthcare data residency expectations, and those expectations shape every medical-tech design that follows.

The definition of PII in a medical context is not the same as in a retail loyalty program. An NRIC number is PII. A name paired with a clinic visit date is PII. A phone number sitting adjacent to a medication name is PII. Combine any two of those fields in a single document and the sensitivity classification climbs further.

The first naive approach is manual redaction before uploading. Staff can be trained to highlight and delete before export. Under production load, manual redaction depends on sustained attention across many notes per shift, so gaps can go unnoticed until an audit. The second naive approach is to trust the enterprise agreement. The argument runs: the cloud provider signs a DPA, the DPA covers medical data, therefore we are compliant. That argument does not survive a line-by-line reading of what those agreements actually guarantee about processing location, sub-processor access, and audit rights.

For clinic audits, the network boundary is the operative control boundary. If the byte left the building, that transfer has to be defensible on its own merits. A DPA assigns accountability for the transfer; it does not prevent one.

Why Regex Fails the NRIC Problem

A proof-of-concept for automated referral routing tends to start with regular expressions.

Singapore NRICs carry a single-letter prefix — S or T — followed by seven digits and a check letter. Foreign identification numbers (FIN) follow the same structure with different prefixes: F, G, or M. The regex is simple to write. It catches the clean cases.

It does not catch what clinicians actually type.

Doctors and nurses type fast and under cognitive load. In a realistic corpus of clinical notes, expect NRICs embedded directly adjacent to vital-sign readings with no whitespace: S1234567D130/85mmHg. Expect NRICs with the check digit transposed. Expect phone numbers formatted as eight digits with no spaces, as four-plus-four with a hyphen, and with a full +65 international prefix — all in the same dataset. Expect patient names split across a line break, or abbreviated to initials in one sentence and expanded three sentences later.

A regex keyed to the canonical NRIC pattern misses S1234567 D (space before the check digit) and s1234567d (lowercase). It misses a referral where the nurse typed the patient’s name in all-caps as part of a template header. A regex-only pipeline will miss a meaningful share of PII in such data. For patient referral data, a miss rate in the double digits is not a tuning problem. It is a compliance failure.

Deterministic rules fail on non-deterministic human input. You need semantic understanding just to locate the name.

Deploying Llama 3 8B on an M2 Mac Mini

The fix is a quantized local LLM running inside the clinic network boundary.

Two sensible candidates are Mistral 7B and Llama 3 8B, both quantized to 4-bit precision using GGUF format and served via llama.cpp. Four-bit quantization introduces some precision loss relative to full-weight inference; for a constrained entity-extraction task, that degradation is typically modest, though it should be validated on your own data. The hardware is a standard Apple Mac Mini with an M2 chip and 16 GB of unified memory. Fits in a clinic rack alongside the existing server stack.

A 4-bit quantized Llama 3 8B model loads into approximately 5.5 GB on the M2. At 16 GB unified, that leaves comfortable headroom for the operating system and the surrounding application stack. The M2’s GPU cores handle inference via Metal and short referral notes of a few hundred tokens should fit comfortably.

The system prompt is deliberately narrow. The local model is not asked to understand the clinical content, diagnose the patient, or produce a summary. It receives one instruction: identify all names, NRIC numbers, phone numbers, dates of birth, and residential addresses in the text, and replace each with a typed placeholder. [PATIENT_NAME]. [PATIENT_ID]. [PATIENT_PHONE]. The model returns the redacted text. The application logs the substitution map locally, encrypted at rest, for audit purposes. Only text that passes the redaction gate proceeds to the external API.

This separation of concerns is the architecture’s load-bearing column. The local 8B model handles compliance. The cloud model handles reasoning. Neither is asked to do the other’s job.

The Latency Trade-Off

Suppose a team compares three approaches on the same test set, with human-reviewed PII annotations as ground truth. The expected ordering is clear, even though the exact numbers must come from your own data.

The regex pipeline is fastest, processing each note in milliseconds, but its recall is the lowest, and at that miss rate it fails the compliance bar.

A 4-bit Mistral 7B adds more latency per note and recalls more. Better — but any meaningful miss rate on this data category is still a concern.

A 4-bit Llama 3 8B adds somewhat more latency per note and should recall the most. Any remaining misses would be edge cases where NRICs are embedded inside OCR transcription artifacts — characters garbled beyond recognition. Those defeat any automated approach short of human review.

Whether the added latency is acceptable overhead depends on context. For a referral routing pipeline that processes notes asynchronously, it is. The note arrives, the local model scrubs it, the scrubbed version goes to the cloud API for summarization, and the summary returns. The clinician still gets the result shortly after the note arrives. That additional local inference time is invisible at the workflow level. A synchronous, patient-facing context might demand different latency targets; this architecture is optimized for background processing.

The ROI framing is direct. A small amount of local compute per referral buys a data transfer the compliance team can defensibly sign off on. Under the Personal Data Protection (Amendment) Act 2020, financial penalties for a data breach can reach S$1 million for organizations with annual local turnover at or below S$10 million; for those exceeding that threshold, the cap rises to 10 percent of annual local turnover (Personal Data Protection (Amendment) Act 2020) — well past the cost of this hardware stack in either case. Below that threshold, the reputational cost of a notifiable data breach in a medical context is not recoverable quickly.

Save Your Cloud Budget for Clinical Reasoning

Local LLMs are not toys for privacy hobbyists. They are utility-grade compliance components for enterprise architectures that need both data sovereignty and AI capability.

The pattern generalizes beyond medical referrals. Any workflow combining sensitive on-premise data with cloud AI reasoning faces this boundary problem — and the structural answer is the same each time. A local, quantized model at the edge handles the compliance-sensitive pass on raw data. The cloud model handles the cognitively expensive pass on clean data.

For a medical group evaluating cloud AI for clinical documentation, the practical rule is this: do not pay cloud API rates to find a name. A 4-bit 8B model on commodity hardware can find the name quickly, inside your network; the defensible compliance argument comes from pairing that pass with confidence thresholds, local audit logs, and human fallback for the spans it cannot clear — not from the recall number alone. What actually requires a hosted large model is interpreting what the patient was referred for, cross-referencing against clinical guidelines, and drafting a structured summary a specialist can act on.

Build the pipeline in two stages. Stage one: local inference, PII removed, audit log written. Stage two: cloud inference, clinical reasoning applied, structured output returned. The network boundary is crossed only after the redaction gate passes — with fallback handling covering the spans the local model cannot clear.

That is not a trade-off between privacy and capability. It is a correct separation of concerns. The compliance officer and the CTO can both say yes to it.