Personal data should not reach the model: what we built and what we measured

Names, phone numbers and national IDs are swapped on the way to the model and restored in the reply. Passports and API keys are never sent at all. The basic level costs 34 MB and 0.8 ms; the full one costs a gigabyte and 12 ms.

An agent receives exactly what a person writes: "call Margaret Hale, here is her number", "check the invoice against this IBAN", "here is the record from the database". All of it then travels to a cloud model in full, because otherwise the model cannot do the task. We made it so that not all of it does.

What happens to the text

text

what was typed — Invoice for Margaret Hale of Northern Mills Ltd, phone +44 20 7946 0958, email m.hale@northern-mills.co.uk, IBAN GB33 BUKB 2020 1555 5555 55

what the model got — Invoice for Cathy Thompson of Gutierrez and Sons, phone +44 00 1846 5951, email dcarpenter@example.net, IBAN GB58 1836 9725 5462 6969 86

what came back — Invoice for Margaret Hale of Northern Mills Ltd, phone +44 20 7946 0958, email m.hale@northern-mills.co.uk, IBAN GB33 BUKB 2020 1555 5555 55

That is not an illustration from the docs: the middle row is what a receiver, put in place of the provider on our own pod, wrote down. The model answers about the stand-ins, and the person reads the real values again. Nothing changes for the agent.

The stand-ins are plausible rather than a "PERSON" placeholder or a row of asterisks, and that is deliberate. A text full of placeholders reads as a template, and models answer templates with templates; a text with an ordinary name and an ordinary phone number reads as ordinary text.

What is refused rather than replaced

A passport number, a national insurance number, a card number, an API key, a JWT, a database connection string — a request carrying any of these is not forwarded at all. The agent gets a 400 and the list of what was found.

The reasoning is simple. Replacement is only as good as the detector, and for these types a miss is unacceptable. Refusing is more reliable than hoping.

Two levels and what they cost

basic — full

Replaces — tax and company IDs, phone numbers in any country, emails, IBAN, BIC, account numbers — all of that plus names, organizations and places

Refuses the request — national insurance and passport numbers, cards, API keys, tokens, connection strings — the same

Memory — 34 MB at start, 44 MB under load — 1046 MB with one language

Added latency — 0.8 ms (median of 30, max 1.4) — 12 ms (median, p95 18)

Start-up — instant — about 40 seconds

Plan — any, including the smallest — medium and up

Status — shipping — shipping, medium plan and up

The basic level is honestly named. Names, company names and places are not detected — those need the language model, which is where the gigabyte goes. In the example above the basic level would have replaced the phone number, the email and the IBAN, leaving the name and the company as they were, and the response says so in a field of its own. In exchange it costs nothing: on the smallest plan it is five percent of the pod's memory.

What was broken in the full level, and how it was fixed

It did not ship at first. A model declines a stand-in name — Russian turns «Фёдотов Аркадий» into «Фёдотовым Аркадием» — and restoration matched the literal string, so the declined form went unrecognized. What came back named an invented person: worse than a miss, because a miss is visible and a plausible falsehood is not. The second problem was blunter: loading a second language pipeline pushed the container past its memory limit, and the agent was left with no model at all.

Both are fixed in the module and verified on a live pod. Declined forms come back real, and a pipeline that will not fit is now an honest refusal with a reason rather than a dead container.

We write this down not out of love for detail. If you run this shield yourself, know where it breaks: restoring across word forms is the part worth testing in your own language, on your own data.

What it looks like in another language

The full level on German text, same receiver in place of the provider:

text

what was typed — Überweisung an Anna Schmidt, IBAN DE89 3704 0044 0532 0130 00, BIC COBADEFFXXX, Frankfurt am Main

what the model got — Überweisung an Ingolf Schüler, IBAN DE09 0762 8428 2880 9322 25, BIC EMNDDEUGUYG, München

what came back — Überweisung an Anna Schmidt, IBAN DE89 3704 0044 0532 0130 00, BIC COBADEFFXXX, Frankfurt am Main

The name became a German name, the city a German city, the IBAN stayed German and kept its grouping, the BIC stayed German. That is not cosmetic: an IBAN from another country would move the money somewhere else rather than hide who sent it.

How to turn it on

The "PII protection" checkbox on the agent's card, then Rebuild. The shield runs as its own container inside the same pod and listens on the loopback only: nothing outside the pod can reach it, neighbouring pods included. That matters more than it sounds — a service holding raw personal data in memory should not be reachable over the network at all.

Today's limits. It works on the Hermes engine with your own key. The shield speaks two request shapes — OpenAI's /v1/chat/completions and Anthropic's /v1/messages — so OpenAI, Anthropic, DeepSeek, OpenRouter, Groq, Mistral and xAI all work; Google's shape is a third one and is not supported yet. The agent's card tells you which of these is in the way.

On a live pod the closed loop with a real model took 2256 ms: DeepSeek answered, and the stand-ins came back as the real values.

Honestly, about accuracy

Open-source PII detection is not a solved problem. On a 2026 cross-domain benchmark the best average F1 among five common approaches was 0.542; Presidio, which we build on, scored 0.481. Anything with a checksum or a fixed shape is found almost perfectly. Names in free text are not.

A restored name also comes back in the nominative, so a Russian reply reads slightly off — the data is right, the grammar is not. Stand-ins also do not always keep the register: in our runs Manchester came out as an invented North Carolburgh, and an account name came back as a person's name. Protection is unaffected, but the text the model reads ends up slightly odder than the original.

So the honest phrasing is: this is risk reduction, not a compliance guarantee. If you need a guarantee, personal data should not reach a cloud model at all — and then the conversation is about a local model, not about a shield.

What we do not know

How this behaves in very long conversations, where stand-ins live across dozens of turns, we have not measured: every number above comes from short exchanges. Detection quality in languages other than Russian, English and German we have not checked ourselves. And we do not know how often a model stumbles over a stand-in and asks again — in our examples it never did, but there were few examples.

The module is open source, MIT, at github.com/radianceteam/pii-shield. You can run it without us, and then check the numbers above yourself.