Remove Personal Data Before an AI Model Sees It

Leaky context mitigation fails when you treat the visible prompt as the whole payload. A model can also receive the customer object behind it, retrieved passages, tool results, attached files, and metadata added by your own request wrapper.

I hold the line earlier. On a patient intake system I built, the public path never collected the sensitive health detail in the first place. That same rule belongs at the boundary of every external model call.

The short version: Leaky context mitigation strips an outbound request down to fields the task needs before any third-party model sees it. Start with a field allowlist, then detect PII and secrets inside every surviving value, replace identifiers with redaction marks or stable local tokens, apply a policy decision, and serialize only the approved object. Run the last scan on the exact bytes leaving your process. Keep token maps, raw files, and rejected payloads local. If the final scan is uncertain, fail closed and send nothing.

What can leak into a third-party model call?

The user message is one input. The real request is an envelope assembled by several parts of your application, often seconds apart.

Channel	What commonly rides inside	Boundary rule
Form or chat input	Names, email addresses, phone numbers, account notes	Keep only the text needed for the stated task
Retrieved context	Old tickets, document fragments, customer history	Filter every passage after retrieval, not only at ingestion
Tool output	Full API responses, headers, internal IDs, error bodies	Project each result into an approved response shape
Nested JSON	Billing objects, contact arrays, hidden metadata	Walk every object and array before serialization
Files and images	EXIF data, faces, signatures, document properties	Extract through a separate file policy or reject the file
Logs and traces	Raw prompts, tokens, rejected payloads	Record reason codes and hashes, never the sensitive value

OWASP LLM02:2025 names PII, financial details, health records, business data, credentials, and legal documents as sensitive information that can leak through an LLM application. Its recommended controls include tokenization and redaction before processing. The word before carries the weight. Output filtering cannot pull data back from a provider that already received it.

A text scrubber does not read a photograph’s metadata, a spreadsheet’s hidden sheet, or a PDF attachment embedded inside another PDF. If a workflow cannot inspect a file type, it should refuse that type. Converting an unknown file to text and hoping the converter drops every secret is not a policy.

How does leaky context mitigation work?

I use one outbound control path for every caller:

• Allow fields. Build a new object from approved paths. Never subtract a few known bad keys from the original object.
• Detect sensitive values. Scan plain text, nested values, retrieved passages, tool results, headers, and extracted file text for identifiers and secrets.
• Transform locally. Redact a value when identity adds nothing. Use a stable token when the model must tell that two mentions refer to the same person.
• Decide by policy. Allow, transform, or deny. The caller cannot override this result with a prompt flag.
• Serialize last. Scan the exact final string, enforce a size ceiling, then send those bytes and no others.

This order follows the principle behind data minimization for sensitive documents. Article 5 of EU Regulation 2016/679 says personal data must be adequate, relevant, and limited to what is needed for the purpose. The same article requires appropriate security and makes the controller responsible for proving compliance. An allowlist gives you a record of what the system intended to disclose. A blacklist gives you a list of things someone remembered to remove.

The company gets a smaller breach surface, clearer vendor review, and logs it can retain without creating a second sensitive store. The data subject gets the stronger benefit. Their identity never leaves the local boundary when the task only needs the facts around it.

Why does the allowlist come before PII detection?

Detection has false negatives. A detector may recognize jane@example.com and miss a client number that only your company knows is identifying. It may remove a patient name and leave a rare job title, a street, and a precise appointment time that identify the same person together.

NIST SP 800-188, published in September 2023, treats de-identification as a risk decision rather than a masking trick. NIST tells agencies to define the sharing model, set measurable performance levels, and test re-identification risk. It also warns that tools which merely mask personal information may not perform full de-identification.

That is why the first gate removes whole categories of data. If the model is classifying a request, it may need request_text and priority. It does not need the contact object, payment history, browser fingerprint, or the database row copied from a convenience function. Detection then works on a much smaller surface.

HHS makes the same point from a health-data angle. Its HIPAA de-identification guidance says the standard does not distinguish between structured fields and free text. A listed identifier has to be removed wherever it appears, and a rich clinical narrative may still identify someone through context. Renaming patient_name does nothing when the progress note says who the patient is.

How do you sanitize nested JSON before the call?

Build the outbound shape explicitly. The example below is runnable in Node.js. It keeps approved nested values, replaces common identifiers, creates a stable local token for account references, and never copies the source object into the request.

import { createHmac } from "node:crypto";

const key = process.env.PII_TOKEN_KEY;
if (!key) throw new Error("PII_TOKEN_KEY is required");

const patterns = [
  ["EMAIL", /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi],
  ["PHONE", /\+?\d[\d ()-]{7,}\d/g],
  ["BEARER", /\bBearer\s+[A-Za-z0-9._~+\/-]+=*/gi],
  ["API_KEY", /\b(?:sk|pk)_[A-Za-z0-9_-]{16,}\b/g],
];

function stableToken(kind, value) {
  const digest = createHmac("sha256", key)
    .update(`${kind}:${String(value).trim().toLowerCase()}`)
    .digest("hex")
    .slice(0, 16);
  return `[${kind}_${digest}]`;
}

function scrubText(value) {
  return patterns.reduce(
    (text, [kind, pattern]) => text.replace(pattern, `[${kind}_REDACTED]`),
    String(value ?? ""),
  );
}

export function buildOutbound(input) {
  return {
    ticketId: stableToken("TICKET", input.ticket.id),
    request: scrubText(input.ticket.request),
    priority: input.ticket.priority,
    context: (input.retrieved ?? []).map((item) => ({
      source: stableToken("SOURCE", item.id),
      text: scrubText(item.text),
    })),
  };
}

This is a boundary example, not a universal PII detector. Phone patterns vary by country. Names do not follow a regular expression. Internal identifiers need a company dictionary. Secret formats change. In a real system, I add domain recognizers and tests taken from the data the system is allowed to process.

Presidio’s current text anonymization documentation separates detection from the action applied to a detected entity. Its analyzer finds candidate PII, and its anonymizer can redact, replace, hash, or encrypt it. The action follows local policy.

When should you redact, tokenize, or keep a local map?

Redaction answers, “the value has no job here.” Stable tokenization answers, “the model needs continuity, not identity.” A reversible local map answers, “an authorized process must restore identity after the external result returns.”

Method	Use it when	Main risk
Redaction	The model can complete the task without knowing whether two mentions match	Too much removal may erase useful context
Stable keyed token	The model must group repeated references across one job or approved series	A weak or shared key can expose patterns
Random token plus local map	An authorized local step must restore the original value	The map becomes sensitive data that needs its own access and deletion rules
Encryption sent with payload	Almost never for a model call, since the provider cannot use unreadable text	Sending the key or reversible mechanism defeats the boundary

Keep stable tokens scoped. A token that stays the same across every product, customer, and year becomes a tracking identifier of its own. Derive it with a secret key and include a tenant or job scope in the HMAC input.

For regulated health data, token design needs legal and statistical review. The HHS guidance permits a covered entity to assign a re-identification code only under specific conditions, including keeping the re-identification mechanism undisclosed. It also explains that a code derived from PHI may require the Expert Determination path. A hash alone is not a declaration that data has left HIPAA.

Store a reversible map in a local encrypted table with a short expiry, a named purpose, and a service identity allowed to read it. Do not place the map in the model prompt, a trace viewer, or the same queue message as the tokenized payload. The external result comes back with tokens. A separate local step restores only the fields the next authorized action needs.

What should the final policy gate reject?

The final gate reads the serialized payload because serialization can add fields, expand templates, or stringify an error object after the earlier scan. It rejects known sensitive keys, residual detector hits, unapproved file references, and payloads larger than the task contract.

import { createHash } from "node:crypto";

const forbiddenKeys = /(?:password|secret|token|authorization|ssn|dob)/i;
const residual = [
  /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i,
  /\bBearer\s+[A-Za-z0-9._~+\/-]+=*/i,
  /\b(?:sk|pk)_[A-Za-z0-9_-]{16,}\b/i,
];

function inspect(value, path = "$", findings = []) {
  if (Array.isArray(value)) {
    value.forEach((item, index) => inspect(item, `${path}[${index}]`, findings));
  } else if (value && typeof value === "object") {
    for (const [key, child] of Object.entries(value)) {
      if (forbiddenKeys.test(key)) findings.push({ path: `${path}.${key}`, type: "KEY" });
      inspect(child, `${path}.${key}`, findings);
    }
  } else if (typeof value === "string") {
    residual.forEach((pattern, index) => {
      if (pattern.test(value)) findings.push({ path, type: `PATTERN_${index}` });
    });
  }
  return findings;
}

export function approveAndSerialize(payload) {
  const findings = inspect(payload);
  const body = JSON.stringify(payload);
  if (Buffer.byteLength(body) > 64_000) findings.push({ path: "$", type: "TOO_LARGE" });
  if (findings.length) {
    const auditHash = createHash("sha256").update(body).digest("hex");
    throw new Error(`OUTBOUND_DENIED ${auditHash} ${findings.map((f) => f.type).join(",")}`);
  }
  return body;
}

Notice what the error omits. It records a payload hash and low-cardinality reason types. It does not print the blocked value. Rejected prompts are often the most sensitive prompts in the system, so logging them raw turns a successful block into a new leak.

OpenAI’s endpoint data controls, checked on August 4, 2026, show why local filtering still matters even when a provider offers controls. Chat Completions and Responses list 30-day abuse-monitoring retention by default. Responses also stores application state for at least 30 days by default, subject to endpoint details. Approved customers can configure Zero Data Retention, but eligibility and endpoint behavior vary. A retention setting changes what happens after receipt. It does not make an unnecessary disclosure necessary.

How do you handle false negatives and outages?

No detector catches every identifier. Treat detection as one control inside a smaller disclosure design, then measure it like any other production guard.

• Seed hostile fixtures. Put identifiers in nested arrays, quoted email threads, OCR text, tool errors, filenames, and Unicode lookalikes.
• Test the serializer. Assertions belong on the final body, not the object you intended to send.
• Track detector recall by data class. Names, account codes, and rare free-text clues fail in different ways.
• Review false positives. Over-redaction can make the task wrong, which pushes teams to bypass the control.
• Deny on uncertainty. If the detector, policy service, file extractor, or token vault is unavailable, queue the job locally or send it to a human.
• Keep an outbound ledger. Store time, caller, policy version, model endpoint, allowed field names, body hash, and decision. Leave values out.

This gate also belongs after tools return. A tool can fetch more than the model asked for, and prompt injection can turn a reader into a writer if one process both consumes untrusted text and performs outbound actions. Filter every new boundary crossing. Trust does not carry forward because an earlier step passed.

The company gains a call it can explain. The person behind the data gains something better: less of their life leaves the system at all.

The safest external payload is the smallest one the task can use.

Frequently asked questions

What is leaky context mitigation?

Leaky context mitigation is the outbound control path that limits what a third-party model receives. It builds a new object from approved fields, scans every surviving value for PII and secrets, transforms identifiers locally, applies an allow or deny policy, and scans the final serialized body before transmission. It covers retrieved text, nested JSON, tool output, files, and metadata as well as the visible prompt.

Is redacting names and email addresses enough before an AI call?

No. Direct identifiers are one part of the risk. A rare title, precise location, appointment time, account code, or several harmless-looking facts can identify someone together. NIST treats de-identification as a measured disclosure-risk decision, and HHS warns that rich free text may identify a person through context. Start by removing unneeded fields, then detect and transform what remains.

Is stable tokenization the same as anonymization?

No. A stable token preserves linkage, which may be useful when a model has to group repeated mentions, but that linkage can still carry privacy risk. A keyed token can be pseudonymous without being anonymous. Scope tokens to a tenant or job, protect the key, set an expiry, and keep any reversible map local. Regulated de-identification claims need the method and review required by the governing rule.

Should blocked model payloads be saved for debugging?

Do not save them raw by default. A blocked payload is likely to contain the exact PII or secret the gate caught. Record the body hash, policy version, caller, allowed field names, detector types, and decision. If a security review needs the original, use a separate access-controlled capture path with a short expiry and an explicit case rather than a general application log.

Do provider retention controls remove the need to filter PII locally?

No. Retention controls govern what a provider stores after it receives data, and the exact behavior differs by endpoint and account setting. Local filtering answers an earlier question: whether the provider needs that value at all. Use both controls. Send the minimum approved payload, set the narrowest supported retention behavior, and verify the endpoint rather than applying one provider-wide assumption.