Building PII Detection and Redaction Pipelines for LLM Inputs and Outputs

Imagine sending a customer’s full name, credit card number, and medical history to an AI model just to summarize their support ticket. It sounds reckless, but it happens more often than you’d think. Without proper safeguards, Personally Identifiable Information (PII) is sensitive data that can identify an individual, such as names, addresses, or financial details leaks into logs, training datasets, or third-party servers. This isn’t just a compliance headache; it’s a direct threat to user trust and legal standing under regulations like GDPR and HIPAA.

The solution? A robust pipeline that detects and redacts PII before it ever touches the Large Language Model (LLM) and again when the response comes back. This guide breaks down how to build these pipelines, the tools you should use, and the architectural patterns that keep your data safe without killing performance.

Why You Can't Trust LLMs with Raw Data

You might assume that because you own the LLM API key, your data stays private. That’s a dangerous assumption. Most commercial LLM providers log inputs for quality assurance, security monitoring, or even future model training unless explicitly opted out-and even then, accidental leakage in system traces is common.

When you send raw text containing PII to an external service, you lose control over that data. If a breach occurs on their end, or if an employee accesses the logs, your users’ sensitive information is exposed. The only way to mitigate this risk is to sanitize the data at the edge-meaning, right before it leaves your application and right after it returns.

This creates two critical checkpoints in your architecture:

• Input Sanitization: Scanning user prompts to remove or mask PII before sending them to the LLM.
• Output Validation: Checking the LLM’s response to ensure it hasn’t hallucinated or leaked PII from its training data or previous context windows.

The Hybrid Detection Strategy: Speed vs. Accuracy

Building a PII detector requires balancing speed and accuracy. If you rely solely on simple pattern matching, you’ll miss contextual clues. If you use heavy AI models for every request, your latency will skyrocket. The industry standard is a hybrid approach.

First, you use Regular Expressions (Regex) is a sequence of characters that specifies a search pattern used for string matching as a fast-pass filter. Regex is incredibly quick at finding structured data like email addresses (`[email protected]`), phone numbers (`555-0199`), and credit card numbers. However, it fails miserably with unstructured data. It can’t tell the difference between "John" as a name and "john" as a variable, nor can it easily detect complex entities like social security numbers embedded in natural language sentences.

To catch what Regex misses, you layer on Named Entity Recognition (NER) is an NLP task that identifies and classifies key information in text into predefined categories. NER models understand context. They know that "Apple" refers to a company in one sentence and a fruit in another. By combining Regex for speed and NER for depth, production systems achieve recall rates of up to 96%, meaning only 4% of PII slips through. Compare that to Regex-only baselines, which often miss up to 35% of sensitive data.

Core Tools for PII Detection

You don’t need to build a detector from scratch. Several mature libraries and services handle the heavy lifting. Here are the most effective options available in 2026:

Microsoft Presidio remains the go-to choice for many engineers due to its flexibility. It allows you to define custom regex patterns and integrate pre-built recognizers for emails, phones, and IPs. For organizations already using Azure or Microsoft Fabric, native AI functions like `ai.extract` offer seamless integration, though they may have rate limits (e.g., 1,000 requests per minute).

If you’re working with massive datasets in batch mode, combining Presidio with PySpark is efficient. For real-time applications where milliseconds matter, consider fine-tuning smaller models or using optimized NER pipelines rather than full LLM inference for detection.

Architecting the Pipeline: Microservices Approach

A monolithic app that handles both business logic and PII scanning is a bottleneck. Instead, decouple your PII detection into a dedicated microservice. This allows you to scale detection independently from your core application.

Here’s a typical high-performance architecture:

1. Interception: An API Gateway or sidecar proxy intercepts outgoing requests to the LLM.
2. Caching Check: Before processing, check a local cache (like Redis) to see if this exact prompt has been scanned recently. This saves compute power for repeated queries.
3. Detection Service: Send the text to a specialized PII Detection Service. This service might be written in Python, leveraging libraries like spaCy or Presidio.
4. Communication: Use gRPC for low-latency communication between your Go-based application server and the Python detection service.
5. Redaction Policy: The service returns a list of identified entities and their positions. Your application applies a masking policy (e.g., replace `[NAME]` with ``).
6. Forwarding: The sanitized prompt is sent to the LLM.

For outputs, repeat the process. The LLM’s response goes through the same detector. If PII is found, you have two choices: mask it completely or use placeholder replacement if the user needs to see the original data later (though this requires secure storage of the mapping).

Implementation Steps: From Code to Production

Let’s look at how this translates into code. Assume you’re using Node.js for your backend and Python for your detection service.

Step 1: Set Up the Detection Endpoint Your Python service exposes an endpoint that accepts JSON payloads. It runs the text through Presidio.

from presidio_analyzer import AnalyzerEngine
import json

analyzer = AnalyzerEngine()

def detect_pii(text):
    results = analyzer.analyze(text=text, language='en')
    # Return entities and their positions
    return [{"entity": r.entity_type, "start": r.start, "end": r.end} for r in results]

Step 2: Intercept and Sanitize in Application Logic In your main app, call this endpoint before hitting the LLM API.

async function sanitizePrompt(prompt) {
  const response = await fetch('http://pii-service/detect', {
    method: 'POST',
    body: JSON.stringify({ text: prompt })
  });
  const entities = await response.json();
  
  // Replace identified spans with placeholders
  let sanitized = prompt;
  for (const entity of entities.sort((a, b) => b.start - a.start)) {
    const replacement = `<${entity.entity}>`;
    sanitized = sanitized.slice(0, entity.start) + replacement + sanitized.slice(entity.end);
  }
  return sanitized;
}

Step 3: Handle Output Redaction Apply the same `sanitizePrompt` function to the LLM’s response before displaying it to the user. This prevents the model from accidentally revealing PII it memorized during training.

Performance Trade-offs and Optimization

Accuracy is useless if your app takes 10 seconds to respond. Here’s how different approaches impact latency:

• Regex Only: <1ms latency. High false-negative rate. Good for structured fields only.
• NER Models (spaCy/Presidio): 50-200ms latency. High accuracy. Best balance for most apps.
• LLM-based Detection: 500ms+ latency. Highest semantic understanding. Expensive and slow.

To optimize, implement caching aggressively. If a user sends the same question twice, don’t scan it again. Also, consider asynchronous processing for non-critical paths. For example, if you’re analyzing historical logs for compliance, you can run batch jobs overnight rather than in real-time.

Another pro tip: limit the scope of detection. Don’t scan entire documents if you only care about the first paragraph. Define clear boundaries for what constitutes a "prompt" in your system.

Compliance and Legal Considerations

Building the pipeline is only half the battle. You must align it with regulatory requirements.

• GDPR: Requires data minimization. You shouldn’t collect PII unless necessary. Redaction helps prove you’ve minimized data exposure.
• HIPAA: Strictly regulates health information. Any mention of diagnoses, treatments, or provider names must be scrubbed.
• PCI-DSS: Protects payment data. Credit card numbers and CVVs must never touch the LLM.

Keep detailed audit logs of what was detected and redacted. In case of a dispute, you need evidence that your pipeline worked correctly. Note that multilingual support is still a challenge; detection accuracy drops significantly for languages other than English. If you serve global users, test your pipeline thoroughly with non-English inputs.

Testing Your Pipeline

How do you know your detector works? You need synthetic data. Real PII is risky to use for testing. Use tools like the NLU-Redact-PII is a GitHub repository providing tools for generating synthetic PII datasets project to generate fake names, addresses, and IDs. Create test cases that include:

• Obvious PII (e.g., "My SSN is 123-45-6789")
• Contextual PII (e.g., "John Smith called yesterday")
• Edge cases (e.g., partial matches, typos)
• False positives (e.g., "I love apple pie" should not flag "apple" as a company)

Measure your precision and recall regularly. Aim for high recall (catching all PII) even if it means some false positives (masking non-PII). It’s better to annoy a user by masking "John" unnecessarily than to leak their actual Social Security Number.