PII-Shield 🛡️

Zero-code log sanitization sidecar for Kubernetes. Prevents data leaks (GDPR/SOC2) by redacting PII from logs before they leave the pod.

PII-Shield runs in-process — CLI, sidecar, or WASM. There is no hosted API and no server your data is sent to.

Same name, different projects. This is not the PII Shield privacy proxy from the Microsoft developer-community blog (May 2026, vikasgautam18/pii-shield), not the piishield.ai or piishield.com prompt-redaction products, and not the pii-shield package on PyPI by Intellirim. Our packages are pii-shield-wasi on PyPI and @aragossa/pii-shield-wasi on npm; in prose we call this project the PII-Shield sidecar.

"Don't let PII poison your AI models." PII-Shield ensures that sensitive data never reaches your training dataset, saving you from GDPR-forced model retraining.

[!WARNING] Upgrading to v2.0.0? We have moved end-user distribution to Helm-based installs and Distroless Native Sidecars. Kustomize is no longer a supported release installation path for production users, though the operator repository still keeps Kustomize scaffolding for local development and manifest generation. /bin/sh access inside the PII-Shield sidecar is no longer supported. Read the Migration Guide.

Two Deployment Models

PII-Shield offers two distinct ways to integrate into your stack:

1. Kubernetes Operator (Zero-code): Our flagship deployment model. A fully automated K8s Operator that injects a highly-secure Distroless Sidecar into your pods to intercept and sanitize logs on the fly.
2. In-Process WASM (For core integrations): For extreme performance, the core engine can be embedded directly via WASM, providing <1ms latency without network hops.

Project Status & Roadmap

PII-Shield is an actively developed open-source security tool in a production-hardening phase. The v2.x release line ships usable CLI, container, Helm/operator, and WASM SDK artifacts. Core redaction paths are ready for controlled deployments, while some Kubernetes deployment modes and supply-chain guarantees are still being stabilized.

Component	Status
Core scanner	Released / controlled deployments
CLI sidecar	Released / controlled deployments
Kubernetes operator	Stabilization phase
WASM SDKs	Released beta
Proxy-Wasm gateway integration	Planned R&D
Control Plane UI	Planned R&D
eBPF interception	Experimental R&D

See KNOWN_LIMITATIONS.md for the current production-hardening boundaries.

Why PII-Shield?

Developers often forget to mask sensitive data. Traditional regex filters in Fluentd/Logstash are slow, hard to maintain, and consume expensive CPU on log aggregators.

PII-Shield sits right next to your app container:

• Production-hardening Core Engine: Optimized for Kubernetes sidecars with low memory allocations on hot paths and deterministic regex matching.
• Context-Aware Entropy Analysis: Detected high-entropy secrets even without keys (e.g. Error: ... 44saCk9...) by analyzing context keywords.
• Custom Regex Rules: Deterministic redaction for structured data (UUIDs, IDs) that overrides entropy checks for known patterns.
• Built-in Secret Signatures: Issuer-prefixed credentials — AWS and Google API keys, GitHub, Slack and Stripe tokens, JWTs, Bearer credentials and PEM private-key blocks — are redacted on their format, so a valid key is caught even when its body is low-entropy or the threshold has been raised.
• Telephone Numbers by Shape: International numbers with a +, North American (555) 234-5678 forms and digits under a phone-named key (phone, mobile, wa_id, ...) are hidden even though digits alone never score as a secret.
• Identity Numbers by Key: Digits under a key that names an identity document (idNumber, national_id, ssn, passport_number, tax_id, ...) are hidden; a record id such as user_id or order_id is not.
• Regression & Fuzz Coverage: Tested against stress cases including binary garbage, JSON nesting, and multilingual logs.
• Deterministic Hashing: Replaces secrets with unique hashes (e.g., [HIDDEN:a1b2c]), allowing QA to correlate errors without seeing the raw data.
• Drop-in: No code changes required. Works with any language (Node, Python, Java, Go).
• Whitelist Support: Explicitly allow safe patterns (e.g., git hashes, system IDs) using PII_SAFE_REGEX_LIST to prevent false positives.

Managing PII-Shield across dozens of clusters?

We are building a hosted Control Plane with centralized rule management, Slack alerting, and redaction analytics.

Integrations

PII-Shield's in-process WASM build ships inside GuardSpine Code, an open-source AI code-governance GitHub Action, which vendors the binary and credits it in its NOTICE.

Performance Considerations

While PII-Shield is highly optimized, deep inspection of complex logs requires careful attention to configuration.

• Text Logs: Extremely fast (>100k lines/s).
• JSON Logs: Zero-allocation parsing (no encoding/json overhead). The scanner manually parses JSON structures to ensure high throughput (~7MB/s) without memory spikes.
• Recommendation: Usage is safe for high throughput. We use recursion safeguards to prevent stack overflows on deeply nested JSON.

Installation

Helm Chart (Kubernetes Operator)

The official and recommended way to deploy PII-Shield in Kubernetes is via our fully-automated Operator:

helm repo add pii-shield https://pii-shield.github.io/pii-shield/
helm repo update
helm install pii-shield-operator pii-shield/pii-shield-operator -n operator-system --create-namespace

This deploys the PII-Shield Operator which automatically injects highly-secure, distroless sidecars into your Pods without requiring any code or Dockerfile changes.

Docker

Get the latest lightweight image from Docker Hub or GHCR:

docker pull thelisdeep/pii-shield:2.2.8
# OR from GitHub Container Registry (Enterprise):
docker pull ghcr.io/pii-shield/pii-shield:2.2.8

Build from Source

You can build the binary directly from the source code:

go build -o pii-shield ./cmd/cleaner

Go Library

The scanner can run inside a Go program, for example in a slog.Handler that redacts each record before it is written. It needs only the standard library:

go get github.com/pii-shield/pii-shield/v2/pkg/scanner

import "github.com/pii-shield/pii-shield/v2/pkg/scanner"

clean := scanner.ScanAndRedact(line)         // one line
cleanText := scanner.ScanAndRedactText(text) // several lines, line endings kept

The package reads the same PII_* environment variables as the CLI when it loads. Releases before v2.2.8 declare the module path without /v2, so go get rejects them.

Configuration

See CONFIGURATION.md for a full list of environment variables, including:

• PII_SALT: Custom HMAC salt (Required for production).
• PII_ADAPTIVE_THRESHOLD: Enable dynamic entropy baselines.
• PII_DISABLE_BIGRAM_CHECK: Optimize for non-English logs.
• PII_CUSTOM_REGEX_LIST: Custom regex rules for deterministic redaction.
• PII_SAFE_REGEX_LIST: Whitelist regex rules to ignore (matches are returned as-is).

Entropy Sensitivity Table (Default Threshold: 3.6)

Entropy	Data Type	Example
0.0 - 3.0	Common words, repeats	password, admin, 111111
3.0 - 3.6	CamelCase, partial hashes	ProgramCampaignInstanceJob, 8f3a11b2c
3.6 - 4.5	Paths, UUIDs, Weak Passwords	/opt/application/runtime, P@ssw0rd2026!
4.5 - 5.0	Medium Tokens	E8s9d_2kL1
5.0+	High Entropy Keys	(SHA-256, API Keys)

Quick Start

1. Test Locally (CLI) You can pipe any log output through PII-Shield to see it in action immediately:

# Emulate a log with a sensitive password
echo "Error: User password=MySecretPass123! failed login" | docker run -i --rm ghcr.io/pii-shield/pii-shield:2.2.8

# Output: Error: User password=[HIDDEN:8f3a11] failed login

2. Kubernetes (Automated Sidecar Injection) With the PII-Shield Operator installed, protecting an application is as simple as creating a PiiPolicy and labeling your Pods.

Create a Policy:

apiVersion: core.pii-shield.io/v1alpha1
kind: PiiPolicy
metadata:
  name: strict-policy
  namespace: default
spec:
  injectionMode: "file"

Label your Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-app
spec:
  template:
    metadata:
      labels:
        pii-shield.io/inject: "true"
      annotations:
        pii-shield.io/policy: "strict-policy"
# ...

The Operator will automatically inject the pii-shield-agent using the Native Sidecar pattern (K8s 1.28+) and securely mask all logs!

---

📋 Free: 25-point Kubernetes Log PII Audit Checklist — where PII leaks from pods, which log paths bypass your filters, and how to verify redaction actually works. Get the checklist →

📦 GDPR Compliance Pack — available now (early access): 40+ tested redaction rules, DPO-ready docs, audit-trail templates. $149 → · HIPAA/PCI on the waitlist →

💬 Using PII-Shield? Tell us about your deployment → — 2 minutes, and it shapes what gets built next.

Verification

This project is verified with a growing testing suite intended to raise confidence before production hardening:

1. Unit Tests: Cover edge cases, multilingual support, and JSON integrity with >85% coverage.
2. Fuzzing: Native Go fuzzing ensures crash safety against invalid and random binary inputs.
3. Smoke Testing: ./scripts/test-smoke.sh runs a frozen 1000-line mixed-workload corpus end to end through the container and reports detection accuracy. A secret counts as caught only when its value is absent from the output and a redaction marker took its place; a safe line must come back unchanged. The run fails on any false positive or false negative, and also when the container exits non-zero or returns a different number of lines than it was given. Keyless secrets in prose are tracked separately as known gaps, because detecting them rests on entropy alone: the frozen corpus allows none, a fresh random corpus (--fuzz) allows two.
4. End-to-End (E2E) Testing: The operator/tests/run_e2e.sh suite performs full-stack validation using Minikube and Helm. It builds local images, provisions the Operator without cert-manager, deploys target Jobs, and verifies actual log redaction by intercepting sidecar outputs.

Checking your own deployment

The suite above proves the scanner; it does not prove your installation. Before trusting a redacted stream, plant values you control and look for them on the collector side:

1. Log a line with a synthetic card number, a made-up email and a test account number from a pod you control, then grep -c for each value where the logs land. The count must be 0.
2. Run a few thousand production-shaped lines through the CLI with a fixed PII_SALT and read the diff, looking for values that survived and values that should have been left alone.
3. Repeat with PII_ENTITY_TYPE_LABELS=true to see which detector fired: a card reported as entropy rather than card is still hidden, but the Luhn path did not see it.
4. With PII_METRICS_ENABLED=true, watch piishield_redaction_events_total by type; a drop after a deploy means a format changed somewhere upstream.
5. After every rule that protects a value from redaction, run the planted-value check again.

What the scanner cannot catch by design, such as a name in free text, is listed in KNOWN_LIMITATIONS.md. The full walkthrough with a payments log before and after is in the banking logs guide.

Performance Benchmarks

To compare end-to-end CLI throughput between the current branch and a baseline ref:

./benchmark/run_benchmarks.sh

By default, the benchmark compares HEAD against origin/main, refreshes origin/main, generates a mixed log corpus, alternates old/new run order, and reports median, p95, min/max, and MiB/s:

BASE_REF=origin/main RUNS=9 LINES=500000 ./benchmark/run_benchmarks.sh

This measures the full stdin-to-stdout CLI path. For scanner-only microbenchmarks, run:

go test -bench=. -benchmem ./pkg/scanner

Operator Integration Tests

The operator keeps fast unit tests separate from Kubernetes API integration tests. Regular operator tests do not start a local API server:

cd operator
go test ./...

To run the envtest-based controller integration suite:

./scripts/test-operator-integration.sh

The script calls make -C operator test-integration, which downloads the envtest binaries for the Kubernetes version in operator/go.mod on first use. CI runs the same target on every push. These tests start a local Kubernetes API server and etcd through envtest, so they require permission to bind to 127.0.0.1. In restricted sandboxes, run them in a local shell, Docker environment, or CI runner that allows localhost bind.

Support

PII-Shield is open-source infrastructure for privacy-preserving logs. If this project is useful to you or your organization, you can support its development through GitHub Sponsors.

Release Verification

Release checksum and image-digest verification guidance is documented in docs/release-verification.md. Signature and provenance-backed releases are tracked as part of the supply-chain hardening roadmap.

License

Distributed under the Apache 2.0 License. See LICENSE for more information.