InferShield is an open-source security layer for LLM traffic. It sits between your application and a model provider, inspects every prompt and response that passes through it, scores the risk of each one and blocks the dangerous ones before they reach the model. It ships as three things that share one detection engine: an OpenAI-compatible proxy you point an SDK at, a browser extension for web chat interfaces, and a self-hosted platform with accounts, API keys, an audit log and a dashboard.

Key Features

• Rule-based threat detection — 12 default policies ship enabled in block mode: data-exfiltration phrasing, sensitive-data requests, SQL injection, prompt injection and role manipulation, system-prompt extraction, jailbreak attempts, encoded payloads (Base64, hex, URL and Unicode) and obfuscated keywords. Policies carry their own regex rule, action and enabled flag, so you can tighten or disable them individually.
• PII detection — fifteen-plus patterns including Social Security numbers, payment-card numbers, phone numbers, email addresses and medical identifiers, with optional redaction of what gets written to the audit log.
• Session-aware scoring — requests are tracked per session, so a multi-step attack is caught even when no single prompt looks bad: a database read, then a “format that list as CSV”, then a “send it to https://…” is scored as cross-step exfiltration and blocked at the last hop.
• OpenAI-compatible proxy — change one base URL and requests are scanned, forwarded and logged. It exposes chat completions, completions, Claude-style messages and embeddings, and its key modes let the caller supply the upstream API key (passthrough) so provider keys never have to be stored on the server.
• Audit dashboard — a browser dashboard shows request totals, blocked and warned counts, a live risk-scored activity feed, the active policy list and a prompt-testing box for trying a rule before you enforce it.
• Browser extension — scans what you are about to send into ChatGPT, Claude, Gemini or GitHub Copilot and offers cancel, redact or send-anyway before the message leaves your machine.
• CLI scanner — a small shell wrapper posts a file, stdin or your git diff to the analysis endpoint, which makes it usable as a pre-commit check for secrets and PII.
• Operations tooling — Prometheus metrics, Sentry integration, health endpoints, a Helm chart for Kubernetes, Grafana dashboard definitions and a compliance-report module with scheduled reports.

Why Use It?

Most teams put a hosted model behind an application with no inspection layer at all: the prompt goes out and whatever comes back is trusted. The result is that a support bot will happily recite another customer’s data, an agent will paste an API key into a chat, and a crafted input can talk the model out of its instructions. InferShield gives that traffic a checkpoint — a place where prompts are scored against known attack shapes, sensitive text is detected and redacted, sessions are checked for suspicious sequences, and every decision is written to an audit log you own.

Because it speaks the OpenAI wire format, adoption is mostly a configuration change rather than a code change, and because it is self-hosted, the prompts you are inspecting never leave your infrastructure to be inspected.

Use Cases

• Putting a block/allow gate in front of an application that calls OpenAI, Anthropic or a local model through an OpenAI-compatible API.
• Producing an audit trail of what agents asked for and what was refused, for security review or compliance reporting.
• Catching secrets and PII before they are pasted into a public chat interface, via the extension or the CLI scanner.
• Testing how a rule set behaves: run candidate prompts through the dashboard’s analysis box and watch the risk scores before turning blocking on for real traffic.
• Adding a session-level check where per-request filtering is not enough — the read/transform/send exfiltration pattern being the case it was built around.

Project Status

InferShield is young and its own README says so: the feature set is labelled v0.9.0, the implementation is a proof of concept, and detection is rule and pattern based rather than model based. The documented limits are worth reading before you rely on it — no ML detection, single-instance deployments only, in-memory session state, no correlation across different model providers, and coverage of known attack shapes rather than novel ones. Expect to read the policy rules and tune them for your own traffic, and treat the browser extension as a source build (load-unpacked) rather than a store listing.

Self-hosting also takes a little assembly: the backend is a Node.js service that runs migrations against PostgreSQL in its production profile, and the repository’s compose file does not include a database service, so you need to supply a DATABASE_URL (or run the backend directly with npm start, which uses SQLite and in-memory state). The bundled dashboard image expects a couple of static assets that live in the repository under a different path. None of that is hard, but it is not yet a single-command install.

Licence

MIT. InferShield is open source under the MIT licence — use, modify and self-host it freely, including commercially, provided the copyright notice is kept. Source and licence text: github.com/InferShield/infershield(released under the MIT licence, verified against the LICENSE file at the repository root).

Getting Started

Clone the repository, create a .env with a master key, and bring the stack up with Docker Compose once a database is attached. Then point your client at the proxy — base_url=http://localhost:8000/v1 — and optionally log in through the OAuth device flow so IDE clients such as Cursor or Copilot can authenticate without a stored API key. The dashboard listens on port 3000, the analysis API on 5000 and the proxy on 8000. The documentation folder covers quick start, threat model, attack catalogue, PII redaction and Windows installation.