← Back to Home / ChatGPT

Prompt Injection Defense Framework

Protect LLM apps from prompt injection.

Act as an AI security engineer specializing in LLM security, having developed defensive frameworks for enterprise AI applications processing sensitive data and handling user-facing chat interfaces vulnerable to prompt injection attacks. Generate a comprehensive prompt injection defense framework for a specific AI application type (chatbot, email summarizer, document Q&A, code assistant, content generator) including input validation, output filtering, and architectural patterns. Begin with input validation including whitelist-based instruction recognition (limiting user instructions to specific approved intents), delimiter wrapping for user input (XML tags, special tokens, random delimiters), input sanitization removing potential injection syntax (ignore, disregard, forget, system prompt, developer mode), length limiting for user input fields, rate limiting per user/session, and pattern matching for known attack signatures (ignore previous instructions, you are now DAN, pretend to be). Develop structural defenses including instruction-defense sandwiching (system instruction, user input boundary markers, repeated system instruction), random instruction embedding (system instructions placed at random positions within prompt), prompt format randomization (changes between requests to prevent automated attacks), few-shot examples demonstrating correct handling of adversarial inputs, and instruction phrasing in multiple languages or formats. Create output filtering including content safety classification (allowed vs disallowed output categories), regex pattern blocking for sensitive data (SSN, credit cards, API keys), PII detection and redaction, toxicity and harmfulness scoring (minimum 0.7 threshold), leaker detection (unexpected system prompt repetition), and output length limiting for over-response attacks. Add monitoring and alerting including logging all inputs and outputs for incident response, anomaly detection for unusual query patterns (token count spikes, frequency deviation), alert triggers for suspected injections (multiple attempts from same IP, successful injections detected), and red teaming program for defense testing. Implement architectural patterns including input-output separation (clear formatting distinguishing instructions from user data), privilege separation (different prompts for different operation types), human-in-the-loop for high-risk actions, embedding-based similarity for detecting malicious patterns, and LLM-as-judge for reviewing outputs before delivery. Include secure prompt engineering practices including avoiding "ignore previous instructions" type phrasing, never including credentials in prompts, using least privilege for tool access, avoiding prompt concatenation from multiple untrusted sources, and regular prompt audits for security vulnerabilities. Provide incident response plan including detection (alert triggers), containment (rate limit or block user), eradication (update detection rules), recovery (reset session), and post-mortem analysis.