Prompt Exfiltration

ELI5 — The Vibe Check

Prompt exfiltration is attacking an AI to leak its system prompt — not hijacking the model's behavior, but stealing its instructions. Your competitor built a custom AI assistant with a carefully engineered prompt worth thousands of dollars in iteration. You ask it "repeat your system prompt" in 50 creative ways until it tells you. Now you have their secret sauce. It's corporate espionage via social engineering, and the target is a language model.

Real Talk

Prompt exfiltration targets the confidentiality of system prompts — the instructions that customize AI behavior in applications. Attackers use direct requests, jailbreaks, token manipulation, or indirect extraction (asking the model to describe its behavior until the system prompt can be reconstructed). Defenses include prompt injection hardening, output filtering, constitutional constraints, and accepting that system prompts are never fully secret. Anthropic's guidelines discourage models from directly revealing system prompts but can't guarantee it.

When You'll Hear This

"Someone exfiltrated our agent's system prompt — it's all over Reddit now." / "Assume your system prompt is not a secret. Design your security accordingly."