The AI Agent That Leaked Your API Key — And the One Guardrail

I watched an AI agent dump a production API key into its own output stream last week. Not a simulation. Not a red-team exercise. Real. The agent called a shell tool, ran `cat .env`, and the key appeared in the chat log.

Terminal output showing an AI agent printing a raw `OPENAI_API_KEY` environment variable to stdout
Terminal output showing an AI agent printing a raw `OPENAI_API_KEY` environment variable to stdout

If you’re building agents that call tools, you’re one `print()` away from a breach.

Here’s the exact pattern.

The Tool That Betrays You

Most agent frameworks give the model a shell, a file reader, or a curl wrapper. The model decides what to run. It sees `env` variables as “just another file” — same as a config.

The bug: no output sanitizer between the tool’s stdout and the user’s screen.

I saw a demo agent fetch a `.env` file, print it, and the log server stored it plaintext. That key hit the log aggregator, then the SIEM, then the incident-response team’s inbox.

> The model doesn’t know what’s sensitive. The tool doesn’t check. The log doesn’t redact.
> Three layers of trust, zero layers of filtering.

The fix is one guardrail: an output filter that strips known secret patterns before the tool returns.

The Guardrail That Would’ve Caught It

Add a `sanitize_output()` function that runs after every tool call. Match against:

- `sk-...` (OpenAI keys)
- `AKIA...` (AWS keys)
- `-----BEGIN` (private keys)
- Any `=`-delimited line that looks like a key-value pair

If the output matches, block it. Return a placeholder: `[redacted: secret pattern]`.

I could be wrong here, but I think most teams skip this because they assume the model “knows” not to print secrets. It doesn’t. It’s a language model, not a security guard.

Flowchart: Agent → Tool Call → Output → Log → Attacker — with a red “Output Filter” gate
Flowchart: Agent → Tool Call → Output → Log → Attacker — with a red “Output Filter” gate

The same pattern applies to file-reader tools, database-query tools, and any tool that returns raw data.

Why This Matters for Bug Hunters

If you’re hunting AI-agent bugs, this is your low-hanging fruit.

- Check if the agent prints raw environment variables
- Check if the agent’s log endpoint stores tool outputs without redaction
- Check if the agent’s prompt says “don’t share secrets” but has no runtime enforcement

One report: “Agent prints API key in tool output” → critical severity.

The Three Layers That Failed

Code diff: adding a `sanitize_output()` regex that blocks `sk-...` patterns before they reach the log
Code diff: adding a `sanitize_output()` regex that blocks `sk-...` patterns before they reach the log

1. The model — no built-in secret awareness
2. The tool — no output filter
3. The log — no redaction

Fix any one of these, and the key stays safe.

What You Can Do Today

- Add a regex-based output filter to every tool call
- Log the filter’s action (not the secret)
- Test with a dummy key: `sk-test-123`

If you’re building agents, this is a five-minute change that saves a five-figure incident.

If you need more context on how these flaws slip through, I wrote about a similar case in The $25M MEV Bot Lesson: Why Your Own Code Can Drain You — worth a read before your next audit.

Related: I found an IDOR on Noones in 15 minutes — how the old REST API leaked owner IDs, and why hunters miss it.

🔎 Found this useful? Support the hunt: EVM 0x0b3bad2b6eC27279f0BcFe6314cF1058d88d136D  ·  SOL 6Ph1ALrdtkkwWbAcSaiVjmHftPjLCA1j7RqznbfTbgyF  ·  or share this post — that helps too.

Disclosure: some links here are affiliate links. At no additional cost to you, I may earn a commission if you buy through them. I only recommend things I have actually used.

Comments

Popular Posts