The password is in the transcript: where secrets handed to a security agent end up

eHVB-IT · · AI security · LLM agents · secrets management · penetration testing · SOC

A test credential passed to an LLM agent is copied into the session transcript and into every delegation. The path we observed during a penetration test, the cleanup that checked the wrong place, and the controls that hold up.

Lire en français

Summary

An authenticated penetration test starts with a credential: a test account created for the engagement, with a password the client hands to the tester. When the tester is an LLM agent, the question of where that password goes changes shape. During an authenticated engagement in October 2026, the engagement folder was clean: the password had never been written there, and the test session token had been deleted. The password was still on disk in plain text, twice, in a place nobody had checked: the session transcript. This note traces the path it took, explains why that path follows from how an agent works, and describes the controls we now apply.

Where the secret went

The engagement was launched from a command that delegates offensive work to a specialised subagent. The test account's password was passed as an argument to that command. We later found it in two places.

  1. The session transcript. Our tooling records each session as a JSONL file under ~/.claude/projects/<project>/<session>.jsonl. That file holds every message, so it holds the command argument, in plain text.
  2. The subagent prompt. To delegate, the main session writes a brief that the subagent receives as its instruction. The password was copied into it, since the subagent needed it to sign in. That brief is recorded too.

The user typed the secret once. The tooling wrote it several times, into files that are not part of the engagement and outlive its closure.

Why this is not a one-off bug

An agent works on its context, and its context is text. Whatever enters it (an argument, a brief, the output of a command) is kept so the agent can refer back to it, and recorded so the session can be resumed. Delegating to a subagent means serialising part of that context into a new message. Each delegation step is one more copy.

The same mechanism applies to tool output. If the agent prints an HTTP response carrying a session cookie, or the contents of a configuration file, that text enters the context and therefore the transcript. We did not measure this on this engagement; it follows directly from the mechanism above, and we treat it accordingly.

The rule that follows: the exposure surface of a secret handed to an agent is every place the tooling copied it, not the place where it was typed.

The cleanup that checked the wrong place

At the end of the engagement the subagent checked its own traces: a search for the password across the whole engagement folder returned no match, and the file holding the test session token was deleted with rm -P.

Both checks were sincere, and neither looked at the right thing. The search covered the engagement folder, while the secret sat in the transcript, outside it. As for rm -P, meant to overwrite a file before deleting it, the macOS manual page states that the flag has no effect and is kept only for compatibility. The file was deleted the ordinary way, with no overwrite.

On an SSD, overwriting a file does not guarantee the data is gone anyway. The control that matters for a session token is on the server: the token is invalidated, or it expires. Deleting the local copy changes nothing for anyone who already read it.

The second blind spot: what the agent cannot see

The same engagement exposed a limit of a different kind. The offensive subagent worked with HTTP requests and no browser. It could analyse the JavaScript files the application served, but not what the running application stores in the browser: localStorage, sessionStorage, IndexedDB, global variables, console messages. A token left in browser storage would have escaped it.

We closed that gap with a separate pass from the main session, using a driven browser: sign in through the form, read the storage, read the console. The result was clean. What matters is that the subagent's report did not state this limit up front. A report that does not say what its tools could not see implies the question was covered.

The engagement's first report, unauthenticated, carried an honest label: "not a full penetration test". That is the format we expect from any coverage statement: say what was tested, and name what was not.

Controls

Never let the value pass through the context. The password must not appear in a command argument or a delegation brief. Pass a reference instead: a path to a file readable only during the session, or an environment variable. The reference only helps if the agent uses it without ever printing its contents. A command like curl -u "test:$(cat /path/secret)" leaves the reference in the transcript; a bare cat /path/secret writes the value into it.

A disposable account, revoked at closure. The test account is created for the engagement, with only the rights it needs, and disabled or rotated at the end. If a copy of the secret survived somewhere, it no longer opens anything.

Search the transcripts for the secret after the engagement. Run the search from your own terminal, not through the agent: asking the agent to look for the secret means giving it the secret, and writing it down once more. In a bash shell, without the secret appearing in history or on the command line:

read -rs SECRET
[ -n "$SECRET" ] && grep -rlF -f <(printf '%s\n' "$SECRET") ~/.claude/projects/
unset SECRET

The command lists the files that contain the secret. If it finds any, rotate the secret first, then delete or redact those files. The order matters: a deleted file may already have been backed up or synced.

Revoke on the server, not only locally. Session tokens issued during testing are invalidated by a sign-out or a revocation, and closure checks that this happened.

A coverage statement that names its blind spots. Each report says which tools produced it and what those tools could not observe. A browser pass is planned whenever an application keeps data on the client.

Limits

  • We describe our own tooling. We have not checked how other agent environments record their sessions; the general mechanism applies whenever an agent keeps its context, but the locations differ.
  • The transcript is stored as JSON. A secret containing a quote, a backslash or some non-ASCII characters is escaped there, and a literal search will miss it. Search for the escaped form as well.
  • The search only covers local files. It says nothing about a backup already taken, a synced folder, or what the model provider retains under its own terms.
  • The browser pass covers storage and console for the session it observed. It does not prove the absence of leaks on paths it did not follow.

Checklist

  1. Never pass a secret as a command argument or in a delegation brief; pass a reference.
  2. Check that the agent uses the reference without printing its contents.
  3. Use a dedicated test account with minimal rights, revoked at closure.
  4. Search the transcripts for the secret from your own terminal, never through the agent.
  5. If you find it, rotate first, clean up second.
  6. Invalidate session tokens on the server; do not rely on deleting a file.
  7. Require every report to name what its tools could not see.

eHVB-IT is a French cybersecurity consultancy working on SOC triage, application security and detection engineering. Contact: contact@ehvb-it.fr

← All research notes