AI-generated code leaks your secrets more often than you think

The model does not know what is real

When you paste a .env file into your chat window to give the model context, you have just handed it real credentials. It will use them in the code it writes. It does not label them as "replace this with your actual key". It will write const apiKey = "sk-prod-abc123xyz..." and move on, and that string will sit there in your source tree, invisible to you because you pasted it in and stopped thinking about it.

This happens constantly. It is not a bug in the model. The model is doing exactly what you asked: it is writing working code with working values. The problem is that "working" and "safe to commit" are not the same thing.

The four failure modes worth knowing

Hardcoded credentials in source. This is the obvious one, but it still catches people. You ask the model to write a Stripe integration, you give it your test key for context, and it uses that key in a constant. You ship it. The key is now in git history forever, even if you delete the file next week.

Secrets baked into log statements. The model is helpful with logging. Maybe too helpful. It will write code that logs full request objects, full response bodies, or full headers to help you debug. That is fine in dev. In prod, if your log output ends up in Datadog, Grafana, Sentry, or any SaaS aggregator, you have just exfiltrated your own secrets to a third party. The model did not make that judgment call. You did not either, because you were not thinking about it.

Environment variables echoed back in error messages. Similar problem. A model writing an error handler might reasonably generate something like:

if (!process.env.DATABASE_URL) {
  throw new Error(`DATABASE_URL is not set. Got: ${JSON.stringify(process.env)}`);
}

That JSON.stringify(process.env) dumps every environment variable you have into the error message. If that error surfaces in a user-facing response or an error reporting tool, your entire environment is now in a third-party system.

Committed .env files. This is less about the model and more about the workflow. When you are moving fast with AI assistance, you sometimes ask the model to scaffold a project. It might generate a .env.example and then also generate a .env with real-looking values. If you happen to have populated those values already, and you git add . without thinking, they go in. The model will not tell you not to commit .env. It generates files; it does not manage your git workflow.

What to actually do about it

The baseline is a .gitignore that covers secrets. Non-negotiable. Before you do anything else:

.env
.env.local
.env.*.local
*.pem
*.key

But .gitignore does not help with secrets that end up in source files rather than env files. For that, you want a pre-commit hook that scans for patterns. gitleaks is the standard tool here:

# Install once
brew install gitleaks

# Run against the current diff before committing
gitleaks protect --staged

You can add this as a pre-commit hook so it runs automatically. If you use a framework like husky or lefthook, it takes about two minutes to wire up.

For the logging problem, the fix is less mechanical. You have to read the log statements the model writes. Look for any code that logs request or response objects wholesale. Look for anything that references process.env outside of a startup config check. These are places where the model optimized for debuggability and skipped the "is this safe in prod?" filter.

The conversation context problem

Here is something people underestimate: the bigger your context window conversation, the more likely you are to have accidentally included real credentials at some point. You paste in a config file, a .env, a curl command with a real bearer token, to help the model understand your setup. It learns from that context and uses those values.

Get into the habit of scrubbing context before you paste it. Replace real values with YOUR_KEY_HERE or similar before sharing with the model. Yes, it is slightly annoying. It is less annoying than rotating all your API keys because a key showed up in a public GitHub repo.

Also: if you are using a git hosting provider with secret scanning turned on (GitHub has this by default for public repos, and it can be enabled for private ones), you will sometimes get an alert right after a push. Do not ignore those alerts. Rotate the key immediately, then deal with the code. In that order.

Checking existing code

If you have been vibe-coding for a while without thinking about this, it is worth running a scan over what you already have:

# Scan entire repo history
gitleaks detect --source .

# Or just the current working tree
gitleaks detect --no-git

You might find nothing. You might find something unpleasant. Either way, you want to know now rather than after someone finds it.

The model is not going to start flagging this on its own. It will write const secret = process.env.API_KEY ?? "sk-fallback-hardcoded-value" without any hesitation if the fallback makes the code work in its context. Reviewing for secrets is your job, and it is one of the few things that does not get faster with AI assistance.

Seaworthy catches this class of issue automatically as part of its security scan.