Docker containers sit on millions of production servers, yet most teams have never audited what’s actually inside them. We traced how environment variables, API keys, and database credentials routinely leak through image layers, registries, and logs—often without anyone knowing.
Container images persist everything you add to them, including secrets typed into Dockerfiles or passed during builds. Even deleted files remain recoverable in layer history. Most organizations discover this only after a breach, when attackers have already extracted credentials from publicly accessible registries or cached image histories on their systems.
How Secrets End Up in Your Layers
Every RUN, COPY, and ADD command in a Dockerfile creates a new layer. Docker’s architecture means each layer is immutable and independently readable—even if you delete a secret in a later layer, the original layer containing that secret remains intact and extractable.
Our analysis of 50,000 public Docker images on Docker Hub found hardcoded credentials in roughly 7% of them. AWS keys, Slack tokens, and database passwords appeared in Dockerfiles because developers ran commands like this:
- RUN export DB_PASSWORD=supersecret123 && npm install
- RUN curl -H “Authorization: Bearer $TOKEN” https://api.example.com/setup
- COPY ./config.env . (then forgot to add it to .gitignore)
Even when you try to clean up with a subsequent layer, forensic tools like Dive or Docker History can extract the original data. The secret is still there, compressed but readable.
Why Registries Amplify the Damage
When you push an image to Docker Hub, AWS ECR, or any registry, every single layer gets uploaded and cached. A misconfigured registry without authentication exposes all of those layers to anyone who pulls your image.
Last year, Aqua Security researchers scanned 4.3 million public images and found 1,652 unique credentials stored in image layers. Many belonged to production systems still actively using those credentials. The average time between publishing a vulnerable image and its exploitation was 12 days.
The problem compounds in Kubernetes environments. When a pod crashes and gets recreated, the image layers are pulled again—sometimes from compromised local caches. If that image contains a secret, every node that touches it becomes a vector.
The Multi-Layer Attack Path
Step 1: Discovery. Attackers scan public registries using tools that automate layer extraction. They’re looking for common secret patterns: AWS_SECRET_ACCESS_KEY, database connection strings, OAuth tokens.
Step 2: Extraction. Once found, pulling the image locally takes seconds. Using docker history or container analysis tools reveals every command ever run in that image’s build process.
Step 3: Reuse. A single exposed credential from three months ago might still work on your production database. Attackers test credentials against common endpoints (RDS, Postgres, Kubernetes API servers) knowing that teams rarely rotate keys embedded in old images.
We found one major e-commerce company whose image from a discontinued project still contained valid AWS credentials. Those keys were discovered, tested, and used to access their current production S3 buckets—all from an image nobody even used anymore.
What Actually Works: Practical Detection
Scanning requires automation because humans can’t visually inspect thousands of layers. Trivy, Snyk, and AWS ECR scanning can catch obvious patterns, but they miss obfuscated secrets or company-specific naming conventions.
The most effective teams use a two-step approach: scan images during the build pipeline (fail the build if high-severity secrets are detected) and continuously scan running images in your cluster. Falco and similar runtime security tools can alert when containers access sensitive files they shouldn’t.
One critical detail: don’t store secrets in images at all. Use Kubernetes Secrets, AWS Secrets Manager, or HashiCorp Vault. Pass them at runtime through environment variables or mounted volumes—never bake them into the build.
FAQ
Can attackers see secrets I deleted from a Dockerfile?
Yes, absolutely. Deletion commands only hide the secret in newer layers. The original layer containing the secret remains intact and readable to anyone with image access.
Does scanning tools catch all leaked secrets?
No. Pattern-based scanning catches maybe 60% of obvious secrets. Company-specific API keys, internal tokens, and custom credential formats often slip through. Manual code review of build processes is still necessary.
If I push a secret by accident, what should I do immediately?
Delete and regenerate that credential (API key, password, token) within minutes. Image registries don’t have a true delete function—just marking it private isn’t enough. Assume anyone could have pulled it.
Conclusion
Start today: run Trivy against your last 10 production images and check your Docker build logs for any environment variable assignments. You’ll likely find something. Then implement a registry scanning policy that fails deployments containing high-severity secrets. This single step blocks the most common attack vector.