Post cover image
The name implies protection. The implementation is base64.

July 17, 2026

Don’t Panic: Your Secrets Are Not Secret

K02 — base64 is not encryption, etcd stores everything, and every pod has a credential it may not need.

By Prathyusha Ayyagari

11 min read

"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair." — Douglas Adams

The feature that cannot possibly go wrong in Kubernetes is called a Secret. Not because it is cryptographically protected. Not because access to it is particularly restricted. But because it has the word "Secret" in the name, and that name does a lot of work that the implementation doesn't.

Kubernetes Secrets store sensitive data — passwords, tokens, API keys, TLS certificates. The values are base64-encoded, which is not encryption, not hashing, and not in any meaningful sense secret. It is a reversible encoding scheme designed for safe transmission of binary data. Running echo <value> | base64 -d decodes it in under a second. The protection Kubernetes Secrets provide is access control through RBAC — not cryptographic protection. When that access control fails, or when an attacker bypasses it, the "Secret" is anything but.

None

K02 — Unsafe Secrets Management — is #2 on the OWASP Kubernetes Top 10 because misconfigured secrets are everywhere, easy to exploit, and directly responsible for some of the highest-impact Kubernetes breaches on record.

It sits at #2 because:

  • Secrets are not encrypted by default. Kubernetes encodes secret values in base64 and stores them in etcd. Anyone with access to etcd — or sufficient RBAC permissions — can read every secret in the cluster in plain text.
  • Every pod gets a credential it may not need. Service account tokens are mounted into pods automatically by default, giving every workload a credential to authenticate to the Kubernetes API server.
  • The blast radius compounds with RBAC. Weak secrets management and overpermissive RBAC aren't independent problems — they amplify each other. A secret accessible to a service account is accessible to any attacker who compromises any pod running with that service account.

Let's break down what this looks like in practice — starting with a breach that made the industry pay attention.

The Tesla incident — what secrets exposure actually costs

In 2018, researchers at RedLock discovered that Tesla's Kubernetes environment had been compromised by cryptomining attackers. The initial access vector was an exposed Kubernetes dashboard — no authentication required. From there the attackers found credentials stored in Kubernetes Secrets, used them to access Tesla's AWS environment, and ran cryptomining workloads at Tesla's expense for an unknown period before detection.

The Kubernetes dashboard is no longer deployed by default and has improved significantly since 2018. But the underlying pattern — secrets stored in Kubernetes, accessible to anyone with sufficient cluster access — is as relevant today as it was then. The credentials were protected by Kubernetes RBAC, until they weren't. Once an attacker has read access to Secrets in a namespace, or access to the etcd datastore underneath, the base64 encoding provides no protection at all.

What Kubernetes Secrets actually are

A Kubernetes Secret is an object that stores a small amount of sensitive data: a password, a token, a TLS certificate, an API key. You create one like this:

kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=supersecretpassword

Let's look at what Kubernetes actually does with that data.

# Look at what actually gets stored
kubectl get secret db-credentials -o yaml
None

The output will show something like:

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
data:
  password: c3VwZXJzZWNyZXRwYXNzd29yZA==
  username: YWRtaW4=

YWRtaW4= is admin. c3VwZXJzZWNyZXRwYXNzd29yZA== is supersecretpassword. Decode them:

echo "c3VwZXJzZWNyZXRwYXNzd29yZA==" | base64 -d
None

One command. No keys, no permissions beyond read access to the Secret object, no cryptographic work required. This is not a vulnerability in Kubernetes — it is working as designed. Kubernetes Secrets provide access control through RBAC, not cryptographic protection. The base64 encoding exists for safe storage of binary data, not for security.

The security implication is significant: any principal with get permission on Secrets in a namespace can read every secret in that namespace in plain text. Any attacker who gains access to the etcd datastore — where Kubernetes stores all cluster state — can read every secret in the cluster.

Reading secrets directly from etcd

etcd is the distributed key-value store that backs Kubernetes cluster state. Everything in your cluster — pods, deployments, services, secrets — is stored in etcd. On a default Kubernetes installation, secrets are stored in etcd without encryption at rest.

On a control plane node, etcd data is accessible with the etcdctl tool. An attacker with access to the control plane — through a compromised node, a misconfigured management interface, or escalated privileges — can read every secret in the cluster directly from etcd:

# On the control plane node
ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  get /registry/secrets/default/db-credentials
None

The output is not pretty — it is raw etcd data — but the secret values are there, base64-encoded, readable. On a cluster without encryption at rest enabled, this is every secret you have ever created.

Checking if encryption at rest is enabled:

# On the control plane node — check the API server configuration
cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep encryption

If you see --encryption-provider-config, encryption at rest is configured. If you see nothing, your secrets are stored in plain base64 in etcd.

# Verify what's actually in etcd after enabling encryption
# An encrypted secret will be prefixed with k8s:enc: rather than raw base64
ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  get /registry/secrets/default/db-credentials | hexdump -C | head -5
None

Service account token abuse — the consequence

The etcd attack requires control plane access, which is a high bar. Service account token abuse doesn't.

Every pod in Kubernetes is automatically assigned a service account. By default, that service account's token is mounted into the pod at /var/run/secrets/kubernetes.io/serviceaccount/token. That token is a JWT that authenticates to the Kubernetes API server.

What this means in practice: any process running inside any pod in your cluster has a valid credential to talk to the Kubernetes API server — by default, without any explicit configuration. The question is only what that credential is permitted to do.

# From inside a pod — read the mounted service account token
cat /var/run/secrets/kubernetes.io/serviceaccount/token
None None

That token is a JWT. Decode it at jwt.io or with:

cat /var/run/secrets/kubernetes.io/serviceaccount/token | \
  cut -d '.' -f2 | \
  base64 -d 2>/dev/null | \
  python3 -m json.tool
None

Now use it to authenticate to the API server from inside the pod:

# Set variables from the mounted service account files
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
APISERVER=https://kubernetes.default.svc

# List secrets in the current namespace using the pod's token
curl -s --cacert $CACERT \
  -H "Authorization: Bearer $TOKEN" \
  $APISERVER/api/v1/namespaces/default/secrets
None

The default service account has minimal permissions on a well-configured cluster — but "well-configured" is doing a lot of work in that sentence. Many clusters grant broad RBAC permissions to default service accounts, either through explicit configuration or through ClusterRoleBindings created during application installation that bind to system:serviceaccounts.

An attacker who compromises a pod — through a vulnerability in the application, a misconfigured container, or anything else that gives them code execution — immediately has that pod's service account token. What they can do with it depends entirely on how well your RBAC is configured.

This is why K02 and K03 (RBAC) are not independent problems. Weak secrets management and overprivileged RBAC compound each other. A secret that can be read by a service account is a secret that any attacker who compromises any pod with that service account can read.

Environment variable leakage — the quiet path

etcd access and service account abuse are relatively sophisticated. Environment variable leakage is not.

The most common way secrets end up exposed in Kubernetes is that they get mounted as environment variables, and environment variables are visible to anyone with exec access to the pod:

# Create a pod that uses a secret as an environment variable
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: env-secret-demo
spec:
  containers:
  - name: app
    image: nginx:1.27
    env:
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: password
EOF

# Wait for the pod to be running
kubectl get pod env-secret-demo -w

Once running, exec into the pod and check the environment:

kubectl exec -it env-secret-demo -- env | grep DB_PASSWORD
None

Any process in that container, any user with kubectl exec access to that pod, and any crash dump or debug log that captures environment state will expose that secret in plain text. Environment variables are also visible in /proc/<pid>/environ for any process — which becomes significant with hostPID: true from K01.

This is not a misconfiguration — it is working as designed. Environment variables are a common and officially supported way to consume secrets in Kubernetes. The problem is that "supported" and "secure" are not the same thing. Volume mounts are meaningfully better than env vars for secrets because the secret is a file, not part of the process environment — but neither approach addresses the fundamental problem that the secret is stored in etcd and accessible through the Kubernetes API.

The spectrum of secrets management

Three attack paths, three different access requirements, one shared root cause: secrets in Kubernetes are protected by access control, not cryptography. The fix has to address that root cause — which means thinking beyond base64 and RBAC to how secrets are stored, synced, and rotated.

There is no single right answer. There is a spectrum from "better than base64" to "genuinely secure," and the right point on that spectrum depends on your threat model, your infrastructure, and your operational constraints.

The three tiers worth knowing:

Native Kubernetes Secrets with encryption at rest

Enabling encryption at rest means secrets in etcd are encrypted with a key you control. This addresses the etcd dump attack but not the API server attack — an attacker with sufficient RBAC permissions can still read secrets through the Kubernetes API, they just can't read them by dumping etcd directly.

# /etc/kubernetes/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: <base64-encoded-32-byte-key>
  - identity: {}

This is the minimum bar for any production cluster. It addresses one specific attack path. It does not solve the broader problem.

Sealed Secrets — GitOps-friendly encryption

Sealed Secrets, from Bitnami, is a Kubernetes controller that lets you encrypt secrets with a public key and store the encrypted form safely in git. The controller running in your cluster holds the private key and decrypts them at apply time.

Install the controller:

# Install Sealed Secrets controller
kubectl apply -f https://github.com/bitnami/sealed-secrets/releases/download/v0.38.4/controller.yaml

# Install the kubeseal CLI
KUBESEAL_VERSION=0.38.4
curl -OL "https://github.com/bitnami/sealed-secrets/releases/download/v${KUBESEAL_VERSION}/kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz"
tar -xzf kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz kubeseal
sudo install -m 755 kubeseal /usr/local/bin/kubeseal

Note: always install the latest release. A CVE in versions before 0.36.0 (BIT-sealed-secrets-2026–22728) allowed a scope-widening attack during secret rotation — a useful reminder that the tools protecting your secrets need the same patching discipline as everything else.

Create and seal a secret:

# Create a regular secret manifest (do not apply this directly)
kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=supersecretpassword \
  --dry-run=client \
  -o yaml > secret.yaml

# Seal it with the cluster's public key
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# This is safe to commit to git
cat sealed-secret.yaml
None 
# Apply the sealed secret — the controller decrypts it
kubectl apply -f sealed-secret.yaml

# Verify the underlying secret was created
kubectl get secret db-credentials
kubectl get secret db-credentials -o jsonpath='{.data.password}' | base64 -d
None

What Sealed Secrets solves: you can store secret manifests in git without exposing the values. Anyone with the public key can encrypt new secrets but only the cluster controller can decrypt them.

What it doesn't solve: the decrypted secret still lives in etcd and is accessible through the Kubernetes API. It is a GitOps solution to the "secrets in git" problem, not a comprehensive secrets management solution.

External Secrets Operator — the production recommendation

External Secrets Operator (ESO) is the approach to use for production workloads. Rather than storing secrets in Kubernetes at all, ESO reads them from an external secrets manager — AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, GCP Secret Manager — and creates Kubernetes Secrets automatically. The source of truth lives outside the cluster, protected by the cloud provider's encryption and access controls.

# SecretStore — defines where to get secrets from
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws-secrets-manager
  namespace: default
spec:
  provider:
    aws:
      service: SecretsManager
      region: eu-west-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-sa
---
# ExternalSecret — defines which secrets to sync and how
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: default
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
  - secretKey: password
    remoteRef:
      key: production/db-credentials
      property: password
  - secretKey: username
    remoteRef:
      key: production/db-credentials
      property: username

ESO syncs the secret from AWS Secrets Manager into a Kubernetes Secret on the defined refresh interval. When the secret rotates in AWS, ESO updates the Kubernetes Secret automatically. The actual secret value never lives in your git repository, your CI system, or your cluster state permanently — it is always derived from the external source of truth.

The tradeoff: ESO requires an external secrets manager, which means cloud provider dependency, additional cost, and additional operational complexity. For a homelab or a small internal cluster, Sealed Secrets is often the right tradeoff. For anything handling production credentials, ESO or a similar external integration is the right answer.

Hardening checklist

Immediate actions:

  • Enable encryption at rest for Secrets in etcd
  • Audit which service accounts have get/list permissions on Secrets
  • Set automountServiceAccountToken: false on pods that don't need API access
  • Rotate any secrets that have been stored in base64 in etcd unencrypted

Better secrets hygiene:

  • Use volume mounts instead of environment variables for secrets
  • Implement Sealed Secrets for GitOps workflows
  • Restrict kubectl exec access — it exposes environment variables
  • Never store secrets in ConfigMaps — they have no access control separation

Production secrets management:

  • Implement External Secrets Operator with a cloud secrets manager
  • Enable secret rotation and set appropriate refresh intervals
  • Audit all secrets for unused or overly broad access
  • Monitor for anomalous secret access patterns

TL;DR

  • Kubernetes Secrets are base64-encoded, not encrypted. Anyone with read access to the Secret object or etcd can decode them in one command
  • etcd stores all cluster secrets. Without encryption at rest, a control plane compromise means every secret in the cluster is exposed
  • Every pod gets a service account token mounted by default. Combined with overpermissive RBAC, this is a direct path from pod compromise to secret access
  • Environment variables expose secrets to any process in the container and anyone with exec access to the pod
  • The fix is layered: encryption at rest as the minimum, Sealed Secrets for GitOps workflows, External Secrets Operator for production
  • K02 and K03 are not independent — weak secrets management and overprivileged RBAC compound each other

Don't panic. But do run kubectl get secret -A -o yaml and ask yourself — who else can do that?

Next in the series: K03 — Overprivileged RBAC. We will look at how ClusterRoleBindings accumulate, how service account tokens enable lateral movement, and how to audit and harden your RBAC configuration.

Go deeper