OpenShift app configuration: environment, ConfigMaps, Secrets and persistent storage

Keep configuration out of the image. The same image should run in development and production, with differences supplied by the platform: plain settings in ConfigMaps, sensitive values in Secrets, and data that must survive restarts on persistent volumes. These are standard Kubernetes objects; oc set env and oc set volume make wiring them up quick.

Environment variables

oc set env deployment/web APP_MODE=production LOG_LEVEL=info
oc set env deployment/web --list
oc set env deployment/web LOG_LEVEL-     # remove a variable

Changing the pod template starts a new rollout, so pods restart with the new values.

ConfigMaps

oc create configmap web-config \
  --from-literal=FEATURE_X=true \
  --from-file=app.properties

# All keys as environment variables, optionally with a prefix
oc set env deployment/web --from=configmap/web-config --prefix=WEB_

# Or as files in a directory
oc set volume deployment/web --add --name=config \
  --type=configmap --configmap-name=web-config --mount-path=/etc/web

Values injected as environment variables are read once at container start. Files mounted from a ConfigMap are updated in running pods after a delay, but your application must reread them; the simple, predictable way is oc rollout restart deployment/web.

Secrets

oc create secret generic db-cred \
  --from-literal=username=appuser \
  --from-literal=password=CHANGE_ME

oc set env deployment/web --from=secret/db-cred --prefix=DB_

# Or mount as files, which keeps values out of the environment
oc set volume deployment/web --add --name=db-cred \
  --type=secret --secret-name=db-cred --mount-path=/var/run/secrets/db

Secret values are only base64-encoded in the API, so anyone allowed to read the secret can decode it. The default view role cannot read Secrets; edit and admin can. Avoid passing real passwords on the command line where shell history keeps them; --from-file or --from-env-file are safer. Change a single key later with oc set data secret/db-cred password=NEW_VALUE.

Persistent storage

Container file systems are thrown away when a pod is replaced. To keep data, a pod mounts a PersistentVolumeClaim (PVC), a request for storage that the cluster fulfills from a storage class. Ask what is available first:

oc get storageclass

The class marked (default) is used when a claim does not name one. Access modes depend on the storage behind the class:

• ReadWriteOnce (RWO): read-write by one node at a time. Typical for block storage and databases.
• ReadWriteOncePod (RWOP): read-write by a single pod.
• ReadWriteMany (RWX): read-write from many nodes, needed when several replicas share files. Only file storage such as NFS or CephFS offers it.
• ReadOnlyMany (ROX): read-only from many nodes.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: web-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

oc apply -f web-data-pvc.yaml
oc set volume deployment/web --add --name=data \
  --type=persistentVolumeClaim --claim-name=web-data --mount-path=/var/lib/web
oc get pvc web-data

oc set volume can also create the claim in one step with --claim-size=5Gi, using the default storage class. A claim stays Pending until a volume is provisioned; with classes that use WaitForFirstConsumer binding, that only happens once a pod uses the claim, so Pending right after creation is normal.

Things that bite with persistent volumes

• Permissions: pods run with a random user ID, so the mounted volume must be writable by that user. Most provisioners set group ownership from the pod’s fsGroup, which OpenShift assigns from the project’s range. If the app still gets permission denied, see the fix article on random UIDs.
• RWO and rolling updates: with a ReadWriteOnce volume, a new pod scheduled on another node cannot attach the volume while the old pod still holds it. Single-replica apps with RWO storage often use the Recreate deployment strategy, or a StatefulSet.
• Reclaim policy: dynamically provisioned volumes commonly use Delete, which removes the volume and its data when the claim is deleted. Check with oc get pv before deleting claims or projects you care about.

Official documentation

• Creating and using config maps
• Using volumes to persist container data
• Understanding persistent storage