网页
综合
更新时间 2026-03-17 11:00:00
When Kubernetes restarts your pod — And when it doesn’t | CNCF
自动蜘蛛
www.cncf.io
正文约 5981 字
Enroll your company as a CNCF End User and save more than $10K in training and conference costs
Posted on March 17, 2026 by Shamsher Khan, Project Maintainer
CNCF projects highlighted in this post
A production internals guide verified against Kubernetes 1.35 GACompanion repository: github.com/opscart/k8s-pod-restart-mechanics
Engineers say “the pod restarted” when they mean four different things. Getting this wrong leads to flawed runbooks and bad on-call decisions.
The practical test: Did the pod UID change? If yes — that is recreation, not a container restart. Restart count resets to zero. If no — same pod object, container process restarted inside it.
kubelet watches the pod spec — not ConfigMaps, not Secrets, not Istio CRDs. If the pod spec didn’t change, kubelet never fires. This single fact explains the majority of “why didn’t my config update?” investigations in production.
Mutating admission webhooks can change the pod spec at creation time, but never after admission — they cannot trigger container restarts post-creation.
The flowchart below translates the same matrix into a decision path you can walk at 2am during an incident.
Diagram 1: Complete decision flowchart — does this change require a pod restart?
[Diagram 1: ConfigMap env var vs volume mount — env var pod frozen, volume pod auto-synced via kubelet symlink swap]
Env var mode (envFrom / valueFrom): The kernel copies env vars into /proc/<pid>/environ at execve(). That memory is owned by the process — no external system can modify it. Update the ConfigMap and kubelet sees no pod spec change, does nothing. The process keeps old values indefinitely.
Volume mount mode: kubelet syncs via an atomic symlink swap, not a file write:
The symlink swap generates IN_CREATE on ..data — NOT IN_MODIFY on the file. Applications watching IN_MODIFY on an open file descriptor miss this entirely. This is why nginx does not auto-reload on ConfigMap changes without explicit inotify handling.
These three scenarios look similar but are fundamentally different:
Diagram 3: Rolling update flow showing new ReplicaSet creation, pod recreation, and old RS retained for rollback.
Diagnostic rule: Climbing restart count with unchanged UID = crash loop. Zero restart count with new UID = rolling update.
StatefulSet note: StatefulSet pods are also recreated on image change, but ordinal identity (pod-0, pod-1) and PVC bindings are preserved. Container restart semantics are identical to Deployments — identity persistence does not imply in-place container restart.
K8s 1.35 made in-place pod resize generally available (kubernetes.io/blog/2025/12/19/kubernetes-v1-35-in-place-pod-resize-ga). Both CPU and memory can be resized without pod recreation — UID and IP never change. In-place resize availability depends on CRI and node OS support; verified on containerd 1.7+ with Linux cgroups v2.
What happens to the container depends on resizePolicy, which you define explicitly:
Important: The default resizePolicy for memory is NotRequired. If you omit it, memory resize silently updates the cgroup without restarting the container — and your JVM heap stays at the old size. Always define resizePolicy explicitly for memory.
Istio VirtualService and DestinationRule changes never trigger container restarts. Istiod maintains a persistent bidirectional gRPC stream to each Envoy sidecar — routing updates are pushed in milliseconds, in-memory swap, no pod touched, no file written.
When apps consume ConfigMaps via env vars, someone must run kubectl rollout restart after every update. Reloader automates this using Kubernetes watch events — detection is near-instant, not polling.
Production gotcha: The default Helm install uses watchGlobally=false — Reloader only watches its own namespace. Annotated Deployments in other namespaces are silently ignored, no error thrown. Always install with watchGlobally=true.
Hot-reload is not always safer than a container restart. Two failure modes worth knowing:
The file updates, the inotify handler fires, no error is thrown — but the new config has a logic error. The pod passes health checks and runs broken for hours. A container restart with a bad config fails immediately and loudly. Hot-reload with bad config fails quietly and late.
Mitigation: Validate config before swapping atomically.
Istiod pushes a RouteConfiguration referencing a cluster not yet propagated. Envoy rejects it and continues with old routing rules. No pod event fires. Mitigation: Monitor pilot_xds_push_errors and use istioctl proxy-status.
A container restart is disruptive but honest — failure modes are immediate and visible. Hot-reload optimizes for availability but shifts failures to be delayed and subtle. Both are valid strategies. The choice should be conscious.
The goal is not to automate restarts faster. It is to understand deeply enough that you trigger a container restart only when the process genuinely needs to die — and use every other mechanism when it does not.
Companion repository: github.com/opscart/k8s-pod-restart-mechanics
Author: Shamsher Khan — Senior DevOps Engineer, IEEE Senior Member. https://OpsCart.com
This article is adapted from an earlier version published on OpsCart.com. Republished here with permission.
Subscribe for updates, event info, webinars, and the latest community news
By submitting this form, I consent to receive marketing emails from the LF and its projects regarding their events, training, research, developments, and related announcements. I understand that I can unsubscribe at any time using the links in the footers of the emails I receive. Privacy Policy.
Copyright © 2026 The Linux Foundation®. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page. Linux is a registered trademark of Linus Torvalds. Privacy Policy and Terms of Use.
Cookie Settings Accessibility Statement Submit an issue with this page
本页为82搜索生成的网页快照,内容来自站内索引;演示数据的原文链接仅作示例。