What happens when a pod exceeds its memory limit

Question

When a pod exceeds its memory limit? What happens?

Answer 1

• Whenever a pod exceeds its memory limit a signal SIGKILL is sent which  immediately terminates the container and spawns a new one with OOM(out of memory) error. 

• The OS, if using a cgroup based containerisation (docker, rkt, etc), will do the OOM killing. 

• Kubernetes simply sets the cgroup limits but is not ultimately responsible for killing the processes.

• 'SIGTERM ' is sent to PID 1 and k8s waits for (default of 30 seconds) `terminationGracePeriodSeconds` before sending the `SIGKILL` or you can change that time with terminationGracePeriodSeconds in the pod.

• As long as your container will eventually exit, it should be fine to have a long grace period. If you want a graceful restart it would have to do it inside the pod. 

• If you don't want it killed, then you shouldn't set a memory `limit` on the pod and there's not a way to disable it for the whole node. 

• Also, when the liveness probe fails, the container will SIGTERM and SIGKILL after some grace period.