Restarting nodes on kubernetes as the node status shows unknown

Question

The status of the node is coming as unknown right now

"conditions": [
          {
            "type": "Ready",
            "status": "Unknown",
            "lastHeartbeatTime": "2015-11-12T06:03:19Z",
            "lastTransitionTime": "2015-11-12T06:04:03Z",
            "reason": "Kubelet stopped posting node status."
          }

kubectl get nodes shows that the node is not ready. How do I fix this?

Answer 1

Considering that the node was previously working and recently stopped showing the ready status restart you kubelet service. Just ssh into the affected node and execute:

/etc/init.d/kubelet restart

Back on your master node run kubectl get nodes to check if the node is working now

For further details, refer to the Kubernetes Training.

Answer 2

Delete the node and then create a node object with the name of the one that died, bounce your schedulers & controllers, and then delete the node object. This seems to allow the scheduler to pick a new, good, node

Answer 3

You can try moving the cluster to the new node pool got rid of the weird stuck pods. But this is not the best solution, probably a temporary solution.

Answer 4

I had faced a similar issue. In my case I had 10 nodes already up and running. I had created two new nodes and the status was stuck at unknown state. After a lot of research I figured I didnt have memory to deploy any more nodes. So in my case the memory was the issue. Try checking if you have sufficient memory.

Answer 5

I had a similar issue, but mine was a very stupid silly mistake, I had made. On my node machine I had forgotten to unstall Docker engine. But what I still dont get it, is how did that node manage to join the cluster with having docker engine running on it?