Kubernetes ingress kepps returning 502 server error

0 votes

I am trying to setup an Ingress in GCE Kubernetes. But when I visit the IP address and path combination defined in the Ingress, I keep getting the following 502 error:

Error:Server error
The server encountered a temporary error that could not complete your request.
Please try again in 30 seconds

Here is what I get when I run: kubectl describe ing --namespace dpl-staging

Name:           dpl-identity
Namespace:      dpl-staging
Address:        35.186.221.153
Default backend:    default-http-backend:80 (10.0.8.5:8080)
TLS:
  dpl-identity terminates
Rules:
  Host  Path    Backends
  ----  ----    --------
  *
        /api/identity/*     dpl-identity:4000 (<none>)
Annotations:
  https-forwarding-rule:    k8s-fws-dpl-staging-dpl-identity--5fc40252fadea594
  https-target-proxy:       k8s-tps-dpl-staging-dpl-identity--5fc40252fadea594
  url-map:          k8s-um-dpl-staging-dpl-identity--5fc40252fadea594
  backends:         {"k8s-be-31962--5fc40252fadea594":"HEALTHY","k8s-be-32396--5fc40252fadea594":"UNHEALTHY"}
Events:
  FirstSeen LastSeen    Count   From                SubObjectPath   Type        Reason  Message
  --------- --------    -----   ----                -------------   --------    ------  -------
  15m       15m     1   {loadbalancer-controller }          Normal      ADD dpl-staging/dpl-identity
  15m       15m     1   {loadbalancer-controller }          Normal      CREATE  ip: 35.186.221.153
  15m       6m      4   {loadbalancer-controller }          Normal      Service no user specified default backend, using system default

Here is my service description: kubectl describe svc --namespace dpl-staging

Name:           dpl-identity
Namespace:      dpl-staging
Labels:         app=dpl-identity
Selector:       app=dpl-identity
Type:           NodePort
IP:             10.3.254.194
Port:           http    4000/TCP
NodePort:       http    32396/TCP
Endpoints:      10.0.2.29:8000,10.0.2.30:8000
Session Affinity:   None
No events.

Also, here is the result of executing: kubectl describe ep -n dpl-staging dpl-identity

Name:       dpl-identity
Namespace:  dpl-staging
Labels:     app=dpl-identity
Subsets:
  Addresses:        10.0.2.29,10.0.2.30
  NotReadyAddresses:    <none>
  Ports:
    Name    Port    Protocol
    ----    ----    --------
    http    8000    TCP

No events.

Here is my deployment.yaml:

apiVersion: v1
kind: Secret
metadata:
  namespace: dpl-staging
  name: dpl-identity
type: Opaque
data:
  tls.key: <base64 key>
  tls.crt: <base64 crt>
---
apiVersion: v1
kind: Service
metadata:
  namespace: dpl-staging
  name: dpl-identity
  labels:
    app: dpl-identity
spec:
  type: NodePort
  ports:
    - port: 4000
      targetPort: 8000
      protocol: TCP
      name: http
  selector:
    app: dpl-identity
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  namespace: dpl-staging
  name: dpl-identity
  labels:
    app: dpl-identity
  annotations:
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - secretName: dpl-identity
  rules:
  - http:
      paths:
        - path: /api/identity/*
          backend:
            serviceName: dpl-identity
            servicePort: 4000
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: dpl-staging
  name: dpl-identity
kind: Ingress
metadata:
  namespace: dpl-staging
  name: dpl-identity
  labels:
    app: dpl-identity
  annotations:
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - secretName: dpl-identity
  rules:
  - http:
      paths:
        - path: /api/identity/*
          backend:
            serviceName: dpl-identity
            servicePort: 4000
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: dpl-staging
  name: dpl-identity
  labels:
    app: dpl-identity
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: dpl-identity
    spec:
      containers:
      - image: gcr.io/munpat-container-engine/dpl/identity:0.4.9
        name: dpl-identity
        ports:
        - containerPort: 8000
          name: http
        volumeMounts:
        - name: dpl-identity
          mountPath: /data
      volumes:
      - name: dpl-identity
        secret:
          secretName: dpl-identity
Sep 10, 2018 in Kubernetes by lina
• 8,220 points
12,770 views

5 answers to this question.

0 votes

Your backend k8s-be-32396--5fc40252fadea594 is showing unhealthy. Thtas exactly when you get the 502 server error.  Ingress doesn not forward the traffic if the backend is unhealthy. Its failing some health checks. 

Check the health check setting and check if all the parameters are suitable for your pod.

You can get the health check setting here: Compute engine -> Health check.

If all the parameters seem fine then there might be something about the traffic that is sent from browser to the container. You can check that using the following command:

kubectl exec -it PODID -- bash

and then curl localhost to check if the container is responding as per expected.

Try all of these and let us know.

If you are interested in knowing more about Kubernetes, then consider taking  CKA Certification Course.

answered Sep 10, 2018 by Kalgi
• 52,350 points
0 votes

Your endpoint does not return Success and that's the issue here. Health checks are failing. HTTP/HTTPS load balancer will not send the request to the cluster node if these health checks keep failing. 

I created an endpoint that always returns 200 OK. Once the health checks start passing, your load balancer also would start without any errors.

answered May 6, 2019 by Girish
0 votes
I totally agree with @Kalgi's and @Girish's answers. I would like to add something to it. If you do not want to make changes to your current container, you can also create another container and mention the endpoint there. This way you'll have your initial workaround safeguarded.
answered May 6, 2019 by Monish
0 votes
I had a similar error, wasted a lot of time behind it. In my case, the problem was that I did not have a web server running on my instance to handle the network requests. Nginx solved the problem!
answered May 6, 2019 by Nagya
0 votes

I have added "/" endpoints in each service with 200 response and it did the trick.

answered May 6, 2019 by Oishi

Related Questions In Kubernetes

0 votes
2 answers
0 votes
1 answer

Error while creating kubernetes dashboard

The installation fails because there is no ...READ MORE

answered Aug 27, 2018 in Kubernetes by Kalgi
• 52,350 points
1,824 views
0 votes
1 answer

Kubernetes error syncing pod

The issue caused by the docker container ...READ MORE

answered Aug 27, 2018 in Kubernetes by Kalgi
• 52,350 points
3,747 views
+1 vote
1 answer
0 votes
3 answers

Error while joining cluster with node

Hi Kalgi after following above steps it ...READ MORE

answered Jan 17, 2019 in Others by anonymous
15,401 views
+4 votes
1 answer

Installing Web UI (Dashboard):kubernetes-dashboard on main Ubuntu 16.04.6 LTS (Xenial Xerus) server

Follow these steps: $ kubeadm reset $ kubeadm init ...READ MORE

answered Apr 12, 2019 in Kubernetes by Kalgi
• 52,350 points

reshown Apr 12, 2019 by Kalgi 7,020 views
0 votes
1 answer
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP