Kubernetes API: Client-side Load Balancer

Alex Egg,

Server-Side Load Balancing

The intuitive way to handle load balancing in Kubernetes is to leverage your infrastructure and setup an Ingress that pipes to a backend Service.

Diagram

For example, the Ingress below terminates SSL traffic on port 80 and then dispatches to the Service pool blog-frontend which may contain any number of Pods listing on port 3000.

ingress.yml

# this is the top level ingress point for the cluster DNS points *.eggie5.com here
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: eggie5-blog
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "com-eggie5-production-ingress"
spec:
  tls:
  - secretName: tls
  backend:
    serviceName: blog-frontend
    servicePort: 80

frontend-service.yml

apiVersion: v1
kind: Service
metadata:
  name: blog-frontend
  labels:
    name: web
spec:
  type: NodePort
  ports:
    # The port that this service should serve on.
    - port: 80
      targetPort: 3000
      protocol: TCP
  # Label keys and values that must match in order to receive traffic for this service.
  selector:
    name: web

However you can move this work from Kubernetes/cloud-provider to the client-side, or in other words, setup the load-balancer yourself. The difficult issue here isn’t setting up the load balancer, we can easily configure nginx for that purpose. The real difficulty, being in a distributed environment, is discovery: how do we know what hosts are in our cluster and how to address them. Discovery something that Kubernetes does best. Maybe we can leverage Kubernetes to build our load balancer.

Client-side Load Balancing

We can leverage Kubernetes API in our load balancer to find what hosts are available and where they are located.

Kelsey Hightower recently published a simple example of how to do this in Kubernetes:

He builds a package (in go) called endpoints.go , which encapsulates the functionality of round-robin endpoint selection and synchronizes the list of endpoints w/ Kubernetes API.

Your implementation would then use the endpoints package with namespace and service arguments to get the next available endpoint within that set of endpoints (per round-robin) and then forward the request respectively. This would be packaged up as a Docker container and then run as a Pod in Kubernetes.

Here’s an example implementation from Kelsey’s documentation:

Load balancers are created using a Config:

config := &endpoints.Config{
    Namespace: namespace,
    Service:   service,
}
lb := endpoints.New(config)

Before an endpoints load balancer can be used endpoints must
be synchronized from the Kubernetes API:

err := lb.SyncEndpoints()

Once the initial set of endpoints have been populated clients
can call the Next method to get an endpoint:

endpoint, err := lb.Next()
// ...
url := fmt.Sprintf("http://%s:%s", endpoint.Host, endpoint.Port)
resp, err := c.Get(url)
// ...

Endpoints can be synchronized in the background:

err := lb.StartBackgroundSync()

Shutdown terminates a load balancer instance by stopping any
background watches and reconciliation loops against the Kubernetes
API server:

lb.Shutdown()

This was a good introduction to the Kubernetes API and powerful example of the flexibility that Kubernetes offers. The full example from Kelsey Hightower is here: https://github.com/kelseyhightower/endpoints

Appendix

Kubernetes API

The endpoints module gets a list of endpoints in our cluster by using the GET /api/v1/namespaces/{namespace}/endpoints/{name} API, where namespace and name are the respective Namespace and Service the contain Pods for which you want the LB to discover.

Permalink: kubernetes-api

Tags:

Last edited by Alex Egg, 2016-11-17 21:10:15
Back | Edit
View Revision History