Kubernetes Support for LiteSpeed Web ADC

LiteSpeed is pleased to introduce the LiteSpeed Ingress Controller, which adds full support for Kubernetes to the LiteSpeed Web ADC.

The LiteSpeed Web ADC has been available for over 10 years and is a mature, stable product. Supporting lots of cutting edge features, including built-in Web Application Firewall, Zero configuration application specification and more, it is well suited for the Kubernetes environment.

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. Originally written by Google to solve their problem of scaling backends, Kubernetes takes containerized applications, usually containerized by Docker, and deploys them into pods which are simply one or more containers, instantiated. This is done using Kubernetes deployments, services, and ingresses. An ingress exposes the application container to a load balancer which then exposes it to the internet.

So many applications can run in Kubernetes without any modifications at all, but not a load balancer. A load balancer needs to know what’s happening in Kubernetes. A Kubernetes administrator generally spends their full time administering Kubernetes. Thus a Kubernetes-aware load balancer needs two components:

• A Kubernetes application. This continually monitors Kubernetes for changes to definitions and reacts to them.
• A conventional load balancer. This can spend its time doing the job of servicing requests from the internet and routing them to the correct containerized backend.

The two must coordinate their activities so that the conventional load balancer can react to changes in Kubernetes definitions.

The Kubernetes-aware program needs to be written in the programming language Go, and spend the bulk of its time monitoring Kubernetes itself.

We determined that since the load balancer component is busy monitoring the internet and backends and routing traffic between them, this was the ideal place for the existing LiteSpeed Web ADC controller.

ZeroConf + Helm

The LiteSpeed Web ADC controller already had a feature which allowed external programs to modify its configuration: ZeroConf. With ZeroConf, the Go program can monitor ingress definitions and when it sees a change, use ZeroConf to apply that change to the load balancer.

Kubernetes is all about containers so the load balancer and Go program are all packaged into a single container for distribution. One of our early goals was to make it easy to use our load balancer. There are a number of load balancing solutions already available on the market, and we wanted to make transition to ours as easy as possible.

Fortunately, Kubernetes has a tool named Helm, which eases distribution and installation of containers:

• We created a Helm chart, which is a complex specification about how to deploy an application, and posted it to our GitHub repository.
• You tell Helm to install our chart, and it does the work of downloading the container and fitting it into your Kubernetes environment.

If you have Kubernetes, you probably are already using Helm.

To support Kubernetes some changes were made to the ZeroConf interface, but ZeroConf remains backwards compatible. Note that since both the load balancer and the Go program are distributed as a pre-built container, there are never incompatibilities between the Go program and the load balancer. They are always distributed in the same container.

Plus, our ZeroConf tool continues to benefit as new features are added with Kubernetes in mind, as these features may be applicable to other users needing API control of the load balancer. All of these changes are fully documented.

Advantages of the LiteSpeed Ingress Controller

Kubernetes uses an ingress to make deployed pods of containers available to the load balancer so they can be exposed to the internet. To recognize traffic which must be routed by our load balancer, you add an annotation which we can recognize to your ingress specification. This is the only specification you need to use our load balancer. And if you want, you can set up Web ADC as the default load balancer and avoid any specifications at all. Since we can recognize traffic destined for our load balancer, you can run multiple load balancers – even from different vendors – and LiteSpeed will stay out of their way.

When compared to the free versions of similar controllers, there are a number of advantages to the LiteSpeed Ingress Controller:

• Web admin interface. Besides the Kubernetes controller commands available with kubectl, the LiteSpeed ADC WebAdmin Console is available with real-time stats.
• High speed, low drag WAF (web application firewall) which uses ModSecurity rules and templates to ease implementation.
• ZeroConf Ingress management, which removes the need to restart the load balancer to apply Kubernetes changes.
• Easy Helm installation with full 24-7 support.
• Seamless integration with the LiteSpeed WebADC engine for load balancing.
• Native ingress support, with no sidecars or Kubernetes modifications required.
• Automatic ConfigMap management. No need to manage Kubernetes ConfigMaps.
• Advanced red/blue and canary routing without the need for special Kubernetes changes.

Security

Security is an important part of any internet connected device, so our load balancer is heavily security aware. HTTPS is the heart of the secure internet, encrypting data and guaranteeing its proper authorization using certificates. Kubernetes stores certificates in an internal structure known as a “secret.” Our Go program is able to extract these secrets and fortunately, ZeroConf supports certificates on a per-domain basis. The certificate verification is done at the load balancer level, which relieves the backend machine of having to manage them. It even allows the communication between the load balancer and the backend to be HTTP if you wish.

Also, Kubernetes has an open source cert manager that is recommended in large environments. Due to our Go program’s ability to access secrets, we are able to utilize it.

By default, if there is a certificate for an ingress, traffic will be redirected to HTTPS. But we recognized that to be flexible, we would need to be able to deny or redirect HTTP traffic by ingress (through annotations) or system-wide.

We also decided that we needed to be able to recognize specific ingress definitions and support HTTPS-only traffic on those pods. Thus we added support for individual ingress annotations that enforce HTTPS-only traffic, and we added support for enforcement, or the lack of enforcement, at the overall level.

WAF

The most advanced security feature in the LiteSpeed load balancer is the Web Application Firewall (WAF) which uses ModSecurity rules and works with the standard rule sets like OWASP or Comodo.

We recognized that the WAF was a critical feature for the Kubernetes Load Balancer. Fortunately, ZeroConf had the perfect mechanism: templates.

Templates must be configured at the startup of the load balancer. To enable easy configuration of templates, you can optionally expose the LiteSpeed Web ADC configurator. It defaults to port 7090 and is disabled by default.

Enabling that port makes the configurator available, along with the rest of the WebAdmin console. The generalized interface for saving configuration information in Kubernetes is ConfigMaps. The Go program monitors file writes, which are generated when you save configuration changes in the WebAdmin interface. When a file write to the configuration directory is performed, it saves the file as Kubernetes ConfigMaps. Similarly deletions are monitored, so the ConfigMap is deleted when the file is deleted. And to complete the cycle, ConfigMaps are automatically re-loaded on load balancer startup.

Additionally, files written to the configuration directory configmaps in the load balancer pod are also saved to ConfigMaps. This gives you a mechanism to persist configuration information, including complex ModSecurity rule files.

Thus you do not need to be aware that ConfigMaps are used. They are managed when you perform configuration and startup.

So you can create a template which specifies, say, the Comodo ModSecurity rules, and applies them to some or all of the inbound traffic to your Kubernetes cluster. And you can do it all through the addition of a single annotation to your ingress specifications.

As a bonus, when you activate the configuration, the real-time stats feature becomes available. It can be quite useful in monitoring traffic in your cluster.

Control

Typically in Kubernetes there are three points of control of the load balancer:

1. Command line parameters when it is started. These are set as helm parameters. It was our goal to offer a large range of these so that the load balancer could be tailored for the needs of specific user environments. Useful defaults are chosen such as 80 for HTTP and 443 for HTTPS,
2. Ingress definitions. These are architected within Kubernetes and as a design requirement, we had to meet the needs of all of the features documented there including dynamic modification. These include path types, host name wildcards, and fan-out by paths, A part of Ingress definitions may be annotations, simple text controls by ingress, and LiteSpeed needs to utilize them as a control mechanism.
3. Configuration. Application configuration In Kubernetes is generally done using ConfigMaps. And as you see above, these are managed without any explicit user interaction.

Advanced Routing

Making the load balancer easy to live with is a continuing goal. The Kubernetes ingress specification helps with this by supporting wildcards in domain names, and paths including both prefix and exact specifications for fan-out routing.

The load balancer also adds support for regular expressions in paths. Thus you can route by path wildcards. For example you could specify /gr[ae]y which would route traffic for /gray and /grey in one specification. Since the regular expression facility is rich, this allows all sorts of imaginative control.

As part of regular expression, there is also a rewrite-target specification which lets you modify the input specification to a replaceable parameter. For example, you could specify a path of /something(/|$)(.*) and a rewrite target of /$2. This asks the load balancer to only rewrite the second wildcard specification, the first being (/|$), and the second being (.*), which is the generic match-all wildcard spec. Thus, if /something/new is received, it would fail the first wildcard, match the second and be routed to the backend using /new. While this is complex to get your hands around, it allows a huge amount of control and flexibility in routing requests to backends you may not have much control over.

Red/Blue and Canary Routing

When you have a new containerized image to deploy you can indeed just patch the deployment and do a RollingUpdate to force it out. The Kubernetes documentation has examples that show how to do it.

However, you might want to try it out on a subset of users. Say, give it to 10% of your user population and see if there are problems. This is typically described as a “canary” deployment. You might want to increase the percentage until most users are using it. Then, pass over control completely to the new group.

Or you may want to set up the deployment, and swap it over knowing you can swap it back quickly. That’s a “red-blue” deployment.

The LiteSpeed Ingress Controller supports both. And it does it using standard Kubernetes objects, no extensions or sidecars required.

We recommend that you set it up in advance and give it a try. However, you can patch it in for immediate deployment if you wish.

The technique uses the fact that you can configure an ingress with two paths or definitions with different names and different service definitions. Then you use an annotation to describe which one is to get traffic based on the name and a weight. A weight is a percentage of traffic that goes to a given name. Once you have set it up you can use a single kubectl command to change the routing from red to blue, add a canary, or change the level that canaries are being routed.

To further simplify the deployment process, regular expressions are supported in the annotation.  They can be used to specify multiple domains or services in a single definition.

The Future of Kubernetes Support for LiteSpeed Web ADC

We see Kubernetes as a strategic platform so it is a work in progress with new features announced regularly. LiteSpeed Ingress Controller is currently in beta. If you try it, please contact us with any questions, difficulties, or feedback. Our development team is committed to providing interesting and unique ways that the evolving world of Kubernetes can be put to use in your environment.