HyperText Transfer Protocol (HTTP) might have been created back in 1989 but it has certainly withstood the test of time. It laid the foundation for the World Wide Web and it remains ever popular and the protocol of choice for the vast majority of APIs (despite the more recent alternatives like GraphQL and gRPC). And for the past couple of decades, we’ve used Layer 7-aware load balancers to control, alter and route HTTP traffic as it entered our network.
In this blog post, we are going to explore how you can alter HTTP traffic as it enters Kubernetes clusters, focusing on three specific use cases:
- HTTP Redirect – to tell the client that the resource they are trying to access has moved
- HTTP Path Rewrite – to rewrite the URL or entire path used by the client
- HTTP Mirroring – to copy the traffic sent from the client to another backend
In the world of Kubernetes, it’s the role of the Gateway API to modify and control traffic as it enters our clusters.
We explored in the previous “Getting Started with the Cilium Gateway API tutorial” how you can use Cilium’s Gateway API support to load-balance HTTP traffic and alter HTTP header requests and responses.
We’ve also recorded several short tutorials on the Gateway API, including the one below on “TLS Passthrough”, where traffic can be encrypted all the way through to the server.
In this tutorial, we will explore these use cases. If you’d rather watch me present it, check out this episode of the eBPF and Cilium weekly show eCHO:
If you’d rather do it yourself, follow the steps below. Let’s start!
Demo environment
Before we get starting with the first use case – redirecting HTTP traffic – let’s get our environment ready.
In this tutorial, we will be using a cluster in Azure Kubernetes Services (AKS) in BYOCNI mode. In this mode, a cluster is deployed without a Container Network Interface (CNI) so that we can bring our own – Cilium in this instance.
To deploy the cluster without the CNI, I am following the docs:
Expect a lengthy JSON output, beginning with this.
My cluster is called nicovibert-4665 in this instance. Note that your nodes won’t show as Ready for now – there’s no CNI and therefore connectivity is limited.
Expect an output such as:
When using kubectl, you might even see some error logs such as couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request which is just because of the lack of connectivity prior to installing Ciliium.
Let’s now use the latest Gateway API Custom Resource Definitions (CRDs). Remember to install them before installing Cilium:
Support for the features we are exploring today – and for the 1.0 Gateway API version – was recently added to the main branch of Cilium – to use them, you will need Cilium 1.15.
Use this Cilium CLI command (install the CLI with the instructions here if you’ve not done already):
Expect this output:
Remember that the kubeProxyReplacement feature (KPR) is required for the Cilium Gateway API. Note that, even when creating the cluster in BYOCNI mode, kubeProxy was deployed in the cluster.
I could have used this new option instead to skip its deployment as it is no longer needed once Cilium is deployed in KPR mode.
After installation, check the Cilium status with:
Expect the status to be:
The status for both Cilium and Operator should be OK. Now that our environment is ready, let’s go ahead and deploy our demo app and our Gateway before deploying the HTTRoutes to illustrate the use cases we will cover in this blog post.
We will use this YAML manifest I have put on a GitHub repo.
This configuration is actually a subset of configs used in the Gateway API conformance tests. Remember that one of the benefits of the Gateway API project is consistency. To make sure that each Gateway API provides a consistent user experience, each Gateway API implementation is tested against a set of conformance tests that creates a series of Gateways and Routes and tests that the implementation matches the API specification.
Deploy this demo app:
Expect this output:
We are re-using the same Gateway configuration we used in the previous blog post. It’s a simple Gateway configuration (it’s simply listening for HTTP traffic):
Expect this output:
Let’s double check that the Gateway has been installed and it has received an IP address. Remember, that in public clouds, the IP address allocation is done automatically for you whereas on private cloud, you would need to use MetalLB or Cilium’s own LoadBalancer IP Address Management feature to assign this IP.
Expect an outcome, such as:
Let’s save this IP as the $GATEWAY variable.
HTTP Redirect
One common use case for ingress gateways is to send HTTP redirects to clients in order to tell them that the resource they are trying to access in the cluster has moved to a different location.
This is a really common requirement for migration, content optimization, SSL/TLS enforcement – the list goes on.
Redirects return HTTP 3XX responses to a client, instructing it to retrieve a different resource. With the Gateway API, we can use redirect filters to substitute various URL components independently, as you will see below.
Let’s use this HTTPRoute YAML for the four examples in this section. We will review each rule in detail below.
Path Redirect
In this first example, we will do a simple redirect: we only replace a portion of the URL and redirect them there:
You should see the IP address allocated to the Gateway (such as 20.115.194.177).
The following rule will match traffic to /original-prefix and redirect the client to a different URL:
Let’s try, using curl.
Notice we use -l in the curl request to follow the redirects (by default, curl will not follow redirects) and that we use the verbose option of curl to see the response headers.
The location is used in Redirect messages to tell the client where to go. As you can see, the client is redirected to http://20.115.194.177:80/replacement-prefix. The prefix was replaced from /original-prefix to /replacement-prefix. Note we support different models to replace the URL – we can replace just a portion of the path or the entire one.
Host & Path redirects
You can also redirect the client to a different host.
Let’s try, with this specification, to redirect the client to example.org:
Let’s make HTTP requests to that external address and path:
Expect an output such as:
As you can see, the client is redirected to http://example.org:80/replacement-prefix.
Both the hostname and the path prefix were modified.
Redirect to new prefix and custom status code
Next, you can also modify the status code. By default, as you saw above, the redirect status code is 302. It means that the resources have been moved temporarily.
To indicate that the resources the client is trying to access have moved permanently, you can use the status code 301. You can also combine it with the prefix replacement.
Let’s use this example:
Try to access this URL:
Expect an output such as:
Note the status code returned is 301 Moved Permanently and the client is redirected to http://20.115.194.177:80/replacement-prefix.
Redirect to new prefix and custom status code
Finally, we can also use the Gateway API to impose tighter security controls. You can redirect the client to use HTTPS instead of HTTP; changing the scheme used by the client.
Look at the last line in this specification:
Let’s try it.
Expect an output such as:
As you can see, the client initially tried to connect via HTTP and is redirected to https://example.org:443/scheme-and-host:
HTTP Rewrite
Sometimes, we don’t need to redirect the client to a different URL. Instead, we can simply rewrite components of a client request.
Let’s explore this with a couple of examples. Create the HTTPRoute by applying the manifest below:
The Gateway will replace the /prefix/one in the URL request to /one.
Let’s now check that traffic based on the URL path is proxied and altered by the Gateway API:
The request is received by an echo server that copies the original request and sends the reply back in the body of the packet.
As you can see, the Gateway changed the original request from /prefix/one to /one.
As we use the Envoy proxy for L7 traffic processing, note that Envoy also adds the information about the original path in the packet (see "X-Envoy-Original-Path").
We can also combine this, with previous Gateway API features we explored in the previous tutorial. You might want to rewrite traffic and add some headers to it to add some metadata (so that the receiving server can interpret it accordingly).
If you look at the HTTPRoute we’ve just created, you will see that traffic to /rewrite-path-and-modify-headers will not only be partially rewritten, but that some headers can be added, removed or modified.
Expect an output such as:
HTTP Mirroring
In this final example, we will review how to mirror traffic. There are plenty of use cases for it – monitoring incoming traffic for forensics, analysis, logging, troubleshooting, etc…
With the following rule, we can mirror traffic meant for infra-backend-v1 to infra-backend-v2.
Apply it:
The Gateway will forward the traffic to infra-backend-v1 as standard, but will also copy it across to infra-backend-v2 (purple arrow in the image below). The response from infra-backend-v1 will be processed normally by the Gateway but the responses from infra-backend-v2 will be ignored (red arrow).
Let’s make a request to the /mirror path.
Look at the output below. Traffic was sent to the infra-backend-v1 Service. Was it also mirrored to infra-backend-v2 ? How can we prove it?
The image used by the echo Pods serving this Service is minimal. Instead of trying to install tcpdump on it, let’s use the Kubernetes debug functionality instead, with an image (nicolaka/netshoot) that is designed to troubleshoot networking issues.
The command below will deploy a container in the same Pod as the infra-backend-v2 Pod and will see the traffic coming in:
Once you run the curl command again from a different terminal, you should see traffic appearing (you may have to run it on a few occasions as traffic will be load-balanced between multiple infra-backend-v2 Pods):
And this shows that the Gateway mirrored traffic to the infra-backend-v2 Service.
Post Demo Clean Up
After testing, you can clean up your demo environment and remove both the cluster and the resource group with the following commands:
Recap
Cilium might be famously known for being a high-performance CNI and its network policy engine but it’s equally capable of doing L7 load-balancing. Unlike my days as a network engineer where I would have to drive to a data center, lug a load balancer onto the rack, install it and configure it, with Cilium, I don’t have to install anything else – it’s just another feature to enable and a YAML manifest to apply.
In this blog post, you will have learned – and hopefully tried alongside me – how Cilium Gateway API can send HTTP redirects, rewrite URL paths and mirror HTTP traffic. It’s another powerful addition to all the other Layer 7 Load Balancing features that Cilium Gateway API already supported.
Think this might be useful for your applications? Got any use case in mind? Let me know, on the Cilium Slack or you can find me on LinkedIn.
Learn More
- YouTube Playlist of Cilium Gateway API videos
- Cilium Gateway API Lab
- Advanced Gateway API Use Cases Lab
Prior to joining Isovalent, Nico worked in many different roles – operations and support, design and architecture, technical pre- sales – at companies such as HashiCorp, VMware, and Cisco.
In his current role, Nico focuses primarily on creating content to make networking a more approachable field and regularly speaks at events like KubeCon, VMworld, and Cisco Live.
