Kuma Meshes Head-On – A beginners guide

To quickly start learning Kuma, on of the most important things we need is cluster. Then we also need a command to find out the status of our pods in kubernets (aka k8s), we also need to be able to install Kuma and finally we also need to be able to issue some Kuma commands.

This is a long way of saying that we need to install 4 essential commands in order to make everything ready for Kuma. These commands are:

kind – This is also known as Kubernetes in Docker. This is a command that leverages the weight of creating stuff with only kubectl.

kubectl – Probaly the most expect one of this list, if you are already used to working with k8s. This is how we can issue commands to our k8s cluster.

helm – Helm allows us to execute some very handy scripts that allow amongs others, the installation of the Kuma control plane.

kumactl – We will no be using this command very often in this guide, but it is important to be aware on how to use it.

This guide will let you know how to do this in Ubuntu. All of this has been tested in an Ubuntu system. If you are interested in a guide on how to install this in Mac-OS or Windows or any other operating system you may have please give me a shout out at my YouTube channel JESPROTECH community.

I. Installing the commands

Kind (k8s) in Docker

In order to install kind, we need to issue these commands:

It is important to note that the command kind will be installed in you /usr/local/bin/kind. This may vary per system, even withing Linux distributions.

Installing certificates and GPG keys

Both helm and kubectl commands need to be installed with the presence of certain GPG keys. This is how we can add them to our local repository of our linux apt distribution:

kubectl

The installation of Kubectl is very easy once the previous step is complete:

The commands kubelet, kubeadm and kubectl aren’t mandatory, but it is a good idea to install them.

helm

As you may have already guessed, helm is also now very easy to install:

kuma

Kuma installation can be a bit cumbersome because it involves one manual step, but first we need to download our dependencies:

Be sure to be in your HOME folder before issuing this command. It is important to have Kuma installed in a place where it is easily accessible and easily spotted should we, for example, decide to remove it.

Once we are done with that, it is also very important to add the bin folder to our PATH:

Adding this line to the end or anywhere in between your star up script will make this process easy. Your startup script may be any of these .bashrc, .zshrc, .profile and possibly take another form.

k9s

Installing k9s is also quite different from other applications. In this case we can either use pacman or brew for Linux. I have used brew mostly for Mac-OS and hardly ever needed it in Linux, but in this case it is very much needed and so to do that first we need to install brew like this:

Once the brew installation completes, all we have to do is to install k9s(“kanines”):

One thing that is important to take into account, and you’ll probably notice this once you installed and start running k9s for the first time, is that k9s will crash if a cluster that it is monitoring gets removed and/or added.

II. Creating the cluster

The first command creates a cluster named wlsm-mesh-zone. This is just a cluster that we will use to install Kuma.
The second command is used to check the status of the cluster.

III. Creating a local docker registry

As I mentioned before, we can create a docker registry quite easily. As easy it may sound to create it, the script to do this is a handful. So the best thing to do is to just copy past what kind already has available on their web-site. Here we can download this script:

This script can be found on the root folder of the project.
And to install the local docker registry, we only need to run this bash script.

IV. How the code has been created

There may be a lot to be said about the code that I have provided for the example for this blog post. However, in this case, let’s just focus on few key aspects. Let’s start from the listener service to the collector and then to the database.
When we run the services locally, or even using a docker-compose configuration to get the containers going, usually we use the dns attributed names, which automatically get assigned to be the container name or the name that we configure with hostname. With k8s there is also a set of rules that make the host names available throughout the cluster. Let’s have a look the listener and collector examples:

Listener example

The listener is an application developed in Java using the Spring framework. Like all application created this way, there is also an application.properties file:

In all of these properties, the most important one to focus on for the moment is the wslm.url.collector property. With the default configuration, we can run this service locally without the need to use any containerized environment. However, in the k8s cluster, we need to be able to access the collector and for that we have a prod profile with the definition file application-prod.properties:

This property tries to reach host wlsm-collector-deployment.wlsm-namespace.svc.cluster.local. This file follows this configuration:

<Service Name>.<Namespace>.svc.cluster.local

We’ve got 5 dot separated elements. The last three are static and the first two depend on the machine we are trying to reach. On the left we place the service name followed by the namespace. This is important to understand how the containers are connected to each other within the cluster.

The part of the code that is interesting to have a look at is of course the controller and the service. The controller looks like this:

And the service looks like this:

As you may have already noticed, this first application, like all of the applications implemented using the Spring Framework in this repository are all reactive and they all use netty instead of tomcat.
For the moment, we can ignore the hazelcast usage in this code. This will be used for later versions of this project.

Collector example

The collector works in exactly in the same way as the listener at this point. Its only duty for now is to relay data from the listener to the database and to do that, the collector only needs to know exactly where the database is. Let’s make the same analysis on the application.properties file of this project:

These properties are the minimum required to get the service going. However this is only to able to run it local. And for this service, we also have prod profile file and we can have a look at it in application-prod.properties over here:

The database connection is in this case referring to the host of the database:

wlsm-database-deployment.wlsm-namespace.svc.cluster.local

Which again follows the same analysis as we have seen before. To the left we see the sevice name, followed by the namespace appending that at the end with svc.cluster.local.

And for this service we also use a controller and a service. The controller looks like this:

And the service looks like this:

The service uses and event publisher that is called applicationEventPublisher, that follows an event streaming architecture that gets handled later on in this event listener, which we can readily see that it uses r2dbc to keep in the reactive architecture implementations paradigms:

V. Deploy scripts

Deploying is normally a very straightforward task to do with k8s. However it is also important to have a look at the configuration needed for our services. For example let’s have a look at the listener implementation:

There are three blocks in this configuration. The first block is the namespace block. The namespace configuration is crucial to allow kuma to be able to inject the envoy sidecars that it needs to apply policies. Without a defined namespace, kuma will not be able to do this. The other thing that we need to pay attention when configuring kuma is that the namespace must containe the proper label that kuma will recognize:

kuma.io/sidecar-injection: enabled

The namespace definition with the correct label is vital to get kuma working. In the second block we find the definition of the deployment. This is how we define how the deployment of our pod is going to look like in our kubernetes cluster. What is important here to focus on is the image, the imagePullPolicy and the containerPort. The image is the complete tag of the Docker image we are using. The port that gets configure for our docker registry created with kind is 5001 and this is included in the tag for our image. It works as a tag but also as a connection to our Docker registry. That way we can pull the images and create our container to run in our kubernetes environment.

But, of course, to be able to use images, we need to create them and for that let’s take a look at how is that done in the listener example and the database example. The docker image for the listener is defined like this:

This is all starting from a base imaged called eclipse-temurin:21-jdk-alpine. After this we just copy the jar created by building the project and then making a copy of it into our image. Before of that we copy the entrypoint.sh to the container as well and define the ENTRYPOINT to use it. The entrypoint simply calls the jar like this:

The database service is quite different because it uses a few scripts that are opensource and available online:

This script makes a copy of the following file and folder to the docker init directory: create-multiple-postgresql-databases.sh and multiple. Finally we simply define the variables used in those scripts to define our database and username/password combinations.

The database is created using the following schema:

And as a data example, we will register one animal by the name of piquinho. Piquinho is simply the name of travelling albatross that is travelling around the world, which has a sensor attached to it and we are reding the data that the sensor is sending to use. There are two tables that define species. That is the species and the genus that define species. These are tables families and genuses. The table species defines the species that animal belongs to. Finally we define an animal in the table of the same name where the species and the name of the animal gets registered. The database looks like this:

In order to build, create the images and start our project we can run the following commands which are available in the Makefile:

The first make is just a gradle build command. The second command used the variable:

to run:

This simply goes through every module, and uses a standard generic command that changes per value given in the MODULE_TAGS to create the images and push them to the local registry on port 5001. Following the same strategy, we can then use the third command to deploy our pods. This third command uses a different loop that looks like this:

In this case, it applies every deployment script to every single one of the services. If we run the command kubectl get pods –all-namespaces, we should be getting this output:

What we should observe here at this point is the presence of the kuma-control-plane, the kube-controller-manager and all the services running in our own custom wlsm-namespace. Our cluster is isolated from the outside, and in order to be able to access the different ports, we need to create port-forwarding for every pod we want to access. For that we can issue these commands in separate tabs:

We can also have a look at this by looking at k9s:

VI. Running the application

In order to run the application, we should open all the ports and when all of them are open we should see something like this in our screens

We can connect to the database using localhost and port 5432. The connection string is this one: jdbc:postgresql://localhost:5432/wlsm. And to access it we then use the username/password combination of admin/admin.

The first thing we need to do, before we perform any test is to know the id of Piquinho and we can do that by using Intellij database tools like this:

In the root folder of the project, there is a file called test-requests.http. This is a scratch file to create REST requests against our open ports:

In order to be able to use this file, we only need to replace the id in this example from 2ffc17b7-1956-4105-845f-b10a766789da to d5ad0824-71c0-4786-a04a-ac2b9a032da4. In this case, we can make requests from the collector, or from the listener. Both requests should work and we should see afterward this kind of response per request:

Because both ports are opened and they at this point share the same payload type, we can perform the same requests to the listener and to the collector. After making those two requests we should find results in the table animal_locations:

So this confirms only that the cluster is running correctly and now we are ready to test policies with our Kuma mesh.

VII. MeshTrafficPermission – Part I

The MeshTrafficPermission is one of the features we can choose in Kuma, and it is probably the most used one.

But first let’s take a moment to explore the kuma control plane. With all the porforwarding on, we can just go to localhost:5681/gui and visualize our kuma meshes. In the main page we should see something like this:

There is nothing much to see at the moment, but let’s now apply the MeshTrafficPermission:

Once we apply this, we should be getting a response like this: meshtrafficpermission.kuma.io/mtp created.

VIII. Mesh

Applying the mesh only doesn’t change much when it comes to the setup of our cluster. What it does do is allow us to set up traffic routing policies. There are many things that we can choose from, but one of the most obvious thing we can choose from is mTLS otherwise referred to as mutual TLS, which in very short terms means that certificates are mutually accepted and validated in order to establish identity between parties and establish encrypted data traffic. This can be automatically done for us using this simple Mesh configuration:

After applying this policy we may come across a warning like this one:

Warning: resource meshes/default is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create –save-config or kubectl apply. The missing annotation will be patched automatically.

For now, we can ignore this warning.

IX MeshTrafficPermission – Part II

Now comes the fun part and the first thing we are going to do is to disable all traffic between all pods:

And after we get the confimation message meshtrafficpermission.kuma.io/mtp configured, if we try to make any request using any of the port-forwarding, we’ll get:

This means that all traffic between pods is being denied. What we now have is an internal system protected against possible bad actors within our organization, but we have also now blocked traffic between all pods. So mTLS is a great thing, but blocking all traffic not at all. The way to make this perfect is simply to make exceptions to that DENY all rule and to do that we need a policy that will allow traffic between, the listener and the collector and the collector and the database. Let’s start with the traffic between the collector and the database:

In this case, what we are doing is allowing data traffic to flow from the targetRef collector to the targetRef database. If you don’t know this, perhaps it is important to note how kuma interprets the name, which, just like the hostname creation, it is also used for functional purposes. The generic way to build these names is like this:

<service name>_<namespace>_svc_<service port>

In this case, the separator is an underscore, and creating a name, this way, lets Kuma know exactly what is permitted. In this case if we apply this policy, we’ll be able to send requests to the collector after getting this response: meshtrafficpermission.kuma.io/wlsm-database created. And when making them, the response should now be 200 confirming that the location record has been sent to the collector:

However, we still didn’t define exceptions to the traffic between the listener and the collector and so making a request that way will result in this:

And this is of course expected. Let’s now apply another policy for this data traffic:

Making it possible to now perform requests from the listener to the collector:

X – MeshFaultInjection

Finally and just to provide another feature as an example, we can also use another feature called MeshFaultInjection, which can be very useful when performing tests with Kuma. We can simulate potential problems within our mesh and check if the error handling is being done correctly for example. We can also check other things like how possible circuit breakers we may have configured may react to faulty connections or high rate requests.

So let’s try it. One way to apply MeshFaultInjection is like this:

With this policy, we are saying that the traffic outbound from the listener and inboud to the collector will have a 50% chance of success. The request results are unpredictable and so after the applying this policy, we may expect errors or successful requests to the listener endpoint.

Finally just out of interest, we can have a look at how our animal_location table looks like now:

XI – Conclusion

I hope you were able to follow this article so far and that you were able to have cluster running on your machine. Thanks anyway for reading this article and for giving up a bit of your time to understand an learn a bit more about Kuma. I personally se a great usage for this and a great future for Kuma as it makes it possible to configure and take a much more granular control of our network and our environment. It’s enterprise version, Kong-Mesh seems quite complete. Kuma is opensource and its great for testing and also for enterprise it seems. I find the subject of meshes very interesting and I think Kuma provides a great way to learn about how meshes work and to get a feel on how can we better control the data flow withing our network.

If we want to see the status of our services, we can just go to our Kuma control plane in this localhost location: http://localhost:5681/gui/meshes/default/services?page=1&size=50:

In Kuma control plane we can also have a look at the policies installed, checkout the status of our pods, monitor what is going on in the background and in generally just have an overview of what is happening in our mesh and how it is configured. I invite you to just go through the application and see if you can check the statues of the policies we have installed. The Kuma control plane a.k.a. the GUI, is made precisely to be easy to understand and follow up on our Mesh.

XII – Resources

What is mutual TLS (mTLS)?
K9s
North-South traffic
East-West traffic
Getting Started With Kuma Service Mesh
Deploy Kuma on Kubernetes
Local Registry with Kind

I have also made a video about it on my JESPROTECH YouTube channel right over here: