After relocating to South Korea, I began provisioning resources in the Korea Central and Korea South regions, moving away from the United Kingdom South. I quickly noticed that specific SKUs, and sometimes even entire resources or services, were unavailable in my new region. While this was manageable for my Proof of Concepts (POCs) and personal projects, such limitations could pose significant challenges for organizations operating in the region, particularly when compliance or security requirements necessitate keeping data within South Korea.

These regional limitations, coupled with the varied performance metrics offered by different Cloud Service Providers (CSPs), the potential benefits of partnership programs (like discounts or free resource allocations), and the strategic advantage of minimizing vendor lock-in, are compelling more companies to adopt multi-cloud environments. However, this shift doesn’t come free and bring with it multiple challenges, such as the increased difficulty in monitoring and the need for engineers to possess a broader range of skills to integrate these diverse resources effectively.

In this article, I aim to explore the deployment of Kubernetes clusters on AWS and then use Azure Arc to manage these resources from a unified control plane along the other native Azure resources.

What is Azure Arc?

Before we delve deeper into the demonstration, let’s clarify what Azure Arc is and how it functions. Azure Arc was introduced in April 2021, initially focusing on managing databases, with a particular emphasis on PostgreSQL servers. However, its support was quickly extended to include a wider range of resources, such as Virtual Machines, VMware vSphere, System Center VMM, and Kubernetes Clusters, both on-premises and from other Cloud Service Providers. Each resource type has its specific requirements for integration. For instance, to connect an external Kubernetes cluster to Azure Arc, the cluster must be running Kubernetes version 1.13 or later. Additionally, there must be connectivity to the cluster through ports 443 and 9418.

The core functionality of Azure Arc involves “projecting” your external resources into Azure. This is achieved by installing Azure Arc agents on these external resources. These agents facilitate communication with Azure, enabling management tasks such as patching, policy enforcement, and tagging across your infrastructure. Once connected, these resources appear in the Azure portal alongside your native Azure resources. Additionally, Azure Arc enhances monitoring capabilities by integrating with Azure Monitor, providing insights for all cloud-based resources registered under its umbrella.

From a financial point of view, this service is available at no additional cost. This is probably a strategic decision by Microsoft that aims to encourage customers to leverage its services over competitors’ or on-premise solutions, especially for those already utilizing other Microsoft services.

Deploying Kubernetes Clusters on Different Clouds

After setting up a new Kubernetes cluster on AWS, the first step is to grant your user account with the necessary permissions. This is done by assigning the AmazonEKSClusterAdminPolicy access policy to your user.

Next, it’s time to create a Node Group. This Node Group will be linked to your EKS cluster and hosts the virtual machines, essentially the backbone where your application pods will reside and run.

With your cluster and node groups in place, the next crucial step is generating a config file. This file will contains all the information required to access the Kubernetes cluster, such as cluster, user, server, and certificate.

Moving to Azure’s part of the setup, you’ll need to enrich your Azure CLI with several extensions crucial for Kubernetes management via Azure Arc:

Additionally, you must register the necessary Azure resource providers to support Azure Arc-enabled Kubernetes features:

Now that we have provisioned out Kubernetes cluster on AWS, we can move forward and connect it to Azure via Azure Arc. The first step is to go to the Azure Arc service page, select Kubernetes cluster, and the click Add a Kubernetes cluster with Azure Arc.

Note: the Create a Kubernetes cluster with Azure Arc is focused on the creation of Kubernetes clusters on-premises which is out of scope of this article.

Next, select the resource group where the subcription and resource is going to be added, and give it a name and region. The region in neccessary as Azure doesn’t know the region o the target Cloud Service provider where the resource is provisioned. You will also need to specify how to connect to the resource; this options are the same across all resoruce types and are the following:

Public endpoint: The resource has direct access to the internet without any intermediary.

Proxy server: The traffic to and from the Azure Arc agent to Azure services are routed through a proxy server. This method is common in enterprise environments where direct Internet access is restricted for security reasons.

Private endpoint: The traffic is routed through the Azure backbone network rather than the public Internet.

After that, you will be able to define some tags, but the magic happens in the scirpt section.

This script will need to be executed on the machine with the EKS clutter context and will take approximatively 10 minutes to run. First it will run a series of prerequisite checks, then it will provision several resources on Azure, and the start of the Azure Arc agent installation on your Kubernetes cluster.

When you inspect your Kubernetes cluster after running the script, you’ll notice several new resource types. These resources enable a range of functionalities from secure communication to Git repositories configuration for GitOps.

You’ll also see multiple new pods within the azure-arc namespace. These pods use the resource types prefiously listed to perform specific activities like managing the cluster metadata and ensures it is synchronized with Azure Arc, collection of logs from across the cluster, and more.

Several of these pods will also internally communicate with each other and the Azure services via the following services.

The deployment also creates several Kubernetes secrets in the azure-arc namespace, which store sensitive information such as private keys, certificates, and tokens necessary for secure communication and operation within the Azure Arc environment:

Non-confidential configuration data required for the operation of Azure Arc like identity requests, Azure-related settings, Fluent Bit are stored in the following Config-Maps.

Finally, you can complete the Kubernetes cluster setup for Azure Arc.

Before being able to inspect the EKS cluster resources from the azure portal, we will need to create a Service Account token within your EKS cluster and add it to the Azure portal.

Next, you need to assign the necessary permissions to your new service account. In this case I liked the azure-user service account to the cluster-admin role, which grants it administrative access to the entire cluster.

Then, I’ll create a secret that holds the service account token.

Finally, extract and decrypt the token from the secret you just created.

With the token in hand, you can complete the connection between Azure Arc and your Kubernetes cluster.

After validating the token, you will be able to freely view and manage your cluster resources directly from the Azure portal.

References

–Azure Arc: https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/