Building Agents On Kubernetes With Kagent

One of the most popular advancements in cloud-native at this time is LLMs and AI Agents, which can help you troubleshoot your environment, build out new environments, and even template out a code base for you.

The problem is that there hasn't been a way to truly run Agents in a practical manner in Kubernetes, which is currently the most popular way organizations are orchestrating distributed systems.

In this blog post, you'll learn how to implement and build Agents in k8s with kagent.

Prerequisites

To follow along with this blog post from a hands-on perspective, you should have the following:

1. A Kubernetes cluster running. This could be Kind/Minikube/whatever other local method or a more production-ready cluster like AKS/EKS/GKE.
   1. If you're using a local cluster, you may need Docker Desktop installed.
2. An API key for a supported provider (Anthropic, Gemini, etc.). You can see all of the supported providers here.

If you don't have a Kubernetes cluster, that's totally fine! You can still follow along from a theoretical perspective and use the hands-on piece at a later time.

Why Kagent

As the AI and ML landscape changes, it's important to understand all of the pieces across the space. You can more or less break it down into two categories:

1. Building/training Models
2. Using/building Agents

For the building and training Models in the Kubernetes space, projects like Kubeflow and MLFlow are how engineers can train and build Models in the traditional sense (training Models based on data sets) on Kubernetes. This is great because one of the biggest things that Kubernetes provides is the ability to run applications at great performance in a consolidated environment (Pods/Containers) without having to use an entire server or virtual machine. Because training Models is so expensive, there's a significant cost and resource savings when using Kubernetes for this type of work.

The one thing that engineers haven't had the ability to do run and secure Agents (Claude, Gemini, GPT, etc.) on Kubernetes.

Enter Kagent.

With Kagent, you get:

• The ability to run Agents on Kubernetes.
• A method of building Agents on Kubernetes.
• CRDs/extended APIs to manage Agents on Kubernetes.
• Built-in troubleshooting help. For example, you can use Kagent to run Agents that can tell you if/when/why a Pod went down.

The overall goal is to run and build Agents in Kubernetes, which makes sense for a lot of organizations that are using Kubernetes as an orchestrator.

Installing Kagent

Now that you have a good understanding of why kagent is beneficial, let's dive into the hands-on piece. For the purposes of this section, you'll see how to get a minikube cluster up and running, along with the installation of kagent.

1. Install Minikube. You can find the installation for your architecture here.
2. Use the start command to run a new cluster.

minikube start --cpus 4 --memory 8192 --nodes=2

You'll see an output similar to the following:

😄  minikube v1.36.0 on Darwin 15.6.1 (arm64)
🆕  Kubernetes 1.33.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.33.1
🎉  minikube 1.37.0 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.37.0
💡  To disable this notice, run: 'minikube config set WantUpdateNotification false'

✨  Using the docker driver based on existing profile
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🚜  Pulling base image v0.0.47 ...
🔄  Restarting existing docker container for "minikube" ...
❗  Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.34.0 -> Actual minikube version: v1.36.0
🐳  Preparing Kubernetes v1.31.0 on Docker 27.2.0 ...
🔗  Configuring CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons:

3. Check to make sure your Nodes are up and running.

kubectl get nodes
NAME           STATUS   ROLES           AGE    VERSION
minikube       Ready    control-plane   297d   v1.31.0

In the next section, you'll install kagent.

Installing kagent

With your Kubernetes cluster up and running, it's time to install kagent. Luckily, the installation is straightforward as it's just a Helm chart.

1. Use Helm to install the kagent CRDs so you can use the objects/kind provided to manage agents.

helm install kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \
    --namespace kagent \
    --create-namespace

2. Create an environment variable containing your OpenAI API key.

export OPENAI_API_KEY=your_api_key

3. Install kagent.

helm install kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
    --namespace kagent \
    --set providers.openAI.apiKey=$OPENAI_API_KEY

4. To see the UI, port forward the kagent UI.

kubectl port-forward svc/kagent-ui -n kagent 8080:8080

You can now configure kagent for your Kubernetes environment.