freundcloud

Unit Testing Kubernetes Client Code: A Real-life Example

This guide demonstrates how to effectively unit test Kubernetes client code using the fake clientset from the client-go package. This approach allows you to test your Kubernetes interactions without requiring a live cluster.

Understanding the Kubernetes Clientset

The Kubernetes Clientset:

  • Contains clients for different API groups (Core, Apps, Batch, etc.)
  • Implements the Kubernetes Interface for interacting with the API server
  • Is typically created using NewForConfig() which requires a real cluster connection

Implementation Example

Step 1: Define Your Client Wrapper

First, create a package called client with a file client.go that defines a wrapper around the Kubernetes clientset:

package client

import (
	"context"

	v1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/klog/v2"
)

// Client is the struct to hold the Kubernetes Clientset
type Client struct {
	Clientset kubernetes.Interface
}

// CreatePod method creates pod in the cluster referred by the Client
func (c Client) CreatePod(pod *v1.Pod) (*v1.Pod, error) {
	pod, err := c.Clientset.CoreV1().Pods(pod.Namespace).Create(context.TODO(), pod, metav1.CreateOptions{})
	if err != nil {
		klog.Errorf("Error occurred while creating pod %s: %s", pod.Name, err.Error())
		return nil, err
	}

	klog.Infof("Pod %s is successfully created", pod.Name)
	return pod, nil
}

Notice that we’re using kubernetes.Interface instead of a concrete implementation. This dependency injection pattern is what enables effective unit testing.

Step 2: Using Your Client in Production Code

Here’s how you would use this client in a real application (main.go):

package main

import (
	"fmt"

	client "github.com/kubernetes-sdk-for-go-101/pkg/client"
	v1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
	"k8s.io/klog/v2"
)

func main() {
	// Build the clientset with a real kubeconfig
	kubeconfig := "<path to kubeconfig file>"

	config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
	if err != nil {
		panic(err.Error())
	}

	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		panic(err.Error())
	}

	// Initialize our client wrapper
	client := client.Client{
		Clientset: clientset,
	}

	// Define a pod to create
	pod := &v1.Pod{
		TypeMeta: metav1.TypeMeta{
			Kind:       "Pod",
			APIVersion: "v1",
		},
		ObjectMeta: metav1.ObjectMeta{
			Name:      "test-pod",
			Namespace: "default",
		},
		Spec: v1.PodSpec{
			Containers: []v1.Container{
				{
					Name:            "nginx",
					Image:           "nginx",
					ImagePullPolicy: "Always",
				},
			},
		},
	}

	// Create the pod
	pod, err = client.CreatePod(pod)
	if err != nil {
		fmt.Printf("%s", err)
	}
	klog.Infof("Pod %s has been successfully created", pod.Name)
}

When executed against a running Kubernetes cluster, this code creates a pod named test-pod in the default namespace.

Unit Testing with the Fake Clientset

Now, let’s write a unit test for our CreatePod function using Go’s testing package and the fake clientset:

package main

import (
	"testing"

	client "github.com/kubernetes-sdk-for-go-101/pkg/client"
	v1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/client-go/kubernetes/fake"
	k8stesting "k8s.io/client-go/testing"
)

func TestCreatePod(t *testing.T) {
	// Create a fake clientset
	clientset := fake.NewSimpleClientset()
	
	// Initialize our client with the fake clientset
	client := client.Client{
		Clientset: clientset,
	}

	// Define a test pod
	testPod := &v1.Pod{
		TypeMeta: metav1.TypeMeta{
			Kind:       "Pod",
			APIVersion: "v1",
		},
		ObjectMeta: metav1.ObjectMeta{
			Name:      "test-pod",
			Namespace: "default",
		},
		Spec: v1.PodSpec{
			Containers: []v1.Container{
				{
					Name:            "nginx",
					Image:           "nginx",
					ImagePullPolicy: "Always",
				},
			},
		},
	}

	// Create the pod using our client
	createdPod, err := client.CreatePod(testPod)
	if err != nil {
		t.Fatalf("Error creating pod: %v", err)
	}

	// Verify the pod was created with the correct name
	if createdPod.Name != "test-pod" {
		t.Errorf("Expected pod name: test-pod, got: %s", createdPod.Name)
	}

	// Verify the pod exists in the fake clientset
	pods, err := clientset.CoreV1().Pods("default").List(context.TODO(), metav1.ListOptions{})
	if err != nil {
		t.Fatalf("Error listing pods: %v", err)
	}
	
	if len(pods.Items) != 1 {
		t.Errorf("Expected 1 pod, got: %d", len(pods.Items))
	}
}

Testing Error Scenarios

Let’s add a test case for error handling by adding a reactor to the fake clientset:

func TestCreatePodError(t *testing.T) {
	// Create a fake clientset
	clientset := fake.NewSimpleClientset()
	
	// Add a reactor to simulate an API error
	clientset.PrependReactor("create", "pods", func(action k8stesting.Action) (bool, runtime.Object, error) {
		return true, nil, fmt.Errorf("simulated API error")
	})
	
	// Initialize our client with the fake clientset
	client := client.Client{
		Clientset: clientset,
	}

	// Define a test pod
	testPod := &v1.Pod{
		ObjectMeta: metav1.ObjectMeta{
			Name:      "test-pod",
			Namespace: "default",
		},
		Spec: v1.PodSpec{
			Containers: []v1.Container{
				{
					Name:  "nginx",
					Image: "nginx",
				},
			},
		},
	}

	// Try to create the pod
	_, err := client.CreatePod(testPod)
	
	// Verify we got the expected error
	if err == nil {
		t.Fatal("Expected error but got nil")
	}
	
	if err.Error() != "simulated API error" {
		t.Errorf("Expected 'simulated API error', got: %s", err.Error())
	}
}

Understanding Fake vs. Real Clientsets

Real Clientset (NewForConfig)

  • Connects to an actual Kubernetes cluster
  • Requires authentication and proper permissions
  • Operations affect real resources in the cluster
  • Network latency and potential failures
  • Complex setup for CI/CD environments

Fake Clientset (NewSimpleClientset)

  • In-memory implementation with no external dependencies
  • No actual cluster connection required
  • Operations only affect an in-memory object store
  • Extremely fast test execution
  • Can simulate various API responses and errors
  • Perfect for unit testing without infrastructure

Best Practices for Testing Kubernetes Client Code

  1. Use Dependency Injection: Pass the kubernetes.Interface rather than concrete implementations to allow for testing with fakes.

  2. Test Error Handling: Use reactors to simulate API errors and ensure your code handles them gracefully.

  3. Validate Side Effects: After operations, check that the expected resources were created/updated in the fake clientset.

  4. Test Resource Interactions: If your code interacts with multiple resources, test those interactions.

  5. Separate Unit and Integration Tests: Use fake clients for unit tests and real clients (with test clusters) for integration tests.

  6. Use Table-Driven Tests: For testing multiple scenarios with different inputs.

  7. Mock the Watch API: For controllers or operators that use watchers, use the fake client’s watch functionality.

// Example of setting up a watch reactor
fakeClient.PrependWatchReactor("pods", func(action k8stesting.Action) (bool, watch.Interface, error) {
    watcher := watch.NewFake()
    // Simulate events
    go func() {
        watcher.Add(testPod)
        // Add more events as needed
    }()
    return true, watcher, nil
})

Running the Tests

You can run these tests using the standard Go testing tools:

go test

Output:

I0520 00:02:03.789351   23893 pod.go:23] Pod test-pod is successfully created
PASS
ok      github.com/kubernetes-sdk-for-go-101    0.681s

Conclusion

Unit testing Kubernetes client code is essential for ensuring reliability and correctness before deploying to production clusters. The fake clientset provides a powerful way to test your code without requiring access to a real Kubernetes cluster, enabling fast and reliable tests.

This approach is particularly valuable in CI/CD pipelines where you want to validate your code without needing to provision test clusters. By combining these unit tests with integration tests against real clusters, you can achieve comprehensive test coverage for your Kubernetes applications.