Field Notes From KubeCon #4: Ensuring Reliability of Production-Ready Kubernetes Operators Using EnvTest

Field Notes from KubeCon India 2024 on Reliability of Production-Ready Kubernetes Operators

Hi Ishan here again! Delivering you the field notes I took from all the sessions I attended during KubeCon Day 1 and 2 — so you don’t miss a thing!

Field Notes From KubeCon #4: Ensuring Reliability of Production-Ready Kubernetes Operators Using EnvTest

After grabbing a quick coffee, I headed to the session led by Aniruddha Basak and Rayan Das. The title alone promised a deep dive into Kubernetes operators—a crucial area for anyone working with cloud-native tech.

The Role of Kubernetes Operators

Aniruddha kicked things off by explaining what a Kubernetes operator is. "Think of it as automating what a human operator would do to deploy and maintain complex applications," he said. Kubernetes operators manage applications using custom resources and controllers.

Rayan added, "At its core, a controller watches the cluster’s desired state, compares it to the current state, and reconciles the differences.” This constant feedback loop ensures that Kubernetes systems stay healthy.

Their example? A Kubernetes operator managing external secrets by automatically syncing sensitive data—a real-world use case that immediately clicked with the audience.

Testing Challenges with Kubernetes Operators

They didn’t sugarcoat the struggles of testing Kubernetes operators. Traditional testing methods—unit, integration, and end-to-end (E2E)—each have pros and cons. Rayan shared candidly, "Most people hate writing E2E tests because they are slow, flaky, and a nightmare to maintain.”

Aniruddha nodded. "They often use fake Kubernetes clients that don't behave like real clusters, leading to blind spots in production." The limitations were clear.

Enter EnvTest: A Game Changer

To overcome these hurdles, they introduced EnvTest, a Go library designed for testing Kubernetes controllers. EnvTest spins up lightweight Kubernetes API components like etcd and the Kubernetes API server, allowing realistic tests without spinning up a full cluster.

"It’s like getting the benefits of integration testing without the usual overhead," Aniruddha explained. Rayan shared a live demo, showing how tests could run faster while still interacting with real Kubernetes components.

Best Practices for EnvTest

They wrapped up the session with actionable best practices:

  • Use Correct CRD Paths: Ensure Custom Resource Definitions are specified properly.
  • Use Cleanup Hooks: Clean up test environments after each run.
  • Isolate Tests: Create unique namespaces dynamically to avoid conflicts.
  • Mock External Dependencies: Use libraries like gomock to mock services like cloud providers.

Final Thoughts

This session was a goldmine for anyone working with Kubernetes operators. It made me rethink how we approach testing at Facets.cloud. Using EnvTest could help us scale faster and reduce bugs in production.

As I left the room, I couldn’t help but appreciate how deep dives like this push the entire Kubernetes ecosystem forward. Looking forward to sharing more from KubeCon in the next installment!