Skip to content
Sign Up
All posts

Dynamically Scaling Kubernetes Infra using KEDA and Karpenter

Picture of Jasim Jamal By  Jasim Jamal  ·  5 minute read

Introduction

At PerfAgents, we understand the criticality of making our monitoring infrastructure robust, scalable, and reliable.

We are glad to share how we leverage the efficacies of

  • Kubernetes
  • Keda and
  • Karpenter to

deliver a scalable and cost-effective synthetic monitoring infrastructure for our customers.

image

What does PerfAgents do?

We specialize in synthetic monitoring solutions designed to empower businesses across various industries with actionable insights into their digital assets

  • Performance and
  • Availability

PerfAgents Uncloud is our SaaS-based Cloud-Native platform that allows users to schedule and run synthetic monitoring tests from various geographical regions within AWS and other major cloud providers.

There are 3 types of synthetic monitoring available through the platform:

 

How does PerfAgents work?

PerfAgents Uncloud mimics real-world user behavior to identify bottlenecks and potential issues. It sends scripted requests to your applications or endpoints based on a configurable time interval.

Under the hood, PerfAgents will need to respond to varying user configurations.

That translates to a demand for managing dynamic workloads on the underlying infrastructure. The number of monitoring jobs can fluctuate significantly based on customer needs.

The Challenge: Unpredictable Workloads and Cost Control

Synthetic monitoring often involves fluctuating workloads.

Users may schedule tests that produce bursts of activity followed by periods of lower demand.

Traditional scaling approaches, such as relying on statically provisioned infrastructure, pose two main challenges:

  • Over-provisioning: Statically sized infrastructure often leads to over-provisioning, where you pay for resources you're not fully utilizing.
  • Under-provisioning: On the flip side, under-provisioning during demand spikes can cause slow test execution, delayed results, or even failures, impacting customer experience.

To address these challenges, we needed a highly dynamic scaling solution that could rapidly adapt to workload changes while controlling costs.

We run microservices at scale, and one of our underlying compute workloads is Kubernetes, fondly known as k8s.

But plain Kubernetes is insufficient to manage the complex scaling process within PerfAgents. KEDA and Karpenter were added to the mix, and the results were incredible.

You can read further to learn how KEDA and Karpenter helped us keep up with scaling demands and keep cloud costs under control.

 

Why Kubernetes, KEDA, and Karpenter?

An overview of why this technology stack was an excellent fit for our use case:

Here is a quick overview of Kubernetes:

  • Container Orchestration: Simplifies deployment and management of containerized applications.
  • Scaling and Resilience: Provides features like auto-scaling, self-healing, and rolling updates.
  • Abstraction: Allows focus on core services rather than infrastructure complexities.

Here is a quick overview of KEDA:

  • Workload-Centric Scaling: Scales based on events like queue length, custom metrics, etc.
  • Flexibility: Supports various event sources for customized scaling triggers.
  • Efficiency: Optimizes resource usage by aligning capacity with real-time demand.

Here is a quick overview of Karpenter:

  • Cost Optimization: Intelligently provisions nodes on-demand, using Spot Instances when possible.
  • Workload-Matching: Selects the optimal instance types based on pod requirements.
  • Simplified Management: Eliminates the need for predefined node pools.

 

Kubernetes: The Foundation for Orchestration

Kubernetes has emerged as the go-to standard for container orchestration in modern cloud-native environments.

At PerfAgents, AWS is our primary cloud provider. Even though we use other cloud providers, AWS runs some of the critical workloads within PerfAgents.

At PerfAgents, we harness the power of Kubernetes to orchestrate and automate our monitoring infrastructure effectively. Amazon Elastic Kubernetes Service (EKS) is a no-brainer solution for running your container workloads in AWS.

As a fully managed Kubernetes service, EKS offers unparalleled scalability, flexibility, and reliability for deploying and managing large-scale containerized applications.

By containerizing our services and deploying them within Kubernetes clusters, we can achieve a high degree of agility and resilience while simplifying resource management and scaling operations.

Kubernetes enables us to abstract away the underlying infrastructure complexities and focus on delivering value-added services to our clients.

Kubernetes is beneficial for building robust, cloud-native applications that can adapt seamlessly to fluctuating workloads and evolving business requirements and the following features of Kubernetes really stand out:

  • Auto-Scaling
  • Security
  • Service Discovery
  • Rolling Updates

 

KEDA (Kubernetes Event-Driven Autoscaling): Scaling Based on Events, Not Just Metrics

While Kubernetes provides native support for Horizontal Pod Autoscaling (HPA) based on CPU and memory metrics, certain workloads exhibit dynamic scaling patterns driven by external events or custom metrics.

Traditional Kubernetes autoscaling based on CPU or memory usage isn't ideal for our use case.

We need to scale based on the number of monitoring jobs – an event-driven approach. This is where KEDA comes in.

KEDA integrates with various event sources, including

  • Message Queues
  • Databases and
  • Custom Triggers

unlocking a whole new world of workload scaling possibilities within Kubernetes.

In our case, KEDA extends our Kubernetes cluster to scale based on the number of jobs placed in the queue.

As the number of jobs in the queue increases, KEDA automatically scales our monitoring service deployments up, ensuring the timely processing of all jobs.

Conversely, KEDA scales deployments down during low-traffic periods, optimizing resource utilization. This allows us to align compute capacity with actual demand seamlessly.

By leveraging KEDA, we can ensure that our monitoring infrastructure scales elastically in response to fluctuating workloads and incoming events, thereby optimizing resource utilization and enhancing overall system efficiency.

Whether it's handling sudden traffic spikes or processing large volumes of monitoring data, KEDA enables us to maintain optimal performance without over-provisioning resources unnecessarily.

 

Karpenter: The Right Node for the Job

While scalability and performance are paramount considerations in any cloud-native environment, cost optimization remains a critical factor for businesses looking to maximize ROI and minimize operational expenses.

We also need to ensure the right compute resources to handle the workload effectively.

This is where Karpenter shines.

Karpenter is a Dynamic Kubernetes Node Autoscaler!

Karpeneter helps in the following manner:

  1. It takes a workload-centric approach, unlike traditional auto scalers that rely on predefined node pools.
  2. Karpenter leverages AWS Spot Instances and on-demand capacity to provision nodes on demand based on the resource requirements of pending pods.
  3. It considers factors like CPU and memory needs to intelligently select the optimal instance type from the AWS cloud provider for running our monitoring workloads.
  4. This eliminates the need for manual configuration and ensures we're not paying for more resources than necessary.

 

The PerfAgents Advantage: Scalability Meets Cost-Efficiency

The combination of Kubernetes, KEDA, and Karpenter allows PerfAgents to achieve several key benefits:

  • Automatic Scaling: Our infrastructure scales automatically based on real-time job demands, ensuring optimal performance and efficient resource utilization.
  • Reduced Costs: Karpenter's dynamic provisioning ensures workload-centric allocation, and we only pay for the resources that we use, minimizing cloud costs.
  • Flexibility: KEDA supports various event sources, allowing us to adapt to future monitoring needs.
  • Simplified Management: By automating scaling and provisioning, we free up our engineering team to focus on core development tasks.

 

Conclusion

PerfAgents' infrastructure architecture exemplifies our commitment to delivering scalable, reliable, and cost-effective synthetic monitoring solutions to our clients.

By harnessing the power of Kubernetes, KEDA, and Karpenter, we have built a robust platform that can adapt to dynamic workloads, scale elastically, and optimize resource utilization in real time.

Through continuous innovation and optimization, we strive to remain at the forefront of the synthetic monitoring industry, empowering businesses worldwide with actionable insights and unparalleled visibility into their digital assets' performance and availability.

As technology evolves and customer expectations rise, we remain dedicated to pushing the boundaries of what's possible and driving positive outcomes for our clients and partners.

Ready to explore the potential of event-driven scaling and dynamic provisioning for your workloads?

We encourage you to dive deeper into Kubernetes, KEDA, and Karpenter. Your applications (and your wallet) will thank you!

Thank you for joining us on this journey towards excellence in synthetic monitoring.

Stay tuned for more insights, updates, and innovations from the PerfAgents team as we continue to redefine the future of digital performance monitoring.

To learn more about PerfAgents, visit https://www.perfagents.com