CAST AI Extends Reach of Kubernetes Automation Platform

CAST AI announced today at the KubeCon + CloudNativeCon 2024 conference that it has added a Container Live Migration capability that enables the movement and resizing of stateful workloads across Kubernetes clusters.

Additionally, the company is making generally available AI Enabler, a tool that enables IT teams to direct queries to the most cost-effective large language model (LLM).

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

Initially launched earlier this year, AI Optimizer leverages an LLM Router CAST AI provides, to analyze users and their associated application programming interface (API) keys, overall usage patterns, and the balance of input versus output tokens to determine which instance of a graphical processor unit (GPU) in the cloud is running an LLM most efficiently.

The CAST AI Playground, a testing resource to compare LLM performance and cost, further makes it possible to benchmark and customize configurations.

Laurent Gil, chief product officer for CAST AI, said both of these latest additions extend the capabilities of a platform that makes use of machine learning algorithms and other forms of AI to automatically rightsize Kubernetes environments.

AI Enabler addresses a critical issue that has arisen as more AI models are deployed on Kubernetes clusters, added Gil. Many of the decisions concerning which LLM to deploy have been based more on personal preference and general accessibility than cost and performance concerns, he added.

Similarly, the Container Live Migration capability that CAST AI has developed reduces costs by giving IT operations teams more control over how applications are deployed. While containers are ripped and replaced all the time, containers running stateful workloads could not previously be combined. The Contain Live Migration capability can not only be used to facilitate backup and recovery without incurring any downtime but also enables IT teams to resize containers as the overall application environments grow and expand, said Gil.

Container Live Migration, in effect, brings a capability that is widely employed in virtual machine environments into the realm of containers, he added.

It’s not clear how IT teams are optimizing the management of what is often becoming fleets of Kubernetes clusters, but as organizations become more sensitive to cost there is a pressing need to optimize those environments. In theory, Kubernetes environments are supposed to scale up and down as application requirements change. In practice, developers are more concerned about ensuring application availability than cost, so there is a natural tendency to overprovision IT infrastructure.

As the number of Kubernetes clusters deployed in production environments continues to increase steadily, more attention is starting to be paid to cost optimization. IT teams are under considerable pressure to rein in costs in an uncertain economic environment, noted Gil. A recent CAST AI analysis found that, on average, IT teams are only using 13% of provisioned CPUs and 20% of memory when deploying clusters that have 50 to 1,000 processors.

Many of those IT teams are, as a result, embracing platform engineering as a methodology for applying best DevOps practices at scale to rein in those costs. There would also be increased adoption of Kubernetes in general if the platform was easier to manage cost-effectively. The only way to close that gap, of course, is to use platforms such as CAST AI to automate as many routine management tasks as possible, based on the level of performance that needs to be attained and maintained within a defined IT budget.