The Kubernetes Singularity

In combination with other trends, emergence of generative AI (especially that of open source generative models) in the past year is once again changing the definition and economics of cloud. And organizational IT strategy and tactics–as always–are pressed to keep up.

Could Kubernetes realize the technological singularity?

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

While the concept of digital transformation has been around for awhile and is aging as a buzzword, the actual process of transformation is accelerating at many organizations. Business leadership can now taste some of the ROI and benefits. They’re excited. Especially about AI–which, unlike a lot of deep-in-the-weeds tech innovation, they are reading about and using every day–creating FOMO and urgency.

At the same time (no surprise to most reading this), the reality of digital transformation is way more complicated than most imagined. It seems the goal is always simplicity, but building and staffing becomes reality around growing complexity … and incurs mounting costs. This is now (if still slowly) being recognized as an existential threat.

Long-standing assumptions are being questioned:

• Public cloud will end up being cheaper and operationally simpler than on-premises cloud. Not true for some use cases. Debatable for some others. While “repatriation” isn’t quite the mega-trend some claimed it would be a year or so ago, the strategy of many organizations requires a mix of public and private cloud and platforms that support containers and Kubernetes everywhere (as well as conventional workloads on VMs and/or bare metal).
• Progress to multi-cloud/cluster, hybrid and edge/IoT will be simple and non-disruptive–and existing platforms (both open source and proprietary) will evolve smoothly to comprehend these architectures. Not true: Demand for all these cloud topologies is increasing, but solutions are still complicated, fragile and involve lock-in.
• Historically ‘safe’ choices will remain safe. Not true: Major acquisitions and other changes are reshaping the priorities of big legacy proprietary private cloud vendors, and some customers are looking for the (open source) exits.

At an even more basic level, organizations are questioning whether the classic deep cloud stack (IaaS, Kubernetes, CaaS/PaaS/serverless, etc.) or variegated public cloud multi-service architecture–all those different APIs and CLIs–is efficient to work with. There’s growing conviction that Kubernetes provides a better abstraction for everything “cloud-related,” and potentially better (easier, more consistent, more interoperable) ways of working for platform engineers, DevOps and developers.

Some of the most intriguing things you’ll see at KubeCon + CloudNativeCon Europe 2024 proceed from the assumption that:

• Every kind of workload—meaning containers, microservices, functions-as-a-service (FaaS), serverless and containers-as-a-service (CaaS), platform-as-a-service (PaaS), plus WASM, VM workloads, etc.—should ‘run on Kubernetes.’
• Many PaaS solutions are too ‘heavy’ (i.e., classic PaaS provides redundant abstractions that don’t reduce platform complexity from the point of view of developers) and public cloud FaaS is too inflexible. Use of either leads to lock-in, and Kubernetes can provide these services natively with the help of open source service frameworks that ‘simplify.’
• Kubernetes (and a stack of Kubernetes-native technologies and concepts like Cluster API, cloud platform providers and Kubernetes operators) should be used to marshal infrastructure, ‘hiding’ the IaaS layer and bare metal management.

The implication is that Kubernetes abstractions should be what everyone works with, day-to-day–one set of tools, one infrastructure-as-code (IaC) paradigm.

Restate the Problem to Find the Answer

Where is this all going? Views of contemporary IT issues–in the recent past, conceptualized as global questions in which Kubernetes would form just one part of a complicated solution–are now being rethought as Kubernetes-centric challenges:

• ‘How do we build hybrid clouds?’ becomes ‘How do we configure Kubernetes to present a uniform, continuous platform across commodified on-premises IaaS (or Kubernetes) and public cloud IaaS (or Kubernetes)?’
• ‘How do we do multi-cluster?’ stops being ‘How can we deploy lots of Kubernetes clusters on IaaS or wherever and manage them?’ and becomes ‘How can we configure Kubernetes to let us create, share and manage clusters efficiently on Kubernetes?’
• ‘How do we do edge and IoT?’ stops being ‘How do we integrate an arcane set of tools or buy into a proprietary public cloud model in order to centralize management while distributing compute/sensing across remote hardware in thousands of locations and/or tiny mobile devices?’ And starts being ‘How do we configure Kubernetes workloads that run on anything and attach them to centralized control planes from anywhere?’ and ‘How can we use Kubernetes and open source applications to empower functions at the edge–and efficiently train, improve, deploy and manage machine learning at distributed endpoints to manage vision and other tasks at the edge?’
• And of course, how do we do all this and create a home for large-scale AI?

The Kubernetes community is now actively building technologies to meet all these challenges. And the result may be new fronts for collaboration–for example, around concepts like Hosted Control Planes and fast-evolving sub-projects like Konnectivity and Cluster API.

If Kubernetes Becomes “The Cloud,” Then What?

If Kubernetes is becoming ‘the cloud’ (or, more accurately, ‘the underlying glue that makes the cloud work’), this also has implications.

Kubectl and YAML and kubeconfigs don’t scale well to large orgs and varying skill sets. So you need a standard environment (applications, tools, services) that helps platform engineers, DevOps and developers use Kubernetes with less friction:

• Providing views and tools that let experts deal efficiently with Kubernetes infrastructure and platform internals
• Extending to many contexts–wrapping user interfaces around Kubernetes system components and services, critical operators (e.g., security, updates management), and other complex technology like CI/CD.
• Simplifying application development, management and operations–providing constrained views of an application namespace, metadata, logs, metrics, security scans, policies and other information.
• Facilitating work at scale: Coordinating secure access to clusters, applications, teams and supporting collaboration

What About AI on Kubernetes?

AI, meanwhile (obviously) has thrown a huge cinderblock into the swimming pool. While the future (as author William Gibson has been quoted as saying) is unevenly distributed, it’s here now. Everyone in tech knows that AI is going to change everything about how organizations build, buy and innovate with tech to compete and gain business benefits.

AI is now making old models for software development—and thus, for digital transformation—obsolete. For many organizations, the big project is now (or will soon be) leveraging AI to help everyone run the business in a tighter and tighter optimization loop with real metrics. Actually using the  dashboards instead of spending so much time and resources building them.

In the middle of all this sits Kubernetes, which, for us in the Kubernetes community, is very exciting and a little scary.

While the Azure supercomputing platforms used to train and run OpenAI’s big models are VM-oriented, Kubernetes is pretty clearly where a lot of enterprise AI is going to be developed and deployed. This is particularly true for enterprise AI that leverages pre-trained open source models–easy to distribute in containerized form–alongside vector databases in so-called Retrieval-Augmented Generation (RAG) implementations. (RAG is a way of avoiding the need to build and operate ‘training runs’–the pre-trained model gives good language performance, while the vector database provides use case-specific ‘knowledge,’ perhaps proprietary, in a form that can remain private.)

How to host and operate RAG setups efficiently on Kubernetes (and other emerging enterprise AI paradigms) is probably something that needs standardization and open source options.

Cloud-Native Collaboration

With all these things happening at once, it’s no exaggeration to say that technology is approaching (or may already have reached) a singularity–with Kubernetes, CNCF and the cloud-native open source ecosystem at the center.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.