Upbound’s New Control Plane Is Purpose-Built for the Chaos of AI Inference

Think of a scenario wherein an organization is using several open-weight models in multiple clouds along with a rack of on-premises GPUs. In addition, each team uses a different serving engine and it is not clear which cluster has any capacity left for new inference requests. All of this is no longer a hypothetic situation since it describes the state of affairs when an enterprise goes beyond merely running a chatbot demo and begins performing serious inference at scale.

However, the Upbound company has found a solution to this issue. The latter has introduced Modelplane, a new open-source control plane that is able to deal with AI inference and deliver models to many GPU clusters, irrespective of whether the clusters are located in the cloud, the neocloud, or on-premises.

In this article, we will unpack what Modelplane actually does, how it fits into the tools teams already use, and why Upbound believes this problem needs a whole new layer of infrastructure.

Why inference outgrows a single cluster fast

Kubernetes is great at running workloads on one cluster. But inference does not stay put. As soon as a team adopts open-weight models, they end up juggling multiple GPU types, multiple regions, and multiple serving stacks like vLLM or SGLang. Cluster-level tools were never designed to answer a simple question, which is where across the entire fleet should this model actually run.

What Modelplane brings to the table

Modelplane is operated as an external element of the cluster, and not as an intermediary element. It does not take over the function of an inference engine, schedule, gateway, but rather serves as a coordination tool. It performs fleet-wide model deployment; through it, the company makes the decision about the modeling infrastructure depending on the available GPUs’ capacity in the cloud and at different geographical locations. 

It is also responsible for the auto-scaling of replicas according to the requirements for the traffic and customers’ costs, management of the cluster’s and accelerators’ lifecycle, delivery of the model parameters to the pods without additional file downloading due to caching, and traffic distribution via one single socalled OpenAi-compatible gateway with canary rollout options and rolling back to the managed providers.

Built on familiar ground

This is where Crossplane comes in. Modelplane is built on top of it, which means teams already using Crossplane to manage cloud infrastructure get a familiar declarative model here too. Scaling works through the standard Kubernetes scale subresource, so kubectl scale and a KEDA ScaledObject work without extra glue code. Upbound CEO Bassam Tabbara has framed it simply, saying inference never really stays on one cluster, and Modelplane exists to treat the whole fleet as a single system instead of a pile of separate ones.

Who actually needs this

Modelplane serves organizations looking to have their own inference capability more than just depending on a managed API. Think of AI-led companies spending a lot on inference, enterprises governed by regulations that care about data sovereignty, and neoclouds that provide compute power on GPUs. The software is currently in developer version v0.1 and is being released under an Apache 2.0 license, with Upbound planning to donate it to a non-partisan open-source organization soon.

If your GPU fleet is handling your operations like a spreadsheet rather than being well-organized enough as it should, then Modelplane can be a good option for you. To check if it suits your needs, visit the project on GitHub, check out the documentation available at docs.modelplane.ai.