Open Source Kubernetes Cluster Management Tools and Approaches¶

"Managing Kubernetes clusters" involves at least six distinct concerns: access and troubleshooting, cluster creation and upgrades, unified identity and permissions, configuration convergence, cross-cluster application distribution, and cross-cluster workload placement. No single tool should be the sole controller for all six layers at once.

Define the problem first, then choose the minimal combination. For most teams, a reliable command-line entry point, a read-only-first Web UI, GitOps, and clear cluster lifecycle tooling are easier to maintain long-term than installing an overlapping "all-in-one platform."

1. First, Separate the Six Capability Layers¶

These layers can be combined, but the sole owner of each object and field must be defined. For example, do not let Argo CD, Fleet and Karmada all modify the replica count of the same Deployment, and do not let Rancher, Cluster API and manual scripts all be responsible for upgrading the same cluster.

2. Approach One: Native CLI Plus Lightweight UI¶

This is the simplest and most transparent management approach:

OIDC / short-term credentials
      │
kubeconfig exec plugin
      │
kubectl / k9s / Headlamp
      │
Kubernetes API + RBAC + Audit

OIDC / 短期凭据
      │
kubeconfig exec plugin
      │
kubectl / k9s / Headlamp
      │
Kubernetes API + RBAC + Audit

kubectl, kubectx and k9s¶

• kubectl is the foundation for automation and troubleshooting and should be retained as the final diagnostic entry point;
• kubectx/kubens are suitable for quickly switching clusters and Namespaces;
• k9s is suitable for real-time resource browsing, logs, events and common operations in the terminal.

This approach does not provide cluster creation, a unified tenant portal, or continuous configuration convergence. As the number of clusters grows, static administrator credentials should be replaced with OIDC, exec credential plugins, or controlled access proxies. Do not store dozens of cluster-admin kubeconfig files on a personal computer long-term.

Headlamp¶

Headlamp is an open source Web/desktop UI under Kubernetes SIG UI. It can connect to one or more clusters, and its interface operations follow the RBAC of the user in the target cluster, with plugin extension support. It is suitable for:

• Providing developers and front-line operators with an entry point for resources, logs and events;
• Adding a lightweight UI to existing authentication, GitOps or internal platforms;
• Unifying basic operational experience without introducing a full cluster management platform.

Headlamp does not create clusters, nor does it automatically establish organization-level tenants, policies and delivery systems. The Web version still requires OIDC, Ingress, TLS, NetworkPolicy, auditing and least-privilege configuration.

Kubernetes Dashboard has been officially marked as deprecated and no longer maintained. New deployments should prioritize evaluating Headlamp rather than continuing to build portals around Dashboard.

References: Headlamp, Kubernetes Web UI

3. Approach Two: GitOps for Cluster Configuration Management¶

GitOps is not responsible for creating physical machines or repairing the control plane, but it is well suited to continuously converging multiple clusters toward an auditable desired state:

Git / OCI Artifact
        │
Review, test, sign, and version
        │
Argo CD / Flux / Fleet
        │
Base components, policies, AI operators, and business applications

Git / OCI Artifact
        │
评审、测试、签名和版本
        │
Argo CD / Flux / Fleet
        │
基础组件、策略、AI Operator 和业务应用

Argo CD¶

Argo CD provides a centralized control plane, Web UI, application health status, Diff, sync and rollback, and can register multiple target clusters. It is suitable for teams that want to view multi-cluster application status from a single entry point and adopt App-of-Apps or ApplicationSet.

A centralized Argo CD holds target cluster credentials and requires careful design of project boundaries, target Namespaces, credential rotation, high availability and controller sharding. It manages declarative resources in clusters and is not equivalent to cluster lifecycle management.

Flux¶

Flux consists of a set of Kubernetes controllers that typically pull desired state from Git, OCI, Helm or Bucket sources inside each cluster. It is suitable for teams that prefer a distributed Pull model, Kubernetes-native CRDs, smaller management failure domains, and per-cluster autonomy.

Flux itself does not provide an official UI; it can be paired with Headlamp plugins or other ecosystem interfaces. Regardless of whether Argo CD or Flux is chosen, repository hierarchy, Secret encryption, change promotion, emergency fix write-back and drift handling rules should be defined first.

References: Argo CD Cluster Management, Flux Core Concepts

4. Approach Three: Unified Portal and Multi-Cluster Governance Platform¶

Rancher¶

Rancher is a relatively complete open source Kubernetes management platform. It can register existing clusters, create and upgrade certain types of clusters, and provides centralized identity, RBAC, projects, cluster browsing, application catalogs and Fleet continuous delivery.

Suitable for:

• Managing RKE2/K3s, cloud-managed and other existing Kubernetes clusters simultaneously;
• Teams that need unified login, permissions and a team-facing Web portal;
• Teams that want cluster lifecycle, access entry point and Fleet integration in one product.

Note: Registering a cluster into Rancher does not mean Rancher gains full lifecycle control over all distributions; the upgrade, node and backup operations available differ by cluster type. In production, the Rancher Server should typically be deployed on a separate, highly available management cluster to avoid a business cluster failure also taking down the management entry point.

References: Rancher Manager, Rancher Cluster Management

KubeSphere¶

KubeSphere provides platform-team-oriented multi-tenant workspaces, projects, application delivery, observability and multi-cluster management interfaces. It is suitable for teams that want a Chinese-friendly portal to integrate development and operations processes and are willing to adopt its platform abstractions.

When selecting, verify the extension components, Kubernetes compatibility matrix and community maintenance cadence of the target version. KubeSphere abstractions such as Workspace and Project need to be aligned with native Namespace, RBAC and GitOps ownership to avoid forming a second permission and delivery model that no one maintains.

References: KubeSphere Multi-cluster Management

Portainer CE¶

Portainer CE can manage Kubernetes, Docker and other environments from a lightweight interface, suitable for small teams, labs and scenarios with existing Portainer experience. The Server paired with Agents can connect to multiple environments, but the feature boundaries between the Community Edition and Business Edition differ, and Portainer Server high availability and multi-instance management of the same environment have architectural limitations.

If the goal is a large-scale Kubernetes Fleet, strict multi-tenancy or full lifecycle automation, Portainer should not be chosen by default just because it is easy to install.

References: Portainer CE on Kubernetes, Portainer Architecture

5. Approach Four: Declarative Cluster Lifecycle¶

kubeadm, Kubespray and kOps¶

These three tools are closer to installers or operations automation than to unified multi-cluster portals:

• kubeadm provides init, join, certificate and upgrade cluster bootstrapping capabilities. It does not create machines, nor does it install the full networking, storage, monitoring and management platform;
• Kubespray uses Ansible to automate host preparation, kubeadm, high availability, network plugins, node scaling and upgrade processes, suitable for bare metal, virtual machines and offline environments;
• kOps manages both Kubernetes and the necessary cloud infrastructure, suitable for its officially supported cloud environments, but capability maturity must be verified against the target Provider.

For a small number of self-built clusters, Terraform can create hosts, load balancers and networking, and Kubespray can then install Kubernetes; infrastructure state and in-cluster resource state should still be managed separately. As the number of clusters grows, imperative pipelines and Ansible Inventories easily become de facto databases of truth; at that point, teams can shift to continuously reconciling declarative controllers such as Cluster API.

References: kubeadm, Kubespray, kOps

Cluster API¶

Cluster API (CAPI) uses Kubernetes-style Cluster, Machine, MachineDeployment APIs to manage cluster creation, scaling, upgrades and deletion, and integrates with public cloud, private cloud or bare metal through different Infrastructure, Bootstrap and Control Plane Providers.

It is suitable for platform teams building "cluster as API" and incorporating environment templates, machine pools and upgrade processes into GitOps. Note that:

• Provider maturity, features and version cadence are not fully consistent;
• CAPI is not a daily resource management UI and does not distribute business applications;
• Existing clusters not created by CAPI generally cannot simply gain full CAPI lifecycle management;
• Backup, upgrade and disaster recovery of the management cluster itself still need separate design.

References: Cluster API Book

Gardener¶

Gardener uses Kubernetes to manage Kubernetes, providing multi-cloud Kubernetes as a Service through the Garden, Seed and Shoot hierarchy. It covers cluster creation, control plane hosting, version and machine image policies, maintenance windows and lifecycle coordination, suitable for platform providers operating hundreds to thousands of relatively standardized clusters.

Its capabilities and operational complexity are significantly higher than "installing a management UI." Teams need to master management clusters, Seed capacity, cloud extensions, networking, backup and Gardener's own upgrades. It is not suitable as a simple console for a small number of existing clusters.

References: Gardener Introduction, Gardener Architecture

Kubermatic Kubernetes Platform¶

Kubermatic Kubernetes Platform (KKP) provides multi-cloud cluster creation, templates, projects, OIDC and lifecycle management. The Community Edition can serve as a starting point for a self-service Kubernetes platform, but some backup, multi-Seed, quota, policy and edge capabilities belong to the Enterprise Edition.

Selection should not only look at whether the code repository is open; the features actually needed should be checked item by item against the version and Edition to verify that a fully open source deployment can form a production closed loop.

References: KKP Community and Enterprise Editions

6. Approach Five: Multi-Cluster Manifest, Policy and Application Distribution¶

Open Cluster Management¶

Open Cluster Management (OCM) uses a Hub-Agent architecture to manage ManagedCluster manifests, and builds registration, placement, work distribution and extension capabilities through ManagedClusterSet, Placement, ManifestWork and Add-ons. It is suitable for:

• Teams that need a composable set of multi-cluster APIs rather than a fixed product portal;
• Network environments where member clusters can only actively connect to the Hub;
• Teams that want to develop an internal Fleet Manager based on Cluster Claims, Placement and Add-ons.

OCM is more like a collection of building blocks for a multi-cluster control plane. Open source OCM and commercial products built on top of it differ in installation experience, policy packages, UI and support scope.

References: OCM Concepts, OCM Placement

Karmada¶

Karmada provides an independent multi-cluster API Server, cluster registration, PropagationPolicy, Override, placement, resource status aggregation and failover capabilities. It is more suitable for cross-region application replicas, environment differential overrides and policy-based distribution.

Karmada does not create underlying Kubernetes clusters, is not a cross-cluster CNI, and does not merge GPUs from multiple clusters into a single node-level scheduling domain. Whether custom AI CRDs can correctly estimate resources, propagate dependencies and aggregate status also depends on the Resource Interpreter and the capabilities of the target version.

References: Karmada Concepts, Cross-Cluster and Large-Scale GPU

7. Open Source Tool Comparison¶

"Strong" in the table only indicates the direction of core capability, not that high availability, security, audit, and disaster recovery requirements are automatically met after installation.

8. Recommended combinations for AI/GPU clusters¶

Small number of GPU clusters¶

Cloud-managed Kubernetes / kubeadm / RKE2
  + OIDC, kubectl, Headlamp or k9s
  + Argo CD or Flux
  + GPU Operator, Kueue, monitoring and object storage

云托管 Kubernetes / kubeadm / RKE2
  + OIDC、kubectl、Headlamp 或 k9s
  + Argo CD 或 Flux
  + GPU Operator、Kueue、监控和对象存储

First stabilize node acceptance, driver matrix, queues, image/model distribution, and observability. Don't introduce a federation control plane prematurely just because you might have multiple clusters in the future.

Multi-team, multi-environment GPU platform¶

Rancher or KubeSphere: unified entry point, teams and access
GitOps: GPU/Network Operator, Kueue, KubeRay, Trainer, KServe
Cluster API or existing IaaS: cluster and GPU node pool lifecycle

Rancher 或 KubeSphere：统一入口、团队和访问
GitOps：GPU/Network Operator、Kueue、KubeRay、Trainer、KServe
Cluster API 或既有 IaaS：集群和 GPU 节点池生命周期

The platform portal handles identity and experience, Git is the authoritative source of configuration, and the lifecycle controller handles machines and versions. The three must avoid modifying the same fields.

Multi-region training and inference Fleet¶

Cluster API / Gardener / cloud API: create and upgrade clusters
Argo CD / Flux: baseline components and runtime convergence
Karmada / OCM: applications, policies and regional placement
MultiKueue: dispatch complete training jobs to an available cluster
Gateway / DNS: online inference traffic switching

Cluster API / Gardener / 云 API：创建和升级集群
Argo CD / Flux：基线组件与运行时收敛
Karmada / OCM：应用、策略和区域放置
MultiKueue：把完整训练 Job 派发到一个可用集群
Gateway / DNS：在线推理流量切换

The cross-cluster management layer should maintain stable capability labels, such as GPU vendor and model, RDMA, region, compliance domain, storage, and model cache status. Real available capacity and GPU/NVLink/RDMA topology are still determined by Kueue, Volcano, kube-scheduler, and the device management layer of member clusters.

Don't let a distributed training Rank span ordinary Kubernetes clusters by default. Cross-cluster network latency, NCCL/RDMA, Gang, Checkpoint, and failure semantics must all be specifically designed; the common approach is still to place a complete training job into a single cluster.

9. Selection recommendations¶

If the team still can't articulate whether the current gap is UI, identity, lifecycle, configuration convergence, or workload placement, don't install a new central control plane yet.

10. Pre-launch checklist¶

• Each cluster and node has a unique lifecycle owner;
• Personnel access via OIDC, short-term credentials, or proxy, without sharing static admin kubeconfig;
• The management platform itself has independent failure domains, backup, upgrade, and recovery processes;
• Network direction, certificate rotation, and disconnection behavior between Agent, Hub, and member clusters have been tested;
• GitOps, portal, and multi-cluster controllers don't compete for the same object fields;
• Cluster registration doesn't grant more than needed by default cluster-admin;
• GPU, RDMA, driver, and AI Operator compatibility matrix is recorded per cluster;
• Multi-cluster labels have a fixed Schema, source, refresh frequency, and audit;
• Training placement is based on queues and full topology, not just aggregate GPU count;
• Recovery drills have been performed for management plane unavailability, member cluster disconnection, and credential leakage.

Further reading¶

• Kubernetes from major domestic and international cloud vendors
• Kubernetes cross-cluster and large-scale GPU
• Platform operations, upgrades, and multi-cluster
• AI cluster architecture design
• AI/LLM cluster component checklist