https://aik8s.run/en/ 2026-10-21T04:00:45.166066+00:00 GPUs, data, scheduling, training, inference, RAG, agents, and production operations https://aik8s.run/en/ai-k8s/rag-agent/openkruise-agents-kubernetes-practice/ 2026-10-21T03:43:55+00:00 2026-10-21T03:59:26+00:00

Deploy OpenKruise Agents 0.6.0 on Kubernetes 1.30.4, verify the E2B SDK, warm and on-demand creation, filesystem separation and expiry, investigate blocked resume, and compare Kubernetes SIG Apps Agent Sandbox

https://aik8s.run/en/ai-k8s/inference/deepinfer-architecture-analysis/ 2026-10-20T09:41:35+00:00 2026-10-20T09:41:35+00:00

A source-level review of DeepInfer's initial public commit, covering DSpark, asynchronous scheduling, Attention DP/CP, MoE EP/EPLB, SWA bounded replay, hardware requirements, and Kubernetes integration boundaries

https://aik8s.run/en/ai-k8s/practices/deepseek-v41-pro5000-h20-observation/ 2026-10-19T11:10:50+00:00 2026-10-19T11:35:57+00:00

A read-only analysis of existing DeepSeek-V4.1-Flash-1017 metrics on RTX PRO 5000 72GB and H20-3e, examining request shapes, TTFT, TPOT, queues, caching, and resource usage.

https://aik8s.run/en/ai-k8s/rag-agent/opensandbox-kubernetes-practice/ 2026-10-19T09:41:59+00:00 2026-10-19T10:07:03+00:00

Deploy OpenSandbox 1.1.1 on Kubernetes 1.30.4, validate its official Console, execution, authentication, concurrency, and cleanup, and compare architecture, isolation, snapshots, and enterprise selection with Tencent CubeSandbox

https://aik8s.run/en/ai-k8s/practices/contiv-to-kubeovn-underlay/ 2026-10-19T03:08:40+00:00 2026-10-19T04:42:07+00:00

Replacing one node's primary CNI on an existing VLAN network: physical wiring, shared-IPAM ownership, controller isolation, NotReady diagnosis, and observed connectivity

https://aik8s.run/en/ai-k8s/rag-agent/agent-rl-verl-miles-overview/ 2026-10-19T04:27:08+00:00 2026-10-19T04:27:08+00:00

An overview of agent reinforcement learning, its frameworks and environment ecosystem, focusing on veRL Agent Loop, Miles TITO, fully asynchronous training, MoE routing replay, and Kubernetes engineering

https://aik8s.run/en/ai-k8s/rag-agent/deepseek-search-fetch-agent-cost/ 2026-10-19T03:29:57+00:00 2026-10-19T03:29:57+00:00

Compare DeepSeek, OpenAI, Claude, Gemini, Alibaba Model Studio, Brave, and Tavily tool pricing, and understand Search, Fetch, and total agent-task costs

https://aik8s.run/en/ai-k8s/rag-agent/digital-employees-enterprise-design/ 2026-10-18T13:07:05+00:00 2026-10-19T01:16:57+00:00

Compare Tencent WorkBuddy Enterprise and Managed Agents, Alibaba QoderWake and DingTalk Wukong, and Feishu Aily through Hema data queries, Meiyijia customer service, and operations reports, covering identity, execution, integration, and acceptance

https://aik8s.run/en/ai-k8s/practices/nvidia-cluster-readiness-engine/ 2026-10-19T01:01:37+00:00 2026-10-19T01:01:37+00:00

Review NVIDIA NVCRE's public release history, purpose, and production integration, using standalone NCCL/DCGM tests on two nodes with 16 H20-3e GPUs to discuss admission, upgrade regression, troubleshooting, and Kubernetes dependencies

https://aik8s.run/en/ai-k8s/rag-agent/firecracker-cubesandbox-enterprise-selection/ 2026-10-18T14:24:21+00:00 2026-10-18T14:24:21+00:00

Compare Firecracker and CubeSandbox through their different responsibilities as a VMM and a sandbox platform, covering isolation, Kubernetes integration, templates, snapshots, network governance, and operating costs

https://aik8s.run/en/ai-k8s/rag-agent/cubesandbox-process-snapshot-recovery/ 2026-10-18T10:51:03+00:00 2026-10-18T11:10:31+00:00

Kill a running guest process on CubeSandbox v0.7.3, then verify rollback and memory recovery after deleting its original sandbox, with a reproducible script and Light evidence screenshots

https://aik8s.run/en/ai-k8s/practices/deepseek-v41-flash-1017-day0/ 2026-10-18T05:14:54+00:00 2026-10-18T05:14:54+00:00

Eight-GPU vLLM deployment of native mixed-precision DeepSeek 1017 weights, 36 throughput and latency rounds, tools and vision checks, 1M retrieval, and measured resource curves.

https://aik8s.run/en/ai-k8s/practices/minimax-h3-h20-benchmark/ 2026-09-07T10:42:59+00:00 2026-10-17T01:48:34+00:00

Using the same model snapshot on four H20-3e GPUs to validate SGLang Diffusion and vLLM-Omni, recording generation, concurrency, media checks, and raw artifact retention

https://aik8s.run/en/ai-k8s/rag-agent/harnessrouter-overview/ 2026-09-13T15:10:17+00:00 2026-10-16T13:38:22+00:00

Explains HarnessRouter, UHP, Cloud and Community Edition, describing the Agent Harness unified interface, task lifecycle, security boundaries, cost, and its relationship with DSH and LLM Gateway

https://aik8s.run/en/ai-k8s/practices/minimax-m3-h20-benchmark/ 2026-09-08T07:49:32+00:00 2026-10-16T13:38:22+00:00

MiniMax-M3 BF16 TP8 dual-engine functionality, text stress test, multimodal and deployment verification.

https://aik8s.run/en/ai-k8s/practices/deepseek-v4-flash-vision-exp-day0-h20/ 2026-09-01T09:28:10+00:00 2026-10-16T13:38:22+00:00

Running DeepSeek-V4-Flash-Vision-Exp on a single node with 4×141GB H20 using SGLang Preview, documenting NVMe warm-up, 13m52s cold start, OpenWebUI image chat, and 79-round multimodal load testing.

https://aik8s.run/en/ai-k8s/rag-agent/omarchy-agent-os-overview/ 2026-10-16T04:23:57+00:00 2026-10-16T08:23:24+00:00

Starting from Omarchy 4's Linux desktop and Agent integration, compare mainstream Agent OS approaches, and actually install the desktop on KubeVirt, connect models, generate configuration, and verify persistence

https://aik8s.run/en/ai-k8s/rag-agent/cubesandbox-agent-adapter-v06/ 2026-10-16T03:43:04+00:00 2026-10-16T03:43:04+00:00

Use one Adapter to connect multiple CubeSandbox clusters, practicing internal OpenClaw templates, long-task log continuation, controlled MCP, and persistent audit, with five-client real-world testing and Light screenshots

https://aik8s.run/en/ai-k8s/rag-agent/cubesandbox-agent-adapter-v05/ 2026-09-11T03:09:25+00:00 2026-10-16T03:43:04+00:00

Understand the audit boundaries and deployment practices of CubeSandbox Agent Adapter v0.5 through execution intent persistence, failure lockdown, and manual reconciliation

https://aik8s.run/en/ai-k8s/rag-agent/hiroute-overview/ 2026-10-15T03:42:30+00:00 2026-10-15T13:57:38+00:00

An analysis of HiRoute's task routing, model routing, Jev decision-making, ContextHold, stage competence scoring, and limited failover, with a discussion of official cost-reduction experiments, plus hands-on testing of Docker integration with DSH, protocol compatibility, failover, and restart-resume behavior

https://aik8s.run/en/ai-k8s/rag-agent/chatgpt-dots-overview/ 2026-10-14T14:41:47+00:00 2026-10-14T14:41:47+00:00

An introduction to ChatGPT Dot's release timing, long-running tasks, memory, cloud and local execution, Codex collaboration, permission controls, and Kubernetes enterprise integration and evaluation methods, based on official OpenAI materials

https://aik8s.run/en/ai-k8s/rag-agent/agent-long-running-durable-execution/ 2026-10-14T02:10:10+00:00 2026-10-14T02:10:10+00:00

Explains how hour-long agent tasks recover after process crashes, network disconnects, and manual pauses, comparing LangGraph, Temporal, Restate, DBOS, Mastra, and session recovery, with reproducible interruption experiments and Kubernetes architecture recommendations

https://aik8s.run/en/ai-k8s/rag-agent/openclaw-enterprise-kubernetes-practice/ 2026-10-13T02:08:45+00:00 2026-10-13T02:36:39+00:00

Documents an isolated pilot of OCE on Kubernetes, from Ceph RBD, PostgreSQL TLS, image preparation, tenant authorization, to office-network Console and real model replies, explaining DNS, probe, and proxy authentication troubleshooting

https://aik8s.run/en/ai-k8s/rag-agent/openclaw-enterprise-control-plane/ 2026-10-12T12:03:13+00:00 2026-10-13T02:08:45+00:00

An analysis of OpenClaw Enterprise's resource model, persistent work queue, per-Agent identity, runtime drivers, storage, and observability based on public source code, discussing the value and limitations of enterprise deployment

https://aik8s.run/en/ai-k8s/rag-agent/openclaw-enterprise-agent-platform/ 2026-08-10T16:16:11+00:00 2026-10-12T12:03:13+00:00

Analyzes OpenClaw's reusable capabilities, enterprise platform gaps, security risks, recommended architecture, and adoption decisions

https://aik8s.run/en/ai-k8s/scheduling/kubernetes-scheduler-extensions/ 2026-10-10T21:46:59+00:00 2026-10-11T00:00:00+00:00

Using GPU node pool placement as an example, this explains the Scheduling Framework extension points, Profiles, the boundaries between Extender and standalone schedulers, and validation and rollback methods

https://aik8s.run/en/ai-k8s/practices/volcano-scheduler-enterprise-limits/ 2026-10-10T22:01:16+00:00 2026-10-11T00:00:00+00:00

Discussing the business prerequisites for Volcano scheduling and rescheduling adoption, from resource views, task grading, cross-department preemption consensus, and pre-deployment instrumentation

https://aik8s.run/en/ai-k8s/practices/contiv-ovs-cilium-network/ 2026-10-10T08:25:48+00:00 2026-10-10T08:25:48+00:00

Explains VLAN networking, Service translation, and kube-proxy's data path boundaries based on CNI configuration, OVS ports and flow tables, Cilium runtime status, and actual packet captures from an anonymized test cluster

https://aik8s.run/en/ai-k8s/practices/large-scale-kubernetes-scheduler-optimization/ 2026-10-10T04:11:50+00:00 2026-10-10T06:23:47+00:00

Drawing on Kubernetes source code and public practices from Alibaba and ByteDance, explains serial decision-making and node parallelism, plugin and retry optimization, multi-scheduler sharding, concurrency conflicts, and results validation

https://aik8s.run/en/ai-k8s/practices/large-scale-kubernetes-optimization/ 2026-09-18T00:13:31+00:00 2026-10-10T04:11:50+00:00

Drawing on public case studies from Alibaba, ByteDance, and overseas sources, this article breaks down control plane, etcd, scheduling, networking, operational risk control, and scale validation methods for Kubernetes at 10,000 nodes

https://aik8s.run/en/ai-k8s/practices/large-scale-kubernetes-monitoring-tsdb/ 2026-10-10T04:11:50+00:00 2026-10-10T04:11:50+00:00

Discusses large-scale Kubernetes monitoring from the perspectives of series cardinality, scrape discovery, write and query isolation, alerting, storage, and migration, including hands-on deployment validation of the VictoriaMetrics K8s Stack

https://aik8s.run/en/ai-k8s/practices/vllm-sleep-model-switch-l20/ 2026-10-08T21:54:47+00:00 2026-10-10T03:11:11+00:00

Compare model process restart vs Level 1/2 sleep recovery on a single L20, verify short-term throughput after recovery, and analyze Kubernetes DRA sharing boundaries.

https://aik8s.run/en/ai-k8s/rag-agent/jev-weekly-review/ 2026-10-07T06:09:34+00:00 2026-10-07T06:09:34+00:00

Continuously tracking versions, independent evaluations, community weights, and Agent applications of Jev and the System One decision model, preserving results and judgments by week

https://aik8s.run/en/ai-k8s/rag-agent/jev-system-one-model-overview/ 2026-09-21T07:39:36+00:00 2026-10-07T06:09:34+00:00

An overview of TypeSafe AI Jev's interface, confidence scores, use cases, community controversy, production architecture, and evaluation methods, explaining its boundaries with LLMs, traditional classifiers, and rule systems

https://aik8s.run/en/ai-k8s/practices/rbg-qwen-l20-serving-resilience/ 2026-10-02T19:19:21+00:00 2026-10-02T19:19:21+00:00

Using Qwen3.8-27B-FP8 with L20, fixed AIBrix routing, to connect RBG scaling, rolling updates, failure recovery, and client-side TTFT and TPOT evidence.

https://aik8s.run/en/ai-k8s/practices/qwen38-a95b-h20-day0/ 2026-09-12T11:36:12+00:00 2026-10-02T19:05:30+00:00

SGLang and vLLM deployment, functional validation, throughput/latency, cross-node communication, and long-context testing of the Qwen3.8 2.4T MoE model on four nodes with 32 H20-3e GPUs

https://aik8s.run/en/ai-k8s/practices/mimo-v26-pro-rl-h20-first-test/ 2026-09-26T00:59:25+00:00 2026-09-26T00:59:25+00:00

Model architecture of MiMo-V2.6-Pro-RL, 8-card H20-3e vLLM deployment, text and image acceptance, fixed-length stress testing, and operational boundaries

https://aik8s.run/en/ai-k8s/rag-agent/harbor-agent-evaluation-framework/ 2026-09-23T05:55:12+00:00 2026-09-23T05:55:12+00:00

Systematically introduces how Harbor unifies tasks, Agents, models, isolated environments, verifiers, and trajectories, analyzes its boundaries with Agent Harness, Inspect AI, SWE-bench, LangGraph, and Langfuse, and provides a Kubernetes production deployment solution

https://aik8s.run/en/ai-k8s/practices/mimo-v26-flash-rl-h20-day0/ 2026-09-23T02:47:56+00:00 2026-09-23T02:47:56+00:00

MiMo-V2.6-Flash-RL model architecture, SGLang 8-card and vLLM 4-card deployment profiles, fixed-length stress testing, long-context boundaries, multimodal validation, and production deployment recommendations

https://aik8s.run/en/ai-k8s/practices/qwen-image21-l20-benchmark/ 2026-09-21T23:55:42+00:00 2026-09-22T08:19:06+00:00

Deploy Qwen-Image-2.1 on a single NVIDIA L20 in Kubernetes, verifying text rendering, transparent PNGs, and SGLang vs vLLM-Omni throughput, latency, GPU memory, and batch generation workbench

https://aik8s.run/en/ai-k8s/scheduling/koordinator-node-qos-runtime-hooks/ 2026-09-21T05:26:26+00:00 2026-09-21T05:26:26+00:00

Understanding how Koordlet implements CPU, memory, cache, I/O and network QoS from the control loop, container lifecycle and Linux kernel mechanisms

https://aik8s.run/en/ai-k8s/scheduling/koordinator-colocation/ 2026-09-21T02:54:05+00:00 2026-09-21T05:26:26+00:00

Understanding Koordinator through resource accounting, QoS, interference governance, and large-scale community practice, and how to safely advance Kubernetes online-offline workload colocation

https://aik8s.run/en/ai-k8s/practices/koordinator-webhook-multireplica-cert-race/ 2026-09-20T08:35:51+00:00 2026-09-20T09:29:56+00:00

A production cluster Koordinator Webhook certificate race — its diagnosis, fix, and verification, including sanitized real Grafana metrics, shared Secret configuration, certificate checks, and production acceptance methods

https://aik8s.run/en/ai-k8s/practices/kube-apiserver-load-balancing/ 2026-09-18T02:51:26+00:00 2026-09-18T03:26:35+00:00

Starting from connection reuse, TLS identity boundaries, rebalancing during scale changes, and policy selection, compare L4 and L7 load balancing approaches for kube-apiserver along with production validation methods

https://aik8s.run/en/ai-k8s/practices/kubernetes-fleet-reliability/ 2026-09-09T03:10:33+00:00 2026-09-18T00:13:31+00:00

Starting from configuration differences, batch changes, and failure recovery in multi-cluster operations, sharing design trade-offs and implementation methods for stability building

https://aik8s.run/en/ai-k8s/training/distributed-training-frameworks-recovery/ 2026-09-17T14:11:27+00:00 2026-09-17T14:32:33+00:00

Compare the parallelism capabilities, checkpoint and elasticity boundaries of mainstream multi-node multi-GPU training frameworks, and provide a Kubernetes recovery architecture after node failure, RPO/RTO, drill methods, and a production checklist

https://aik8s.run/en/ai-k8s/distributed-training/ 2026-08-02T13:42:27+00:00 2026-09-17T14:11:27+00:00

Training controllers, Gang Scheduling, collective communication, fault tolerance, and job lifecycle

https://aik8s.run/en/ai-k8s/practices/opentelemetry-kubernetes-components-grafana/ 2026-09-16T11:25:45+00:00 2026-09-16T14:55:03+00:00

Use the OpenTelemetry Collector to aggregate Kubernetes control plane, DNS, and object metrics, integrate with Prometheus, Tempo, and Grafana, and test metric cardinality governance and Span troubleshooting in practice

https://aik8s.run/en/ai-k8s/opentelemetry-kubernetes-fundamentals/ 2026-09-16T06:48:09+00:00 2026-09-16T11:25:45+00:00

Explaining OpenTelemetry's Trace, Span, Metric, Log, Context, Resource, OTLP, and Collector using a single Kubernetes request, plus a five-minute experiment that requires no observability backend

https://aik8s.run/en/ai-k8s/practices/opentelemetry-kubernetes-practice/ 2026-09-16T06:20:28+00:00 2026-09-16T06:20:28+00:00

Deploy an OpenTelemetry Collector Gateway on a Kubernetes 1.30 cluster, verify Trace, Metric, Kubernetes metadata, Prometheus ServiceMonitor, and Langfuse OTLP ingestion with FastAPI, and provide production deployment recommendations