DeepSeek-V4-Flash-Vision-Exp Day 0: 4×H20 Multimodal Deployment and Load Testing¶

On the evening of August 31, 2026, DeepSeek published DeepSeek-V4-Flash-Vision-Exp on Hugging Face. This is not a renamed text-only DeepSeek-V4 model, but an experimental version of DeepSeek-V4-Flash with a complete vision encoding pipeline added: the repository page marks it as 305B, the checkpoint consists of 48 Safetensors shards totaling approximately 156.31 GiB; the vision side includes a 32-layer ViT, Aligner, image Marker, and image-visibility Attention, with a single image expanding to at most 384 Image Tokens, and native context still at 1,048,576.

After the model was released, I completed deployment, image correctness checks, OpenWebUI hands-on testing, and 79 rounds of multimodal performance testing on Kubernetes using a single node with 4×141GB H20 and the SGLang dedicated Preview Runtime. In the end, all 3,856 requests succeeded with 0 failures; after testing, the Deployment automatically scaled down to 0 and all 4 GPUs were released.

Conclusions first:

• It runs. The raw HF checkpoint loads directly from node NVMe with TP4; SGLang OpenAI-compatible API, single-image, dual-image interleaved, and OpenWebUI image upload all pass.
• Being able to Ready does not mean fast startup. NVMe read of the 48 shards took only about 18 seconds, but first-time MHC compilation, DeepGEMM/FlashInfer Autotune, and two sets of CUDA Graph Capture pushed full cold start to 13 minutes 52 seconds.
• 720p cold images reached 175.06 Output tok/s at concurrency 16, but P99 TTFT rose to 3.86 seconds. The Warm path with the same image reused reached 332.52 Output tok/s, with P99 TTFT dropping to 1.10 seconds.
• Image count significantly reduces request throughput. 2×720p at C8 was 1.71 req/s; 4×720p at C4 was 2.11 req/s. The two groups have different concurrency, so the latter cannot be used to directly infer that "four images is faster than two."
• This is not an SGLang vs. vLLM A/B. As of 2026-09-01, SGLang provides a Vision-Exp dedicated Preview implementation; vLLM's public DeepSeek Recipe is still the text-only V4, and a text service cannot masquerade as vision support.

References: Model repository, SGLang DeepSeek-V4 Cookbook, vLLM DeepSeek Recipe list.

Model Structure and Runtime Boundaries¶

The checkpoint's outer config.json does not use the common standalone vision_config auto-discovery path; the complete vision parameters and reference implementation are located under the repository's inference/ directory. This directly affects model registration, weight loading, and image_url handling in general-purpose serving engines, so the one-line launch command auto-generated on the Hugging Face page cannot replace end-to-end validation.

This run pinned the SGLang Runtime to commit 914197146f8a3407960e5c7037d0463e03c37be9, which is a dedicated Preview Build, not a stable release. The runtime automatically selected DSV4 Attention, FlashInfer MXFP4 MoE, FlashInfer AllReduce Fusion, and FP8 KV Cache. The logs also warned that the checkpoint does not provide KV Scale, so the runtime fell back to 1.0; therefore, the image correctness Gate cannot be omitted just because throughput testing succeeded.

Experimental Environment¶

This is single-node TP4; Tensor Parallel communication does not cross nodes, so RDMA is not used in this round. A node having RDMA devices does not mean a single-node experiment can be written up as "RDMA verified." The H20 is also not in the current Vision-Exp official signed hardware matrix; this article can only prove that this combination of H20, Driver, CUDA, and Preview commit works in practice.

Why Put Weights on NVMe First¶

The shared file system only handles distribution, not the serving hot path. After selecting the node, the full HF checkpoint was copied to local NVMe and verified before startup:

1. All 48 index-referenced shards exist and are non-empty;
2. Total shard file byte counts match between source and destination directories;
3. The destination directory's underlying block device is indeed NVMe;
4. Serving is only allowed to start after the write-completion Marker is in place.

This time, copying the 156.31 GiB of weights took 3 minutes 39 seconds. The public example retains only directory semantics and does not expose internal storage addresses:

findmnt -T /srv/model-cache
lsblk -o NAME,TYPE,SIZE,MOUNTPOINTS

rsync -a --partial --info=progress2 \
  /shared/models/DeepSeek-V4-Flash-Vision-Exp/v1/ \
  /srv/model-cache/DeepSeek-V4-Flash-Vision-Exp/v1/

Kubernetes Launch Method¶

Core parameters are as follows. Image registry, node name, Namespace, and real host directories all use public placeholders:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deepseek-v4-vision-sglang
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    spec:
      nodeSelector:
        accelerator.example.com/model: h20-141g
      containers:
        - name: sglang
          image: <INTERNAL_SGLANG_VISION_PREVIEW_IMAGE>
          command: ["bash", "-lc"]
          args:
            - |
              exec sglang serve \
                --model-path /models/DeepSeek-V4-Flash-Vision-Exp/v1 \
                --served-model-name DeepSeek-V4-Flash-Vision-Exp \
                --trust-remote-code \
                --tp 4 \
                --speculative-algorithm DSPARK \
                --mem-fraction-static 0.85 \
                --reasoning-parser deepseek-v4 \
                --enable-metrics \
                --host 0.0.0.0 \
                --port 30000
          resources:
            requests:
              nvidia.com/gpu: "4"
            limits:
              nvidia.com/gpu: "4"
          startupProbe:
            httpGet:
              path: /v1/models
              port: 30000
            periodSeconds: 10
            failureThreshold: 720
          volumeMounts:
            - name: model-cache
              mountPath: /models
              readOnly: true
            - name: dshm
              mountPath: /dev/shm
      volumes:
        - name: model-cache
          hostPath:
            path: <NVME_MODEL_ROOT>
            type: Directory
        - name: dshm
          emptyDir:
            medium: Memory
            sizeLimit: 64Gi

The Preview Runtime's first-time compilation is very long, so startupProbe must cover the real cold-start window. Allowing up to 120 minutes here is not because normal startup takes two hours, but to prevent Kubelet from killing the pod mid-compilation and restarting from scratch.

Cold Start: NVMe Is Not the Whole Answer¶

These numbers show: NVMe solves large weight reads, but cannot eliminate runtime first-time compilation and graph capture. If every pod recompiles in an empty cache environment during rolling releases, even a 156 GiB model will struggle to recover in minutes. Before production, compilation cache persistence, pre-compiled artifacts in images, and phased rollout strategies should be further validated.

Functional Correctness and OpenWebUI¶

First, four deterministic Gates were run using the cat and dog images bundled in the Preview image: text-only returns only TEXT_OK, cat image returns only CAT, dog image returns only DOG, and cat-to-dog dual-image interleaved returns only CAT,DOG. All passed; answers change with image substitution, and dual-image order is also correct, so this is not "HTTP 200 but images ignored."

The same OpenAI-compatible API was then connected to OpenWebUI. After uploading a cat image in Light mode, the model correctly identified the cat, pink hooded leather jacket, and sunglasses, completing the loop across cluster access, model discovery, image upload, Chat Template, and multimodal response.

Load Testing Methodology¶

The main test does not evaluate quality with real datasets; instead, it generates deterministic 360p, 720p, 1080p, and multi-image JPEGs specifically to measure serving performance. Cold Cases use a different image per request; Warm Cases reuse the same image. All cases fix output at 64 Tokens, with request-rate=inf, ignore-eos, and Seed fixed; each valid point records three rounds, and the table takes the median metric per round.

vllm bench serve \
  --backend openai-chat \
  --base-url http://<sglang-service>:30000 \
  --endpoint /v1/chat/completions \
  --model DeepSeek-V4-Flash-Vision-Exp \
  --dataset-name custom_image \
  --dataset-path <CASE_JSONL> \
  --custom-output-len 64 \
  --custom-ensure-client-side-data \
  --enable-multimodal-chat \
  --request-rate inf \
  --ignore-eos \
  --disable-shuffle \
  --save-result --save-detailed

Although the client command name is vllm bench serve, the service under test is still SGLang. It only sends requests through the OpenAI-compatible HTTP interface; a unified client facilitates future framework A/B testing. SGLang's built-in Benchmark Client can be used for engine-internal diagnostics, but numbers from two different clients cannot be directly combined into a fair comparison.

This round generated 79 result JSONs total: 112 warm-up requests and 3,744 official requests, for a combined 3,856 successes, 0 failures. The full matrix took 46 minutes.

Performance Results¶

Single-Image Cold and Warm¶

For 720p at C16, the Warm path relative to Cold: both request throughput and output throughput improve by about 90%, P50 TTFT drops 54.2%, and P99 TTFT drops 71.4%. This is not "the model itself suddenly became twice as fast," but rather that same-image reuse reduces repeated vision preprocessing and Encoder work. If real business traffic has mostly unique images, the Cold table should be used as the capacity baseline.

High Resolution and Multi-Image¶

For multi-image scenarios, you should look at Req/s, TTFT, and image count together, not just Token/s. 4×720p at C4 processes about 8.44 images per second, but each request itself contains four images; its scheduling pressure differs from 2×720p at C8, so you cannot rank horizontally by Req/s alone.

First-time new-shape JIT must be recorded separately¶

The first officially recorded P99 TTFT for 360p at C16 was 7.91 seconds, and the following two rounds dropped to about 1.22–1.24 seconds. All raw JSON was retained; the main table follows the pre-defined rule of taking the median of three rounds, which is 1.24 seconds, without deleting the first round. For the Preview Runtime, after /v1/models is Ready, there may still be first-time compilation for new Batch Shapes, so Canary warm-up needs to cover the resolutions, image counts, and concurrency planned for production use, not just send a single text request.

OpenWebUI integration¶

OpenWebUI only needs an OpenAI-compatible Base URL that is reachable from its own network:

http://<gateway-or-service>/v1

Before integration, verify /v1/models and one /v1/chat/completions carrying a Data URL from the network where the OpenWebUI Pod resides. If the model appears in the selector but there is no image upload button, you also need to explicitly mark Vision in OpenWebUI's model capability settings; in this version, image upload can be completed directly from the returned model, without needing a frontend plugin.

In cross-cluster scenarios, do not write temporary Pod IPs into OpenWebUI. You can use a fixed Ingress, Gateway API, or create a Selectorless Service + EndpointSlice in the OpenWebUI cluster pointing to a stable entry point. Public configurations should remove real internal network addresses.

Why there is no vLLM data¶

As of 2026-09-01, the DeepSeek-V4-Flash listed on the vLLM Recipe page is a text model, and there is no Vision-Exp under the Multimodal category either. The text V4 being able to load language weights from the same family does not mean it can load ViT/Aligner, process image_url, maintain image visibility masks, and correctly return visual answers.

Therefore, this article does not do the following two things:

1. It does not use the text-only vLLM image to run "image requests" and then write the errors off as poor vLLM performance;
2. It does not let images be silently ignored and then fake an SGLang/vLLM A/B comparison just because of an HTTP 200.

Once vLLM has explicit model registration, a dedicated Recipe, and image E2E tests, the same client, the same JSONL, the same H20 node, and the same NVMe Checkpoint from this article can be reused for a fair retest.

Production recommendations¶

1. Pin the Preview commit and image Digest. Do not use a drifting latest.
2. CFS is only the source; NVMe is the hot path. Verify the Block Device and complete Marker before startup.
3. Reserve at least 15 minutes of cold-start budget. Also persist the compilation/Autotune Cache and verify whether it can shorten the second startup.
4. Apply Admission Control by image count and resolution. The same text token count does not mean four-image requests have the same cost.
5. Build capacity baselines separately for Cold/Warm. Do not cite Warm results when business images are unique.
6. Keep the first-time new-shape tail latency. Canary warm-up covers real concurrency; do not delete the "ugly" first round.
7. Separate quality from performance. The cat-and-dog Gate in this article proves the image pipeline works, but it cannot replace quality evaluations such as MMMU, MathVista, RealWorldQA, and OCRBench.
8. Automatically release GPUs when the task ends. This time, a least-privilege cleanup Job scaled the Deployment to 0 after the Benchmark Complete and confirmed the Serving Pod exited.

What this article did not prove¶

• Did not verify the model's claimed 1M Context with images as joint input;
• Did not run standard vision quality datasets, so no model capability ranking can be given;
• Did not verify cross-node TP/EP, RDMA, or P/D separation;
• Did not obtain vLLM Vision-Exp data, so no framework winner can be declared;
• Preview Runtime, FP8 KV Scale fallback, and H20 non-officially-signed hardware remain launch risks.

But the most critical Day 0 engineering question has been answered: DeepSeek-V4-Flash-Vision-Exp can provide real multimodal service on 4×141GB H20 via SGLang Preview with TP4; the NVMe weight hot path, OpenWebUI image pipeline, and the 3,856-request performance matrix are all closed-loop, and the GPUs were automatically released after the test ended.

Reproduction materials¶

• examples/deepseek-v4-flash-vision-exp/
• DeepSeek-V4-Flash-Vision-Exp model repository
• SGLang DeepSeek-V4 Cookbook
• vLLM DeepSeek Recipe list