DeepSeek-V4.1-Flash-1017 on 4× DGX Spark with SGLang
DeepSeek-V4.1-Flash-1017 is a weight update with the same config and the same 96,085 tensors, so the SGLang recipe runs it unchanged. What it takes on a serving fleet: a memory-capped download, new Engram row copies with the fleet's own row ranges, the snapshot placed where the NFS mounts already point, and a short switch window. Plus where SGLang's built-in prompt encoder now differs from DeepSeek's reference.
4× DGX Spark GB10 · TP4/EP4 · SGLang 1.7.1 · aidendle94/sparkrun-sglang-dsv41-gb10:production-1.7.1 · measured Oct 18, 2026
- DeepSeek-V4.1-Flash
- SGLang
- multi-node
- TP4
- DGX Spark
- FP8
- MXFP4
- MoE
01Overview
- A running fleet from the DeepSeek-V4.1-Flash SGLang recipe on this site (4× DGX Spark GB10, image aidendle94/sparkrun-sglang-dsv41-gb10:production-1.7.1)
- deepseek-ai/DeepSeek-V4.1-Flash-1017 weights (510 GB, 48 shards), plus room for them on the node that holds the checkpoint
- ~50 GB of local NVMe per node for the new Engram row copies
- huggingface_hub with hf_xet (some shards are too large for the plain HTTP download)
On 2026-10-17 DeepSeek published DeepSeek-V4.1-Flash-1017, which it calls a minor update to V4.1-Flash. For a serving stack it is a weight swap: config.json and inference/config.json are identical, and the safetensors index lists the same 96,085 tensors in the same 48 shards. Every shard changed. The SGLang recipe on this site serves it with the same image, overlay and flags. This page covers the swap on a running fleet, what was measured after it, and where SGLang's built-in prompt encoder now differs from the reference DeepSeek ships with the release.
| Benchmark (DeepSeek's model card) | V4.1-Flash | V4.1-Flash-1017 |
|---|---|---|
| HLE, no tools | 36.8 | 42.6 |
| Terminal-Bench 4.0 (Claude Code harness) | 31.2 | 42.3 |
| DeepSWE v1.1 | 74.2 | 77.0 |
| ProgramBench | 20.3 | 29.1 |
| Automation-Bench v1.0.6 | 54.8 | 60.5 |
| Apex Bench v1.1 | 64.6 | 72.1 |
DeepSeek's numbers, not measured here. The model card lists eleven benchmarks; 1017 is ahead on all of them.
02Build
The swap
- Download on a node with room, memory-capped. The checkpoint is 510 GB. A downloader on a serving GB10 node competes with the engine for the same unified memory: in high-performance mode the xet backend grew to 11.4 GB and left a serving node 3 GB free. Run it in a cgroup, for example
systemd-run --user -p MemoryMax=4G -p MemorySwapMax=0 hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 --revision <rev>. Plain HTTP (HF_HUB_DISABLE_XET=1) stays near 2 GB but refuses the largest shards. Fetch those with xet at one worker and no chunk cache (--max-workers 1,HF_XET_CHUNK_CACHE_SIZE_BYTES=0). - Verify before you copy. Compare every LFS file with the sha256 the Hub lists for that revision (
/api/models/<repo>/revision/<rev>?blobs=true). Read with direct I/O (dd iflag=direct) so the check does not fill the page cache of a node that is serving. On GB10 the GPU driver fails allocations rather than evict cached files. - Put the snapshot where the NFS mounts already point. The fleet exports the checkpoint node's HuggingFace repo directory for V4.1-Flash. Copy the new revision into that same directory, blobs into
blobs/and the snapshot intosnapshots/<rev>. Blobs are content-addressed, so nothing collides with the old revision, and the other nodes see the new snapshot through their existing mounts. Stage the copy in a side directory and move it in when it is complete (a rename on the same disk). Cap the receiving rsync as well (--rsync-path='systemd-run --user --scope -q -p MemoryMax=2G rsync'). - New Engram row copies, with the fleet's own row ranges. The rows changed with the weights, so each rank's node-local copy has to be rebuilt (
tools/engram_local.py, ~48 GiB per rank, about 90 s each from local NVMe plus a byte-for-byte spot check). Use the ranges the running fleet already uses: they are in each copy'sengram-local.jsonand inENGRAM_ROWS_HOST. The ranges must tile each table exactly. A copy built from a different partition fails at boot, even one only a few hundred rows off. - Change two lines of `fleet.env`.
SNAPandENGRAM_LOCAL_DIR. Diff the launcher's--dry-runoutput against the current one: the only changes should be the snapshot path and the Engram mount, and every other flag should match. - One window. Disarm the watchdog, stop every rank, swap any copy that could not be staged next to the old one, edit
fleet.env, then runlaunch/relaunch.sh, which boots, waits for/healthand re-arms the watchdog. Ours was 17.5 minutes down. Eight of those were one node's Engram copy: the node had no room to hold both copies, so it could only be replaced while the engine was stopped.
-SNAP=snapshots/<V4.1-Flash revision>
+SNAP=snapshots/<V4.1-Flash-1017 revision>
-ENGRAM_LOCAL_DIR=dsv41-engram-local
+ENGRAM_LOCAL_DIR=dsv41-engram-local-101703Deep dive
SGLang's prompt encoder against the 1017 reference
DeepSeek ships no Jinja chat template. The release carries encoding/encoding.py, a reference encoder with 55 tests. SGLang has its own port of that encoder, so we ran the reference tests with prompt building routed through SGLang's copy (the one in the production-1.7.1 image). 25 pass and 30 fail. The prompts are identical where most traffic is: four of the five golden conversations, and multi-turn tool loops that send their tool definitions, as Claude Code does. The failures group into four changes the reference made after SGLang's port:
| Change in the reference | SGLang (production-1.7.1) | Who it affects |
|---|---|---|
| named reasoning efforts: low 50, high 75, max 100 (xhigh is gone) | low 25, high 50, xhigh 75, max 100 | clients that send a named effort; a numeric budget renders the same in both (the fleet default is the integer 100) |
| images in tool results follow the prompt order when parallel results arrive out of call order | the results are sorted, the image list is not, so the pictures can land in each other's slots | parallel tool calls whose results carry images and come back out of order |
namespaced tool names (namespace::name) | not supported | clients that send OpenAI tool namespaces |
| the current turn keeps its reasoning across tool results even when no tool definitions are sent | drops it in that case | tool loops sent without the tool list |
The fifth golden conversation differs only in the default effort line (75 vs 50). SGLang is also more lenient with invalid efforts, which the reference rejects; that one is harmless.
04Performance
Measured 2026-10-18 (switch from V4.1-Flash to V4.1-Flash-1017 on the serving fleet). Four-node TP4/EP4 · lmsysorg/sglang:dev-dsv41 + the MIT overlay · DSpark block 5 · same flags as the SGLang recipe; only the snapshot and the Engram row copies changed. Image: aidendle94/sparkrun-sglang-dsv41-gb10:production-1.7.1.
Across recipes
Different prompt sets, all single stream at temperature 0 with completion_tokens accounting: exact within a recipe, indicative across them. DeepSeek-V4.1-Flash · SGLang (1.2): code prompts, DSpark block 5, RoCE collectives, early Engram staging; 110 counting, 37 prose · DeepSeek-V4.1-Flash (1.1, DCP2): code prompts, DSpark k=5, b12x MoE, DCP2 (77.5 on 1.0 with a different prompt set) · GLM-5.3-Flash (1.2): structured output, thinking off · DeepSeek-V4-Flash (3.76.1): mixed prompts, 8-rep mean · DeepSeek-V4-Flash-Vision (3.73): code prompts, DCP2 + DSpark
Cold means the prompt has never been seen by the server; with prefix caching on, a repeated prompt is a cache hit and reads 3–25× higher. DeepSeek-V4.1-Flash · SGLang (1.5): 32K depth, unique random words (2,913 at 128K) · DeepSeek-V4.1-Flash (1.0): 36K depth (1,940 at 132K) · GLM-5.3-Flash (1.2): 32K depth · DeepSeek-V4-Flash (3.76.1): 334K depth · DeepSeek-V4-Flash-Vision (3.73): 92K depth (2,088 at 334K)
Measured after the switch
| Same fleet, image production-1.7.1 | V4.1-Flash | V4.1-Flash-1017 |
|---|---|---|
| 10-turn coding conversation over ~62K tokens of source, overall decode | 65.9 tok/s | 66.3 tok/s |
| KV pool (auto-sized at fraction 0.80) | 5,163,008–6,042,368 tokens on recent boots | 6,198,016 tokens |
| boot to healthy | ~10 min | 575 s |
| batched correctness (same greedy chats alone and four at once) | OK | OK, 0 of 8 broken |
| image, short prompt and inside a 15K-token prompt | OK | OK |
/v1/messages tool round trip with thinking | OK | OK: calls the tool, answers from its result |
Decode speed does not move: the architecture and the kernels are the same, and a decode step is at the MoE bandwidth wall. The KV pool varies with free memory at boot (4.2–6.6M tokens across boots). Thinking is on at reasoning effort 100: 25 reasoning tokens on a one-line puzzle, 628 on a counting problem, both answers correct.
05Notes
Notes
- Send numbers, not names, for reasoning effort.Until SGLang follows the new mapping,
"reasoning_effort": "high"means 50 on this stack and 75 in DeepSeek's reference. An integer from 1 to 100 means the same thing in both. - Context is 524K here.The model card recommends a 1M-token window and at least 256K output tokens. The fleet keeps the SGLang recipe's 524,288-token context, which is what its KV pool and memory budget were sized for.
- Sampling.DeepSeek recommends temperature 1.0 and top_p 1.0, the same as for V4.1-Flash.
- The old revision can stay.Its blobs sit next to the new ones in the same repo directory, so going back means the old
SNAPand rebuilt Engram copies, not another download.