---
title: "DeepSeek DeepInfer puts the serving stack on GitHub"
description: "DeepSeek DeepInfer is Apache 2.0 offline inference for V4.1-Flash. EAGLE-4 is source-available, Muse Glimmer fits a GPU, Viggle prices a 3D scene in cents."
canonical_url: https://www.oguzhan.co/deepseek-deepinfer-eagle-4-muse-glimmer/
author: "Oğuzhan Koçaklı"
author_url: https://www.oguzhan.co/about/
date_published: 2026-10-22T06:31:45+00:00
date_modified: 2026-10-22T06:31:48+00:00
language: en
translations:
  tr: https://www.oguzhan.co/tr/deepseek-deepinfer-eagle-4-muse-glimmer-yerel-model/
---

# DeepSeek DeepInfer puts the serving stack on GitHub

DeepSeek DeepInfer is public: offline Apache 2.0 inference for DeepSeek-V4.1-Flash, posted 20 October 2026 at 09:30 UTC (12:30 Istanbul). EAGLE-4, Meta’s Muse Glimmer report and Viggle’s JST-2 landed in the same 48 hours, and all four are about making AI cheaper to run. I read the repos, papers, posts and price pages. I did not install, run or benchmark any of it, so every number below belongs to the vendor.

## 🧰 DeepSeek DeepInfer: the serving stack goes public

![Isometric teal and amber GPU server stacks trading glowing tokens, two empty dashed slots on top for cache and router](https://oguzhanco.s3.eu-west-3.amazonaws.com/wp-content/uploads/2026/10/22063143/deepinfer-serving-stack-missing-cache-router.webp)

*Prefill and decode are in the open release. The KV cache system and the load balancer, drawn here as empty slots, are not.*

[DeepSeek’s post on X](https://x.com/deepseek_ai/status/2112476496283775135) says “Introducing DeepInfer, our in-house inference framework, now fully open source!” The company says it powers DeepSeek-V4.1-Flash at scale and that it is sharing the stack “to help the community lower compute costs, reduce energy use, and make AI more affordable.”

The [README](https://github.com/deepseek-ai/DeepInfer) says this release “provides offline inference” under Apache 2.0. It supports exactly one model, DeepSeek-V4.1-Flash, with the default checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017 (the open weights I covered in the [19 October digest](https://www.oguzhan.co/openai-misalignment-reports-claude-ecdsa-seeds/)). It runs on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs.

The feature list covers MoE expert parallelism with load balancing, speculative decoding (DSpark with confidence-scheduled verification), an asynchronous batch scheduler, deterministic sampling and structured outputs. The engine code says it is derived from vLLM and modified by DeepSeek.

What is missing matters more. The README lists no KV cache system, no prefill/decode disaggregation and no load balancing across replicas. According to [DeepSeek’s deployment Q&A](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md), KV cache management and load balancing are “tightly coupled to our internal infrastructure, so that code is not yet fully open-sourced.” There is no online-serving wrapper either.

The same Q&A drops a few production numbers, all DeepSeek’s own:

- one cluster serves 155 trillion tokens a day
- overall revenue is US$38 per GPU-hour “at valley pricing”
- KV cache is kept for more than 72 hours
- the smallest efficient unit is 64 prefill GPUs plus 64 decode GPUs, with decode at EP64 or above
- storage runs about 14.4 TB per GPU (logical size)

[deepseek-recipe](https://github.com/deepseek-ai/deepseek-recipe) (MIT on GitHub) maps mainstream LLM API protocols to token sequences, “fully aligned with the official DeepSeek API.” DeepSeek says inference code for later models will land in DeepInfer too.

My read: this is a recipe for inference providers and big self-hosters, and a laptop will not get far with it. With 128 GPUs as the efficient floor, the cheap V4.1-Flash price now has its engine room on GitHub. The two parts that turn an engine into a service, the KV cache and the load balancer, stayed home.

## 🦅 Does EAGLE-4 stay fast when the queue fills?

![Glowing wireframe bird sketching a branching token tree while a stone owl turns the accepted branch into gold](https://oguzhanco.s3.eu-west-3.amazonaws.com/wp-content/uploads/2026/10/22063144/eagle-4-speculative-decoding-draft-tree.webp)

*Speculative decoding: the drafter guesses a tree of tokens, the target model keeps the branch it agrees with. EAGLE-4 resizes that tree with the load.*

Speculative decoding in one breath: a small drafter guesses several next tokens and the big target model checks them in one pass. The [EAGLE-4 repo](https://github.com/eagle-hq/EAGLE-4) says the drafter “keeps the target model’s output distribution.” The new trick: one drafter per target model sizes its guess tree from drafting and verification costs it measures at warm-up for that batch size, hardware and workload. [Eagle’s blog](https://blog.eagle.ai/blog/introducing-eagle-4) (20 October 2026) says a draft tree that helps at low concurrency “can become expensive under load.”

Eagle’s own comparison runs Qwen3.8-27B on 4x NVIDIA B300 (TP4, T=0, same engine). For one request, vanilla decoding does 240.5 tok/s, DSpark 618.6, DFlash 2 672.6 and EAGLE-4 863.8. At 512 concurrent requests the aggregates are 21,203, 14,229, 16,641 and 28,340. So in Eagle’s table both rivals fall below plain decoding under load while EAGLE-4 stays above it. The headline 961.6 tok/s is a single request on the TokenSpeed engine (837.6 on the SGLang reference).

It plugs into patched builds of SGLang, vLLM and an experimental TokenSpeed backend. According to [AMD](https://x.com/AIatAMD/status/2112659635295404176), day-0 support on Instinct MI355X gets about 300 tok/s for a single request on one GPU (Qwen3.8-27B, BF16), 1.6x faster than built-in MTP.

Plenty of reposts called it open source. The repo disagrees: it says EAGLE-4 is “source-available software, not OSI-approved open-source software.” Commercial use is limited to aggregate revenue under US$20 million over 12 months. A “Competing Offering” needs a separate commercial license regardless of revenue.

## 💻 Muse Glimmer: Meta’s 30B agent on one consumer GPU

On 19 October 2026, [AI at Meta](https://x.com/AIatMeta/status/2112265822274302066) published a technical report on Muse Glimmer, “our 30B open-weight model built for agentic tasks on consumer hardware.” The [model card](https://huggingface.co/meta-models/Muse-Glimmer-30B) dates the release to August 2026, so the weights are old news and the report is the new part. They got one line in the [15 October digest](https://www.oguzhan.co/muse-wonder-gemini-4-argon-frontier-agents/), after Alexandr Wang told Axios that Meta had recently open-sourced them.

Per the [technical report](https://arxiv.org/abs/2610.24628) and Meta’s [research page](https://ai.meta.com/research/publications/muse-glimmer-30b-an-on-device-agentic-model/), Glimmer is a dense causal transformer of about 29.6B parameters, including a perception encoder of about 1.8B. It takes text and images, writes text, holds 128K+ tokens of context and covers about 140 languages under Apache 2.0. Meta distilled it from Muse Spark, which the report calls Meta’s [frontier-class model](https://www.oguzhan.co/what-is-a-frontier-model/). The target is a device with 32GB of dedicated memory: roughly 4-bit weights plus a DFlash speculative-decoding head, which the report says gives “practical speeds on a single RTX 5090 with minimal quality loss.”

The training detail I liked: agentic data came from simulated “digital lives”: a persona with 10 to 14 contacts and weeks of correspondence, calendar events, documents and messaging history, with tools that mimic real ones down to the JSON.

[Meta’s technical report](https://arxiv.org/abs/2610.24628) scores Muse Glimmer on high reasoning against Gemma4-31B thinking and Qwen3.6-27B thinking.

| Benchmark | Muse Glimmer | Gemma4-31B | Qwen3.6-27B |
| --- | --- | --- | --- |
| MCPAtlas (tool use) | 75.5 | 54.2 | 62.5 |
| SWE-Bench Pro | 51.2 | 36.9 | 50.2 |
| SWE-Bench Verified | 76.0 | 66.6 | 77.2 |
| OSWorld-Verified | 65.9 | 58.5 | 75.6 |

Meta’s numbers, Meta’s runs. Glimmer leads clearly on tool use, while Qwen3.6-27B still wins OSWorld-Verified and SWE-Bench Verified in Meta’s own table.

## 🌍 How cheap is a 3D scene now?

[Viggle’s research post](https://viggle.ai/jst2) (19 October 2026) calls JST-2 (Joint Space-and-Time, generation 2) a “physics-native world foundation model.” Instead of learning in pixel or video space, it models the world as persistent atoms in 3D, each with an explicit xyz position, and decodes them into Gaussian splats that render locally in the browser. Viggle claims a scene takes about one second on an NVIDIA B200 at “an inference cost below $0.01,” and its chart puts World Labs’ Marble at about 5 to 6 minutes. It also turns one image into a rigged, animation-ready character.

Under a cent is the compute cost, though. The [API price page](https://viggle.ai/developers) sells a prompt-built scene for $0.06 (6 credits, because a panorama is generated first) and other tasks for $0.01, with 100 free credits to start. It is free to try in the PINOC browser app and reachable over MCP from Codex or Claude Code, per Viggle’s [launch post](https://x.com/ViggleAI/status/2112202242048528540) and site.

Collisions and object interactions still use rule-based physics. JST-3, in development, aims to learn those dynamics.

Four releases in two days, and all of them about plumbing. My bet is the next real price drop comes from the serving layer rather than a new model. License files also deserve a spot next to the benchmark charts. EAGLE-4 showed how fast “open source” gets repeated by people who never opened the LICENSE.

## Sources

- [DeepInfer repository](https://github.com/deepseek-ai/DeepInfer)
- [DeepInfer deployment Q&A](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md)
- [deepseek-recipe](https://github.com/deepseek-ai/deepseek-recipe)
- [DeepSeek on X](https://x.com/deepseek_ai/status/2112476496283775135)
- [Eagle blog](https://blog.eagle.ai/blog/introducing-eagle-4)
- [EAGLE-4 repository](https://github.com/eagle-hq/EAGLE-4)
- [AMD on X](https://x.com/AIatAMD/status/2112659635295404176)
- [Muse Glimmer research page](https://ai.meta.com/research/publications/muse-glimmer-30b-an-on-device-agentic-model/)
- [Muse Glimmer technical report](https://arxiv.org/abs/2610.24628)
- [Muse Glimmer model card](https://huggingface.co/meta-models/Muse-Glimmer-30B)
- [AI at Meta on X](https://x.com/AIatMeta/status/2112265822274302066)
- [Viggle JST-2](https://viggle.ai/jst2)
- [Viggle API pricing](https://viggle.ai/developers)
- [Viggle on X](https://x.com/ViggleAI/status/2112202242048528540)