Edge0’s launch post leads with one number. Its new 145-billion-parameter model runs at “29GB peak memory” on “a compatible personal laptop, such as a MacBook Pro”. Then you open the Hugging Face model card from the same team. For the NVFP4 build on an RTX 5090 D 24 GB, the card lists 23.25 GiB of VRAM and 38.27 GiB of host RAM, with a 51.2B-parameter n-gram table memory-mapped from disk on top of that. Different units. Different hardware. Different story.
Both numbers can be true at once. The memory engineering underneath Edge0-145B is real and worth an afternoon of any builder’s time. The marketing just squeezes three footnotes into one headline. I read the launch post, both model cards, the license and the card of the Qwen model it is built on. I did not run it. Everything below comes from those pages, and when a figure is Edge0’s own, I say so.
What is Edge0-145B, in one paragraph?
Edge0-145B is an open-weights mixture-of-experts model that Edge0, the AutoArk team, released on 12 Nov 2026 together with a smaller Edge0-120B, an inference framework on GitHub and an Edge0 app that, per the launch post, works offline without an account. The launch post calls it the “world’s first open-source on-device AI model with 145 billion parameters”. The Q3_K model card has a plainer tagline: “A 145B agent, running on one consumer GPU.” According to the card, 145B is the total and 10.12B parameters are active per token, spread over 48 layers with 512 experts each, a 256K context, and training done “natively at 4 bits”. The pitch is scale on small machines. Edge0’s launch post says most on-device models before September 2026 were under 10B, naming Apple’s on-device model at ~3B and Gemini Nano at 1.8B and 3.25B. It also points back to Edge0-35B, the company’s phone model from 10 Sep 2026, which Edge0 says does 37+ tok/s on an iPhone 18 Pro Max at ~2.5GB.
How does a 145B model fit on one 24GB GPU?
It doesn’t. Not in one piece, anyway.
Most of the model sits outside the GPU at any given moment, and the whole design is about making that cheap. Two ideas carry it. One is how the experts are shaped. The other is where they are kept.
Latent experts. The model card walks through a single layer. Attention runs at full width, 2560. A shared projection, which the card calls D, drops each token down to a latent width of 1792. The router picks 10 of the layer’s 512 experts. Each expert is a small SwiGLU block that runs 1792 to 640 to 1792, a shared expert runs alongside them, and a second projection, U, lifts the result back up to 2560.
What makes this work is that D and U exist once per layer, not once per expert. Take them out of the experts and each expert becomes a fixed record of about 1.63 MB, according to the card. Fixed size is the part I would study, more than small size. The card says the engine keeps a few thousand experts on the GPU and streams the rest from host memory “at a cost that tracks how many experts are resident and not how big the model is”. That sentence is the whole design in miniature.
The launch post describes the same idea at a coarser grain: each group of 64 experts shares two large matrices, each expert keeps a small set of its own parameters, and ~10B of the 145B are active per step. The card is more exact. It lists 10.12B active, an expert pool of 84.6B parameters at latent width 1792, and 10 routed plus 1 shared expert per token.
Three memory tiers. This is where the pieces live, per the launch post:
- GPU memory (VRAM): the hot experts, the ones the router keeps asking for.
- System RAM: the expert library. The launch post puts it at ~40GB.
- SSD: vocabulary data, ~48GB in the launch post’s figures, read on demand. The card describes this tier as a 51.2B-parameter n-gram embedding table in fp8, memory-mapped.
Keep those two sets of numbers apart. The launch post talks in rounded GB per tier. The card talks in GiB of residency and in parameter counts. They describe one system from two angles, and neither document maps one onto the other for you. I won’t pretend to either.
The rest of the stack is more familiar. Per the card, there is hybrid attention (full attention every 4 layers plus a sparse indexer), an int8 KV cache with 32K resident, and a 1.26B MTP head that acts as a draft model for speculative decoding. The launch post adds two claims of its own. The model was trained in its on-device architecture from the start rather than compressed afterwards, and the runtime schedules the CPU and GPU together.
Does “29GB on a laptop” hold up?
Put the two documents side by side and most of the gap comes down to what each one chose to count.
| Launch post (Edge0) | Hugging Face model card | |
|---|---|---|
| Memory | 29GB peak. Also “enabling local use on a 48GB laptop” | NVFP4 on RTX 5090 D 24 GB: 23.25 GiB VRAM + 38.27 GiB RAM. Q3_K: 23.14 GiB VRAM + 31.28 GiB RAM |
| Disk | ~48GB vocabulary data on SSD | 51.2B fp8 n-gram table, memory-mapped. Builds: Q3_K 40.81 GiB, NVFP4 47.79 GiB |
| Context | 128K to 256K | 256K, KV int8 with 32K resident |
| Platforms | Day-one support claimed for macOS, Windows, Linux, Apple M-series, NVIDIA RTX Spark, RTX 4090, RTX 5090 | Build instructions target NVIDIA GPUs with CUDA 13 on Linux or Windows (x64 or ARM64) |
| Mac | “a compatible personal laptop, such as a MacBook Pro” | M2 Ultra figures use Metal and “do not establish macOS release availability” |
| App | Edge0 app, offline, no account | Desktop app download: Windows |
| Speed | 1,100+ tok/s prefill, 80+ tok/s decode, ~120W typical power | See the speed table below |
The 29GB figure is Edge0’s own. The launch post’s chart labels it “measured”, and the post says the hardware environment arrives with the technical report. Until then, the card is the only place with a breakdown. It describes a 24 GB GPU, a host RAM pool that on its own is bigger than 29GB, and an SSD doing real work. The same launch post also mentions “enabling local use on a 48GB laptop”. If I were buying hardware for this, I would plan around 48, not 29.
Here are the card’s speed numbers, in tok/s, as published on the Q3_K and NVFP4 cards:
| Hardware | Build | 16K prefill | Chat decode | Agent decode |
|---|---|---|---|---|
| RTX 5090 D 24GB | Q3_K | 3554.9 | 99.7 | 121.2 |
| RTX 5090 D 24GB | NVFP4 | 3165.7 | 79.6 | 98.3 |
| RTX 4090 24GB | Q3_K | 3179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60 GiB | NVFP4 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal) | Q3_K | 800.3 | 60.9 | 57.0 |
| RTX 5090 D capped at 16 GiB VRAM + 20 GiB RAM | Q3_K | 335 | 36.8 | 44.9 |
Two rows matter most to me. NVFP4 chat decode on the RTX 5090 D is 79.6 tok/s, which is the card’s closest figure to the launch post’s “80+”. And the last row. That is the same 5090 D, capped by cgroup at 16 GiB of VRAM and 20 GiB of RAM with no swap, still running Q3_K, at 335 tok/s prefill and 36.8 tok/s chat decode. It is the honest answer to “what if my machine is smaller”. It runs. Slower.
The only Apple row is an M2 Ultra on Metal, and we will get to what the card says about that.
How good is it, and who measured?
Edge0 measured. Every row below is an Edge0 internal evaluation, competitors included, and the launch post says “the complete evaluation setup, hardware environment, and reproduction instructions are being released with our technical report”. That report is not out yet.
| Model | DeepSWE 1.1 | Terminal-Bench 2.1 | HLE | GPQA-Diamond | IFBench | AA-LCR |
|---|---|---|---|---|---|---|
| Edge0-145B | 64.6 | 86.52 | 35.4 | 90.4 | 80.6 | 79.3 |
| Qwen3.8-Flash-Next | 58.7 | 83.3 | 35.9 | 91.7 | 81.3 | 79.1 |
| DeepSeek-V4-Flash-0731 | 54.4 | 82.7 | 33.8 | 90.8 | 79.2 | 79.7 |
| Claude-Opus-4.6 (Max) | n/a | 78.2 | 40.0 | 91.3 | 62.5 | 78.0 |
Source: Edge0 launch post, Edge0 internal evaluations.
Read across and the pattern is clear. Edge0-145B leads in the agent and coding columns: 64.6 on DeepSWE 1.1 against 58.7 for Qwen3.8-Flash-Next, and 86.52 on Terminal-Bench 2.1 against 83.3. On HLE, GPQA-Diamond and IFBench, Qwen3.8-Flash-Next scores slightly higher. That is the model Edge0-145B is built on. So the base edges out its derivative on three of six columns, by Edge0’s own numbers. The Claude row is Claude Opus 4.6, an older Claude, not the current Opus 5.5. Read it as a comparison with that generation.
The launch post also plots a six-benchmark mean against memory. Edge0-145B sits at 72.8 with 29GB, Qwen3.8-Flash-Next at 71.7 with ~70GB, DeepSeek-V4-Flash-0731 at 70.1 with ~87GB, and Qwen3.8-27B at 66.1 with ~18GB. Only the Edge0 point is measured. The other memory figures are Edge0’s estimates for a standard 4-bit deployment, which in the post’s words “remain to be measured”. One measurement plotted against three estimates is a chart I read twice.
Recursive Flint, which I covered in the 19 to 25 Oct weekly, also turns up in Edge0’s comparison. The launch post adds company demos too, including a Blender Manhattan scene built within an hour and a KiCad PCB turned into an Unreal Engine 5 animation, and I have no way to check any of them.
Is it really open source?
The launch post says “open-source”. The license file says something narrower.
There are two documents. The Edge0 Community Model License 1.0, dated 12 Nov 2026, covers only Edge0’s own contributions. Everything that came from Qwen stays under the Qwen Community License 1.0, and the Edge0 license spells it out: “Permission from Edge0 is not permission from Qwen”. If your plans go beyond personal use, you are reading both.
Section 3 of the Edge0 license is generous. It permits:
- personal use and local deployment
- research and benchmarking
- noncommercial sharing of modified models
- internal business use
- paid freelance work, as long as customers don’t get access to the model
- commercial use of outputs
Section 4 is where the fences go up. You need a separate written license from Edge0 to:
- sell or rent the model, or distribute it for pay
- run a paid inference or fine-tuning service for third parties, such as an API or a hosted endpoint
- ship a paid standalone product aimed mainly at AI-assisted coding or office productivity, hosted or on-device
Builders will hit that last clause first. On-device is named explicitly. A paid local coding assistant built on Edge0-145B needs a written deal with Edge0, even if nothing ever leaves the user’s machine.
The code is a separate matter. Per the card, the framework code is Apache-2.0, and Edge0 extends the Strata engine, which is MIT, with latent MoE support. Open-source code, open weights under a community license. “Open-source model” in a headline is carrying a lot.
Then there is lineage. The card lists Qwen3.8-Flash-Next as the base model, and its acknowledgements read: “We thank the teams behind Qwen3.8-Flash-Next and Strata.” The Qwen3.8-Flash-Next card describes that model as “125B with 6B activated, plus 51B n-gram embedding and 4B MTP”, with hidden dimension 2560 and 48 layers. The n-gram table, the hidden size and the layer count all line up with Edge0’s card. Qwen even pitches n-gram embedding as an axis “more amenable to offloading than MoE”, and Edge0’s card keeps exactly that table on disk. What neither card explains is how 125B became 145B. I looked. It isn’t there. The Terminal reported on 13 Nov that the launch post never names Qwen as the base, while the card does.
Should you run it?
Depends on the machine, and on the business.
A 24 GB NVIDIA card, personal or internal use. Yes, it is worth a weekend. The build instructions target NVIDIA GPUs with CUDA 13 on Linux or Windows, and the card has measured rows for the RTX 4090 and RTX 5090 D. Plan your RAM from the card’s residency figures, not from the 29GB headline. If RAM is tight, the card documents a low-RAM mode. --mmap-experts reads uncached experts from SSD, and --resident-budget-gib 20 together with an expert profile file caps the host RAM expert cache. The server exposes OpenAI- and Anthropic-compatible endpoints at localhost:8000:
edge0-infer serve --model /path/to/Edge0-145B-NVFP4
A Mac. Wait. The card does publish Apple M2 Ultra numbers, but it says those measurements use Metal and “do not establish macOS release availability”. The desktop app download is Windows. The MacBook Pro line in the launch post is running ahead of the shipping docs.
A paid coding assistant. Read Section 4 of the license before you write a line of product code. Then read the Qwen license. Then talk to Edge0.
Edge0 built something clever and then described it with the least precise number available. Read the card, not the headline. If you want another open-weights release read card-first, my Inkling 2 Small deep dive is the closest companion piece. The rest of this week’s launches, including Gemini 4 Carbon going public and two world models, are in the midweek digest.
Sources
- Edge0, Edge0-145B launch post, 12 Nov 2026
- Hugging Face, Edge0/Edge0-145B-NVFP4 model card
- Hugging Face, Edge0/Edge0-145B-q3k model card
- Edge0, Edge0 Community Model License 1.0, 12 Nov 2026
- Hugging Face, Qwen/Qwen3.8-Flash-Next model card
- Edge0 inference framework on GitHub
- The Terminal, report on the Edge0-145B base model and model card, 13 Nov 2026
