🔍 Read the full analysis: Why Consider Inkling 2 Mini For Your Local AI Setup? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters, with about 3 billion active per token. It supports several inference runtimes, but the supplied information does not establish independent benchmark results, practical hardware requirements or real-world speed.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local AI setups and datacenter inference. The company says it has about 26 billion total parameters, activates about 3 billion per token, and supports tools including Transformers, llama.cpp, MLX and SGLang; independent performance results and consumer-hardware requirements are not established in the supplied material.
The model accepts text and image input and is described as having a 1 million-token context window, a relevant consideration when planning local AI workloads on a Mac mini. Its architecture has 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers are reused for three passes, producing 48 logical layer visits per token. According to the company’s technical description, this shares weights across repeated layers rather than assigning a distinct set of weights to every logical position.
Inkling 2 Mini uses a mixture-of-experts design: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says the model’s expert weights are distributed in NVFP4 and MXFP8 mixed-precision formats, and that it includes a DSpark drafter intended to speed generation with a small memory cost. The fixed-pass release is supported by Transformers, llama.cpp and MLX; SGLang is listed for day-one inference support in datacenter deployments, among the considerations for building a local AI workstation.
A separate research checkpoint, TML-d24-varloop-research, allows users to change the number of passes through the repeated layers. The company says it can run with one or two passes instead of three, potentially increasing speed with a possible quality trade-off. Its description also says all 48 visits in the fixed model require execution, with each visit retaining its own key-value and short-convolution state.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
What Local AI Users Gain
The release gives people evaluating local models another option that combines image and text input with a large claimed context window and support for several commonly used inference tools. That range of runtimes may make it easier for developers to test the model in different environments, but compatibility does not by itself show that it will run well on a particular computer.
The architecture illustrates a trade-off relevant to local deployment: repeated layers reuse weights, while expert routing activates only part of the expert pool for each token. Those design choices may balance capacity and computation, but they do not establish the model’s actual memory footprint, generation speed or output quality. Local users still need device-specific measurements before deciding whether it fits their hardware or workload.
The adjustable-pass research version could also help researchers study how reducing computation affects speed and quality. However, the available description does not provide independent results for one- or two-pass configurations, so the practical value of that option remains to be tested.
Popular right now

Dyson Cyclone V10 Origin Cordless Vacuum

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Ninja Foodi Smart 6-in-1 Indoor Grill and Air Fryer, 4 qt, FG551
As an affiliate, we earn on qualifying purchases.
How Inkling Reuses Its Layers
The central design feature is the reuse of the 12 middle physical layers for three passes. The release says each pass begins with the same input from the first six layers, while learned messages carry information between passes and a learned gate controls how much information is passed along. This lets the model make 48 logical layer visits using 24 physical decoder layers.
Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as making 48 layer visits per token, but Mini has a 2,304 hidden width compared with Small’s 4,608, and routes among 128 experts per layer rather than 384. The source material calls Mini substantially smaller but does not supply a complete model file size or minimum hardware configuration. It also says earlier architectures, including Nanbeige, have explored looping designs.
The company reports that Inkling 2 Mini performs strongly on selected benchmarks, including comparisons with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It claims the model leads in 32 of 48 benchmarks and reports different outcomes on other tests, including a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Hardware and Benchmarks Still Unverified
The available information does not specify a minimum hardware configuration, complete memory requirements or measured generation speeds across consumer devices. It also does not provide independent measurements of image performance, long-context behavior or general output quality. The stated one-million-token context window is a release specification, not evidence in this material that performance remains consistent across the full window.
The benchmark results are attributed to Thinking Machines, and the supplied source lacks the full score table and outside replication. It is also unclear how much quality changes when the research checkpoint runs with fewer passes. The full model card may contain more technical and evaluation details, but it was not included in the supplied material.
Awaiting Reproducible Model Tests
The next useful evidence would include the full model card, reproducible benchmark scores and hands-on reports that identify the hardware, runtime and settings used. Testing across Transformers, llama.cpp, MLX and SGLang could show how performance varies by deployment path, while measurements on consumer systems would clarify whether local use is practical.
Independent evaluations can also test image tasks and long-context behavior, and compare the fixed three-pass model with the one- and two-pass research configurations. Until those details are available, buyers and developers should treat the company’s benchmark claims and speed expectations as vendor-reported, not independently verified.
Key Questions
What is Inkling 2 Mini?
It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image input and has about 26 billion total parameters, with about 3 billion active per token.
Can Inkling 2 Mini run locally?
The release lists support for Transformers, llama.cpp and MLX, which can be used in local inference workflows. The supplied information does not state minimum hardware requirements or confirm performance on specific consumer devices.
What does the adjustable-pass research version do?
The TML-d24-varloop-research checkpoint lets users change how many times the repeated middle layers run. The company says fewer passes may improve speed but could reduce quality; independent results for those settings were not supplied.
Are the reported benchmark results independently verified?
Not in the supplied material. The results are reported by Thinking Machines Lab, and the source provided does not include the full score table or independent replication.
Primary source: Hugging Face · via ThorstenMeyerAI.com
