AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Why Consider Inkling 2 Mini For Your Local AI Setup? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters, with about 3 billion active per token. It supports several inference runtimes, but the supplied information does not establish independent benchmark results, practical hardware requirements or real-world speed.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local AI setups and datacenter inference. The company says it has about 26 billion total parameters, activates about 3 billion per token, and supports tools including Transformers, llama.cpp, MLX and SGLang; independent performance results and consumer-hardware requirements are not established in the supplied material.

The model accepts text and image input and is described as having a 1 million-token context window, a relevant consideration when planning local AI workloads on a Mac mini. Its architecture has 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers are reused for three passes, producing 48 logical layer visits per token. According to the company’s technical description, this shares weights across repeated layers rather than assigning a distinct set of weights to every logical position.

Inkling 2 Mini uses a mixture-of-experts design: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says the model’s expert weights are distributed in NVFP4 and MXFP8 mixed-precision formats, and that it includes a DSpark drafter intended to speed generation with a small memory cost. The fixed-pass release is supported by Transformers, llama.cpp and MLX; SGLang is listed for day-one inference support in datacenter deployments, among the considerations for building a local AI workstation.

A separate research checkpoint, TML-d24-varloop-research, allows users to change the number of passes through the repeated layers. The company says it can run with one or two passes instead of three, potentially increasing speed with a possible quality trade-off. Its description also says all 48 visits in the fixed model require execution, with each visit retaining its own key-value and short-convolution state.

At a glance
announcementWhen: Released; the supplied source material…
The developmentThinking Machines Lab released Inkling 2 Mini, a multimodal model aimed at local and datacenter inference, alongside a research version with adjustable layer-loop passes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

What Local AI Users Gain

The release gives people evaluating local models another option that combines image and text input with a large claimed context window and support for several commonly used inference tools. That range of runtimes may make it easier for developers to test the model in different environments, but compatibility does not by itself show that it will run well on a particular computer.

The architecture illustrates a trade-off relevant to local deployment: repeated layers reuse weights, while expert routing activates only part of the expert pool for each token. Those design choices may balance capacity and computation, but they do not establish the model’s actual memory footprint, generation speed or output quality. Local users still need device-specific measurements before deciding whether it fits their hardware or workload.

The adjustable-pass research version could also help researchers study how reducing computation affects speed and quality. However, the available description does not provide independent results for one- or two-pass configurations, so the practical value of that option remains to be tested.

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Ninja Foodi Smart 6-in-1 Indoor Grill and Air Fryer, 4 qt, FG551

Ninja Foodi Smart 6-in-1 Indoor Grill and Air Fryer, 4 qt, FG551

As an affiliate, we earn on qualifying purchases.

How Inkling Reuses Its Layers

The central design feature is the reuse of the 12 middle physical layers for three passes. The release says each pass begins with the same input from the first six layers, while learned messages carry information between passes and a learned gate controls how much information is passed along. This lets the model make 48 logical layer visits using 24 physical decoder layers.

Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as making 48 layer visits per token, but Mini has a 2,304 hidden width compared with Small’s 4,608, and routes among 128 experts per layer rather than 384. The source material calls Mini substantially smaller but does not supply a complete model file size or minimum hardware configuration. It also says earlier architectures, including Nanbeige, have explored looping designs.

The company reports that Inkling 2 Mini performs strongly on selected benchmarks, including comparisons with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It claims the model leads in 32 of 48 benchmarks and reports different outcomes on other tests, including a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Hardware and Benchmarks Still Unverified

The available information does not specify a minimum hardware configuration, complete memory requirements or measured generation speeds across consumer devices. It also does not provide independent measurements of image performance, long-context behavior or general output quality. The stated one-million-token context window is a release specification, not evidence in this material that performance remains consistent across the full window.

The benchmark results are attributed to Thinking Machines, and the supplied source lacks the full score table and outside replication. It is also unclear how much quality changes when the research checkpoint runs with fewer passes. The full model card may contain more technical and evaluation details, but it was not included in the supplied material.

Awaiting Reproducible Model Tests

The next useful evidence would include the full model card, reproducible benchmark scores and hands-on reports that identify the hardware, runtime and settings used. Testing across Transformers, llama.cpp, MLX and SGLang could show how performance varies by deployment path, while measurements on consumer systems would clarify whether local use is practical.

Independent evaluations can also test image tasks and long-context behavior, and compare the fixed three-pass model with the one- and two-pass research configurations. Until those details are available, buyers and developers should treat the company’s benchmark claims and speed expectations as vendor-reported, not independently verified.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image input and has about 26 billion total parameters, with about 3 billion active per token.

Can Inkling 2 Mini run locally?

The release lists support for Transformers, llama.cpp and MLX, which can be used in local inference workflows. The supplied information does not state minimum hardware requirements or confirm performance on specific consumer devices.

What does the adjustable-pass research version do?

The TML-d24-varloop-research checkpoint lets users change how many times the repeated middle layers run. The company says fewer passes may improve speed but could reduce quality; independent results for those settings were not supplied.

Are the reported benchmark results independently verified?

Not in the supplied material. The results are reported by Thinking Machines Lab, and the source provided does not include the full score table or independent replication.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

10 Best Home Theater Projectors For AI And Big-Screen Entertainment In 2027

A source-based guide to 10 home theater projector picks, their stated features, and the resolution, room and streaming tradeoffs buyers should check.

Apple Broke An Essential Presentation Feature Of iPads

Educator Teddy Svoronos reports that iPadOS 27 can show other open iPad apps on an AirPlay screen during presentations.

Street Fighter™ 6 Climbing The Steam Charts

Street Fighter™ 6 has climbed to the 8th position on Steam’s most-played games, reaching a peak of over 34,800 players, signaling strong player interest.

Nvidia’s First GPU: GeForce 256

Nvidia released the GeForce 256 on Oct. 11, 1999, and marketed it as a GPU, helping establish a term and product line still used today.