The Small-Model Trade-Off: Nemotron 3.5 Bolt’s Token Costs

🔍 Read the full analysis: The Small-Model Trade-Off: Nemotron 3.5 Bolt’s Token Costs on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

NVIDIA released Nemotron 3.5 Bolt 74B-A9B on October 15, with an Artificial Analysis Intelligence Index score of 21.4, near the 22.9 earned by the company’s much larger Nemotron 3 Ultra. But the benchmark comparison comes with a practical cost: Bolt used about 82,000 output tokens per index task, nearly three times Ultra’s 28,000, and Artificial Analysis has not yet reported its inference speed or hosted price.

Bolt is the first model in NVIDIA’s new Bolt family. It is an open-weight, mixture-of-experts model with 74 billion total parameters and 9 billion active per token. It accepts text and images, produces text, supports reasoning, and has a stated context window of 262,144 tokens. Artificial Analysis evaluated the reasoning version.

Against NVIDIA’s previous flagship, Nemotron 3 Ultra 550B-A55B, Bolt trails by 1.5 Intelligence Index points despite having roughly one-seventh as many total parameters and one-sixth as many active parameters. Its score is 8.6 points above Nemotron 3 Super, the company’s 120B-total, 12B-active model released in March, according to the supplied benchmark figures.

The result varies by task. Bolt scored 15.5% on AutomationBench-AA, which tests multi-step workflows using software tools, compared with Ultra’s 3.0%. But Ultra scored higher on long-context reasoning, knowledge-related measures and several other tests. Both models scored close to zero on Terminal-Bench 4.0, so the results do not support treating Bolt as a capable shell-automation agent.

Token Use Changes the Local Cost

The headline score measures benchmark performance, not how quickly or cheaply a user can get an answer. Artificial Analysis reports that Bolt generated about 230 million output tokens across the Intelligence Index, describing it as very verbose compared with a 100 million-token median for open-weight models of its size. Its per-task figure of 82,000 tokens is about 2.9 times Ultra’s.

For organizations running a model on their own hardware, generated tokens consume compute time. The source’s examples are arithmetic estimates, not measured Bolt speeds: at 100 tokens per second, 82,000 tokens would take about 14 minutes; at 50 tokens per second, about 27 minutes. The same calculation puts Ultra’s 28,000 tokens at under five minutes at 100 tokens per second. Actual completion times depend on hardware, implementation and workload.

That trade-off could matter differently by use. A batch job that runs unattended may tolerate lengthy reasoning, while an interactive assistant is judged partly by how long a person waits. Bolt’s comparatively strong AutomationBench result may interest teams testing tool-using workflows, but the benchmark does not establish that it will be fast or reliable in their own systems.

A Strong U.S. Model, Not the Class Leader

Artificial Analysis ranks Bolt fourth among models with fewer than 100 billion total parameters, and first among U.S. models in that group, according to the supplied account. It trails Qwen3.8 27B, listed at 33.7; K2 Horizon MoVA 36B-A4B at 25.3; and G9v3-39A5B at 21.8. The comparison is based on each model’s listed best configuration, and the models differ in size, capabilities and token use.

The ranking gives a more qualified picture than the near-match with Ultra alone. The leading model’s score is 12.3 points above Bolt’s, while K2 Horizon MoVA is listed as scoring four points higher with fewer parameters and fewer tokens. Bolt’s reported differentiators include image input and its standing among U.S. models in the size class; those distinctions may matter to buyers with specific capability or provenance requirements, but do not make it the overall class leader.

Bolt is available as open weights for self-hosting. The supplied source says Artificial Analysis listed no hosted API provider at the time of its report, leaving no listed API price or measured hosted speed. It also says the model page classed the license as permissive without naming the license there.

““Bolt is not a shrunken Ultra.””

— Thorsten Meyer, ThorstenMeyerAI.com

Speed and Deployment Costs Unmeasured

There is no reported Artificial Analysis speed measurement for Bolt in the supplied material, so the estimated 14- or 27-minute completion times are illustrative calculations rather than observed results. Real inference time will vary with hardware and serving setup. No hosted API provider or price was listed, so the financial cost of running Bolt through a third party cannot be compared here.

The benchmark scores also do not establish how Bolt performs on a particular company’s tasks. The source reports a negative score of −19 on AA-Omniscience, a benchmark that rewards correct answers and penalizes confident wrong ones, but the supplied material ends before giving further detail about that result. The license is described as permissive, but its name is not stated. Those points limit conclusions about knowledge reliability and the exact terms for deployment.

Watch for Speed and API Results

The next useful evidence will be independent or provider-reported measurements of tokens per second, task completion time and hardware requirements, alongside any hosted API launch and pricing. Those figures would show whether Bolt’s lower active-parameter count translates into a practical speed or cost advantage despite its high token output.

For now, readers can compare the published benchmark profile with their own needs: Bolt’s reported agentic-workflow result, image input and open-weight availability may warrant testing, while its long-context and knowledge scores, verbose outputs and unmeasured speed call for caution. Performance on real deployments remains to be established by evaluations that report both answer quality and time or compute per task.

Key Questions

What is Nemotron 3.5 Bolt?

It is NVIDIA’s first model in the Bolt family, released as open weights. The mixture-of-experts model has 74 billion total parameters and 9 billion active per token, accepts text and images, and outputs text.

How close is Bolt to Nemotron 3 Ultra?

Artificial Analysis gives Bolt an Intelligence Index score of 21.4, compared with 22.9 for Nemotron 3 Ultra. The scores are close, but individual benchmark results show different strengths and weaknesses.

Why are Bolt’s output tokens a concern?

The reported 82,000 output tokens per index task are nearly three times Ultra’s 28,000. When running locally, more generated tokens can mean more compute time; the actual duration depends on the hardware and setup.

Is Bolt the top model in its size class?

No. The supplied Artificial Analysis ranking places Bolt fourth among models under 100 billion total parameters, though first among U.S. models in that group.

Is Bolt available through a hosted API?

The supplied report says Artificial Analysis listed no hosted API provider for Bolt at the time. It therefore gives no hosted price or measured API speed.

Source: ThorstenMeyerAI.com