🔍 Read the full analysis: Why AI Metrics Need Context: What Amazon’s Bets Bring Into View on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Amazon published engineering guidance on October 27 saying AI usage measures such as token counts can be useful signals but are flawed performance targets. The memo arrives as reporting describes a retreat from several Nova models and Amazon’s Anthropic investment has gained substantial paper value, underscoring the difference between activity and outcomes.
Amazon published new engineering guidance on October 27 telling software teams not to treat token use, AI adoption or other activity measures as performance targets. The memo says those metrics can correlate with useful results today, but using them to score work can raise costs and complexity without showing whether customers or the business benefit.
The guidance was written by Amazon’s most senior engineers and published on the company’s news site. It applies to software engineers and deliberately avoids defining their roles around particular AI tools, saying that such definitions could quickly become outdated. Instead, it centers the role on impact on customers, the business and the team.
The memo distinguishes between tracking activity and setting targets. Teams may use token counts, AI-tool adoption, code changes and reviews as inputs, it says, but those measures are “deeply flawed” as targets because they are disconnected from outcomes. The memo warns that target-setting can drive up costs and add complexity. It also says the metrics currently “correlate well with the outputs that matter”, so its position is not that teams should stop using AI.
Instead, the guidance calls for judgment. Leaders should examine both lower and higher AI adopters, asking why tools help some teams and not others, and whether heavy use is producing better work. The memo says that when current AI tools do not help a task, that is a finding to document and share—not a performance failure. It also asks leaders to use the tools on their own production work and report where they helped and where they did not.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
Why Activity Scores Can Mislead
The memo matters beyond Amazon because companies are trying to measure whether workplace AI improves productivity. Usage is easy to count; customer impact, quality and time saved are harder to measure and may take longer to appear. Treating the first set of numbers as proof of the second can reward activity without establishing value.
Amazon’s guidance offers a practical distinction: keep usage data as a diagnostic signal, but do not make it the goal. A high token count could indicate productive work, unnecessary tool use or a task that required more effort. Without an outcome and a comparison point, the count alone cannot tell managers which explanation applies. The memo’s approach puts the burden on leaders to examine results and reasons, rather than assuming that more AI use is better.
The stakes include both cost and work quality. The memo warns that usage targets can increase spending and system complexity. For engineers, its approach also leaves room to report when AI does not help, which could reduce pressure to use a tool simply to meet a numerical expectation. Those are the memo’s stated concerns; it does not provide measured company-wide savings or productivity gains.
Popular right now
Vitamix 5200 Professional-Grade Blender, 64 oz. Container, Black
Ninja Specialty Coffee Maker CM401, 10-Cup Carafe, Fold-Away Frother, Black
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
As an affiliate, we earn on qualifying purchases.
Nova and Anthropic Set Different Tests
The memo arrived amid two contrasting developments in Amazon’s AI portfolio. Business Insider reported in July that most flagship Nova models had been moved to “keep the lights on”. Amazon Web Services’ lifecycle information, according to the source material, lists Nova Canvas, Nova Reel and the first Nova Sonic as retired. These details suggest changes to parts of Amazon’s model offering, but do not establish that Amazon has abandoned its broader AI work.
Amazon’s investment in Anthropic provides a different kind of scorecard. The source material says Amazon invested about $18 billion and that a June 30 filing valued its holdings at about $190 billion. That is a reported paper valuation, not cash proceeds or proof of operating returns. It also reflects the value of an investment in a company Amazon does not control, rather than the performance of Amazon’s own models.
There was an earlier warning about activity-based incentives. The Financial Times reported in May that some Amazon employees assigned unnecessary tasks to an internal agent to raise token counts. Those accounts were anonymous and are reported allegations, not independently confirmed facts in the supplied material. Amazon did confirm to Business Insider at the end of May that its KiroRank leaderboard had been deprecated, describing it as an informal employee-created tracker that was not intended to encourage AI use for its own sake.
““Tokenmaxxing is at odds with frugality.””
— Amazon’s engineering memo
What the Metrics Cannot Prove
The memo does not publish company-wide data showing how much AI improves productivity, lowers costs or changes software quality. Its statement that activity measures currently correlate well with important outputs is not accompanied in the supplied material by a quantified correlation, timeframe or method. The strength and limits of that relationship remain unclear.
The Nova developments also need careful boundaries: reports about models moved to maintenance and lifecycle listings for retired products do not, on their own, explain Amazon’s full model strategy or the commercial performance of each offering. The Anthropic figure is a reported valuation of holdings on a filing date, not a realized gain. And the accounts of employees inflating token use were reported anonymously; Amazon’s confirmation that KiroRank was deprecated does not confirm every detail of those accounts.
Amazon’s third-quarter results, scheduled for October 29, may provide financial context, but the source material does not establish that the results will disclose specific returns from Nova, Anthropic or internal AI tools. It is also unclear how Amazon will evaluate the new guidance across teams or whether it will publish outcome measures.
Results Due October 29
Amazon is scheduled to report third-quarter results on October 29, 2026, one day after publishing the engineering guidance. Those results will be the next near-term company update, though it remains unknown whether Amazon will break out returns or costs tied to particular AI products and investments.
For now, the memo sets out a management principle rather than a new scoring system: examine adoption patterns, investigate why they differ and judge whether work produces meaningful outcomes. The open test is whether Amazon can turn that principle into clear evaluation practices without replacing one incomplete leaderboard with another.
Key Questions
What did Amazon’s new AI guidance say?
Amazon said measures such as token use and AI-tool adoption can be tracked, but should not be treated as performance targets in place of results.
Is Amazon telling engineers to stop using AI?
No. The memo says activity measures currently correlate with useful outputs and does not reject AI use. It asks teams to judge whether tools improve the work and to explain when they do not.
What is tokenmaxxing?
In this reporting, it refers to using an AI tool for unnecessary tasks to increase token counts, which can make activity look higher without showing that the work produced more value.
What do Amazon’s Nova and Anthropic developments show?
They are different kinds of evidence: reports describe changes to several Nova models, while Amazon’s Anthropic stake has a reported paper valuation. Neither alone measures the overall business impact of Amazon’s AI strategy.
When will Amazon report its next results?
Amazon is scheduled to report third-quarter results on October 29, 2026. The supplied information does not say whether the report will include separate AI-related performance figures.
Source: ThorstenMeyerAI.com
