Ancient Slashdot reader Gilmoure shares a report from Tom's Hardware, adding: "Huh. Looks like consuming one's own slop does not turn the wheels of industry." From the report: Amazon wants its engineers to quit chasing token counts, warning that heavy AI usage does not automatically mean better work. In updated engineering guidelines published on October 27, the company laid out how developers and managers should gauge AI's real impact -- and the takeaway is blunt: consuming more tokens is not the same as delivering more value. [...]
According to the memo, metrics such as AI adoption rates, token consumption, code changes, and code reviews become "deeply flawed" when they are disconnected from actual results. Turning them into targets, Amazon says, encourages wasteful behavior that clashes with its emphasis on "frugality and simplicity" and can raise both compute bills and software complexity.
The company's preferred yardstick is long-term results. "We believe that it is best to measure, and be measured by, long-term outcomes," Amazon wrote. That does not mean managers should ignore usage data altogether, as input metrics remain useful when examined "with care and judgment" alongside other indicators. Amazon suggests looking at both the teams using AI the least and those using it the most to determine whether adoption is translating into better delivery.
[...] The memo also pushes back on the idea that every task needs an AI tool. If current tooling does not benefit someone's work, Amazon says, that is "a finding, not a failing." Engineers who can explain where today's tools fall short are demonstrating judgment rather than reluctance to adopt AI.
According to the memo, metrics such as AI adoption rates, token consumption, code changes, and code reviews become "deeply flawed" when they are disconnected from actual results. Turning them into targets, Amazon says, encourages wasteful behavior that clashes with its emphasis on "frugality and simplicity" and can raise both compute bills and software complexity.
The company's preferred yardstick is long-term results. "We believe that it is best to measure, and be measured by, long-term outcomes," Amazon wrote. That does not mean managers should ignore usage data altogether, as input metrics remain useful when examined "with care and judgment" alongside other indicators. Amazon suggests looking at both the teams using AI the least and those using it the most to determine whether adoption is translating into better delivery.
[...] The memo also pushes back on the idea that every task needs an AI tool. If current tooling does not benefit someone's work, Amazon says, that is "a finding, not a failing." Engineers who can explain where today's tools fall short are demonstrating judgment rather than reluctance to adopt AI.
Read more of this story at Slashdot.
