Mastodon Feed: Post

Mastodon Feed

jonny@neuromatch.social ("jonny (nonvenomous)") wrote:

Love how every model just gets less and less efficient. to increase performance on the (always compromised) benchmarks by 10%, simply increase token use by 50%. Everyone knows that real intelligence works by reciting 1,000 Brothers Karamazovs worth of words in your head before saying anything. Surely there are no negative consequences to just continually scaling the amount of heat energy, er, compute per usage.

(hashtags for filters)
#LLMs #Claude

total output tokens to run a benchmark test, the grand majority of all token use for all models is "thinking" tokens. Claude sonnet 5 is a huge outlier, using 304m total tokens, next highest is GPT-5.4 with 216m, then Claude sonnet 4.6 with 197m, and the rest taper smoothly down to 117m on the low end, Opus 4.8 with 117m.