Mastodon Feed
brib@bribstodon.xyz ("brib :neofox_floof: :Nonbinary:") wrote:
@soatok @ireneista @Kye If Ed Zitron is anything to go by, it does sound like there's a lot of inference costs with the bigger AI systems (the ones everyone uses).
In particular, agentic systems seem to run on repeated cycles of "generate -> check if it looks good, perhaps with a deterministic system and perhaps with an LLM -> if not, regenerate".
And from my (limited) personal experience with generative AI, as well as the recent Copilot price hikes, this tracks, e.g. I realised I used an AI filter in Canva because it took many seconds to process when a normal filter takes less than a second.
You can still apparently do scary things even with these limitations; I saw a paper today that had combined an open-weight LLM with various context-providing tools to create a worm 😦 . I don't have enough background to scrutinise the paper properly though