Mastodon Feed
dysfun@treehouse.systems ("gaytabase") wrote:
anyway, assuming a typical backlog of infinite length, and generally that nothing is getting in its way, the real problem is quality.
i saw from @dakkar playing yesterday that hosted gemini is a little better than the self-hosted gemma model i'm using (actually he's using a slightly less compressed version of gemma than me to compare it against).
one thing i would be interested in is quantifying how much better the paid models are. because i'm always being told that the latest models change everything. this always seems to ignore the fact that the latest models are presumably to blame for all the outages we see on a near constant basis these days.
anyway i am decidedly not paying for any premium services (not least because they're all fash), but anyone with an employer who is already paying and enough of a gpu to run models locally could do this if they were interested.
as i go further down this ridiculous rabbit hole, what i find myself doing is trying to think of ways to steer it towards better output. prompting carefully seems more like trying to find the cheat code than a reliable process.
if the model produced significantly better code, it should generally work better and i could see how people could convince themselves it was 'good', though i'm fairly sure already it wouldn't meet my personal standards of good.