aredridel@kolektiva.social ("Mx. Aria Stewart") wrote:
But seriously, this is a major LLM failure mode: following the _form_ of what everyone else does without any actual embedding of _why_. Bigger models will tend to have more of that information at hand embedded, but in a lot of domains, what we see is general reasoning-by-analogy and some raw data, not the actual subtle latent expertise that the status quo embeds.
So there be dragons.