Mastodon Feed
Boosted by glyph ("Glyph"):
dpnash@c.im ("David Nash") wrote:
@glyph @ildiavolorosso @Firesphere @pluralistic
Case in point follows.
For context, bear in mind I have been, and remain, extremely skeptical of the long-term utility of this tech ever since ChatGPT gave me a giant wall of bullshit in response to a question I knew the answer to, back in ChatGPT 3.5.
I was testing out a then-current release of Claude Sonnet to be sure I wasn’t creating a strawman argument about the state of the art based on older impressions.I handed it a simple programming task with a graphical component (plot some data in 2D or 3D). A program like Claude Sonnet might be able to generate code for the plot, but being a large *language* model, it can’t actually analyze the plots at all competently. After getting the “base” program from Sonnet, and having it come up with an expectedly garbagepants analysis of the plot, I had it tweak the parameters of the plot a few times so it was easier for a fellow human to see that the analysis was consistently garbage. I had Claude do this a few times before collecting enough plots to make the case clear: even with a syntactically clean program, it wasn’t able to comment intelligently on the output.
The biggest problem here was *not* that Claude produced crappy analyses of graphical data, though of course, that is indeed a problem.
The biggest problem here is that I didn’t read the program itself for a while and tinker with the output it gave — something I have done many times with programs I’ve been developing — but instructed Claude to do the tinkering itself.
I had gotten sucked into an approach to debugging things that I not only did not intend to at the start, it’s one I know I generally dislike. I’d let my hands come off the wheel and let Claude take over much of the actual interaction with my source code, *even though I actively believe this is a bad idea*.
I take this as very strong evidence that personal assessments of how you are *actually* using apps like these are fundamentally unreliable, and only non-subjective impressions, agreed on in advance by the people involved, are appropriate for making judgements about how useful and productive working with gen-“AI” coding tools actually is.