Mastodon Feed
Boosted by jonny@neuromatch.social ("jonny (nonvenomous)"):
jedbrown@neuromatch.social ("Jed Brown") wrote:
We all hate this
@gregkh dropping necessary bombs about LLMs in Linux security and development.
https://www.youtube.com/watch?v=NnV%5FcWeoo5Q
The purported 79 vulnerabilities were mostly bogus (not a bug, fabricated data) or had already been patched; the few real (all very minor) bugs took him an hour to fix. He says "do not panic", and emphasizes that time to apply patches is the main vulnerability these days (not an LLM problem).
"These are pattern-matching tools", "they do not have intent". (Vindicating to @emilymbender and @timnitGebru re "stochastic parrots"; it was accurate at the time and accurate today, protestations from boorish AI boosters notwithstanding.)
New code is 50% wrong, generated code is littered with new vulnerabilities; need to reject a lot but believes review processes are adequate to prevent a shipping a flood of LLM-generated vulnerabilities.
High false-positive rates make the models incredibly irritating. Cf. Coverity's post-mortem with a deterministic tool that had low false-positive rates.
Push back hard [on corporate marketing to developers]
With regard to higher CVE rates:
We'll grind it down like we did with the fuzzers.
If you want to look for bugs with an LLM (which amounts to fuzzy pattern-matching), use a local model and never upload to platforms.
I have banned LLMs from driver staging, unless you have the hardware and can prove you have tested the patch. That's not what staging is for. Staging is to learn how to do development and get involved with our community. [...] Kernel development is all about trust. If I take patches from you and I don't know who you are, now I am responsible for that patch. [...] You need to build up trust.
It's put a lot of additional burden on us, as maintainers.