Mastodon Feed
Boosted by baldur@toot.cafe ("Baldur Bjarnason"):
vathpela@infosec.exchange ("Farce Majeure") wrote:
Today in being forced to use LLMs because "vulnerability" disclosures have been produced by them:
- no version numbers or git revisions anywhere
- the reports are malformed markdown which has then been filed into a jira ticket which doesn't accept markdown anyway
- Many of the CVSS scores look like: "CVSS:3.1/AV/AC:L/..." where it should be "AV:L", but the bot, which again only knows statistically what a CVSS score looks like, is generating gibberish.
- the diffs for "suggested fixes" have been generated by something that doesn't actually know anything about how diffs are formed, either in terms of process or meaningfully in structure:
- they're missing file names, line numbers, and function names
- all leading whitespace, including after "-" or "+" line markers, is missing
- in some of the diffs, hyphens (especially for delete lines) are just missing? But sometimes "--" has been combined to "—"
I want my money back.