Mastodon Feed: Post

Mastodon Feed

jonny@neuromatch.social ("jonny (nonvenomous)") wrote:

> "GPT-6 score 99% on the AGI test
> looks inside AGI test
> the arcade pack CD-ROM you'd shake out of a cereal box in 2003

[bar plot of GPT-6 at 99%, opus 5 at 30% score. score is undefined. output tokens are undefined.] ARC-AGI-3 tests how well agents learn as they solve unfamiliar interactive tasks. GPT‑6 Astra saturates the eval, scoring 99.9%. The average human tester scored 48%. GPT‑6 Astra was measured with our responses API harness⁠, which better reflects real-world performance than the original benchmark harness, which discards past reasoning and past messages. With this harness, we estimate Sol would score in the ballpark of ~30%.
literally a gameboy looking color matching game where you flip cards to match a color grid that is printed on the board itself. like. "red color square says make these squares around it red" level shit.