(C) BoingBoing
This story was originally published by BoingBoing and is unaltered.
 . . . . . . . . . .



AI models benchmarked with Pac Man [1]

['Rob Beschizza']

Date: 2026-09-30  

PacBench is an unusual benchmark for AI models: "Create a Pac-Man game in a single html page." Each entry shows the platform, the model, how long it took, how many tokens it used, and the cost. And unlike the pelican, we can play the results.

Claude does best, with Opus 5.5 getting it nearly perfect in 9 minutes at a cost of $1.99. Using Fable just runs up the tab ($5.87!), takes longer, and gets a couple of things wrong. Grok and GPT did pretty good (and the latter bangs it out for pennies) but it's already starting to feel like "Adobe Flash implementation circa 2005" and things get strange and broken therafter.

Here's creator Jon Clegg, answering questions from Hacker News commenters:

The game mechanics are shockingly hard for them to get right. You also reveal some of their personality traits. The GPT‑6 and Astra models tend to include a lot of cringe copy as well as their own personal graphics style.

I find it interesting that even the most accurate results don't implement Pac-Man's low-resolution pixel art, even when they do add scanlines. I wonder if this represents something interesting about Pac Man's presence in training data (a heavily written-about game, but written-about in very 1980s ways) or reflects something else about how models are made (i.e. an otherwise accurate implementation of an obscure game will get the same treatment.)

Contemporaneous books about beating Pac Man (the first I read, as a youngster, was "The Winners Book of Video Games" by Craig Kubey) describe the game's mechanics in insane detail without describing anything about the technology. From Kubey's book, the Pac Man fruits:

[END]
---
[1] Url: https://boingboing.net/2026/09/30/ai-models-benchmarked-with-pac-man.html

Published and (C) by BoingBoing
Content appears here under this condition or license: Creative Commons BY-NC-SA 3.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/boingboing/