[HN Gopher] Show HN: Experiments in AI-generation of crosswords
___________________________________________________________________
Show HN: Experiments in AI-generation of crosswords
Hi HN, I've been experimenting on-and-off over the years trying to
automatically generate crosswords [1]. Recently I've been feeling
like my results are good enough that I want to share them and see
what other people think. I'm not trying to claim that these could
appear in, say, the NYT in their current state, but honestly the
velocity of progress makes me feel like I will inevitably be able
to automatically generate NYT-quality crosswords within just a year
or so. A write-up is here:
https://abstractnonsense.com/crosswords.html And you can play the
crosswords here: https://crosswordracing.com (They should work well
on both desktop and mobile, and there's a leader-board for each
crossword if you want to leave your name when you solve one). [1]:
Just in case anyone is interested, my very first attempt at this
problem was way back in 2006! I used multiple wordlists (e.g. list
of British monarchs, with reign dates), and wrote little functions
to generate clues from each list (e.g. "British monarch who ruled
from {date1} to {date2}"). Even with randomized synonym
substitution and similar tricks, this approach was too labor-
intensive, and the results too robotic, for it to work well. Can't
complain though, that project led to me getting hired as the first
engineer at Justin.TV!
Author : abstractbill
Score : 24 points
Date : 2024-12-23 19:27 UTC (3 hours ago)
(HTM) web link (abstractnonsense.com)
(TXT) w3m dump (abstractnonsense.com)
| vunderba wrote:
| Not bad.
|
| As someone who has dabbled in AI generated crosswords I found
| that providing samples of "good crossword clues" (which I curated
| from historical NYT _monday_ puzzles) as part of the LLM context
| helped tremendously in generating better clues.
|
| There was also a Show HN for a generative AI crossword puzzle
| system a few months ago so I'll include what I mentioned there:
|
| _Part of the deep satisfaction in solving a crossword puzzle is
| the specificity of the answer. It 's far more gratifying to
| answer a question with something like "Hawking" then to answer
| with "scientist", or answering with "mandelbrot" versus "shape"._
|
| So ideally, you want to lean towards "specificity" wherever
| possible, and use "generics" as filler.
|
| Link:
|
| https://news.ycombinator.com/item?id=41879754
| abstractbill wrote:
| Thanks. Yes, specificity of solutions seems like a good metric
| to optimize for.
|
| In some of my crosswords I get clues that are specific in
| clever ways (e.g. one of these has "Extreme, not camping" which
| I thought was really strange until I found the answer "intense"
| and was very impressed by that level of wordplay from an LLM!)
| furyofantares wrote:
| I've tried to get o1 to generate Xordle puzzles.
|
| Warning: post contains a spoiler for a recent Xordle.
|
| Xordle is Wordle with two target words that share no letters in
| common. Additionally, there is a "free clue" given at the start,
| and all three words are thematically linked. It's not always a
| straightforward link, for example a recent puzzle had the starter
| word 'grief' and targets 'empty' and 'chair'. All puzzles today
| are selected from user submissions.
|
| o1 is the first model that's been able to solve Xordles reliably,
| or to generate valid puzzles at all. It's well-known that these
| things are massively handicapped for this type of task due to
| tokenization.
|
| But since o1 can in fact achieve it, I wanted to see if I could
| get it to make puzzles that are at all satisfying. Instead it
| makes very bland puzzles, with straightforward connections and
| extremely broad themes.
|
| Prompting can swing the pendulum too far in the other direction,
| to puzzles where the connection is contrived and impossible to
| see even after it's solved. As I've often experienced with LLMs,
| being able to hit either side of a target with prompting does not
| necessarily mean you can get it to land in the middle, and in
| fact I have had no success in doing so with this task.
|
| This is one of the most basic examples I know of lack of
| creativity or "taste" to an LLM. It is a little hard for a human
| to generate two 5-letter words with no overlap, but it is
| extremely easy for a human to look for a thematic connection
| among 2-3 words and say if it's satisfying. But so far I've been
| totally unable to make the LLM make satisfying puzzles.
|
| edit: Nothin' like making a claim about LLMs to get one up off
| one's ass and try to prove it wrong immediately. I'm getting some
| much better results with better examples now.
| IanCal wrote:
| Have you tried using an llm to say whether the puzzles are good
| or not?
| abstractbill wrote:
| Great observation, yeah, I've had very similar experiences with
| prompting, exactly as you said -- one direction giving very
| bland literal clues, and the opposite direction giving clues
| that are a stretch even when you know the answer!
| gowld wrote:
| The "American" grids aren't American. An American grid almost
| always has 2 answers (both directions) per square.
| abstractbill wrote:
| Oh that's really interesting thanks! That would actually be an
| easy constraint to add too.
| korymath wrote:
| Great post.
|
| Funny, I just posted this to X
|
| 2025 GenAI challenge
|
| Create a 5x5 crossword puzzle with two distinct solutions. Each
| clue must work for both solutions. Do not use the same word in
| both solutions. No black squares.
|
| I try with each new model that lands. Still can't get it.
| alberto_balsam wrote:
| Do you know if there is a solution to this by humans? I'd be
| interested in seeing it.
| echelon wrote:
| That's algorithmically hard.
|
| Ask the LLM to generate a program to solve the problem.
| dgreensp wrote:
| I found this article a bit disappointing.
|
| The link at the bottom doesn't work.
|
| The grids shown do not follow the well-known rules of (American)
| crosswords: every square is part of two words of three or more
| letters each.
|
| Coming up with a pattern of black squares, and writing good
| clues, are two parts of making a crossword puzzle that are IMO
| fun and benefit from a human touch, and are not overly difficult.
| There are also databases of past clues used in crossword puzzles
| (eg every NY Times clue ever, and various crossword dictionaries)
| for reference and possible training. If you don't care about
| originality (or copyright) and want quality clues, you can just
| pull clues from these. If you do care about all those things, you
| can surface the list of clues used in the past to the human
| constructor and let them write the final clue. Or you can try to
| perfect LLM clue-writing. In my experience, LLMs are terrible at
| clues. Like sometimes if I try to give it feedback about a clue,
| it will just work the feedback into the clue... it's a little
| hard to describe without an example, but basically it doesn't
| seem to understand the requirements of a clue and the process of
| a solver looking at a clue and trying to come up with an answer.
|
| Coming up with an interlocking set of fun, high-quality words and
| phrases is the hard part. I agree that LLM wordlist curation is a
| great idea, and I started playing around with that once.
|
| Beyond that, I don't think LLMs can help with grid construction,
| which is a more classic combinatorial problem.
___________________________________________________________________
(page generated 2024-12-23 23:01 UTC)