[HN Gopher] Show HN: ChessCoach - A neural chess engine that com...
___________________________________________________________________
Show HN: ChessCoach - A neural chess engine that comments on each
player's moves
Author : cbutner
Score : 82 points
Date : 2021-10-05 20:31 UTC (2 hours ago)
(HTM) web link (chrisbutner.github.io)
(TXT) w3m dump (chrisbutner.github.io)
| cbutner wrote:
| This took about a year and a half - a little over a year coding
| in between experiments and training.
|
| It's a chess engine with a primary neural network just like
| AlphaZero or Leela Chess Zero's, but it adds on a secondary
| "commentary decoder" network based on Transformer architecture to
| comment on positions and moves. All of the code and data for
| training and search is from scratch, although it does use
| Stockfish code to generate legal moves and manage chess
| positions.
|
| You can watch it play on Lichess here:
| https://lichess.org/@/PlayChessCoach/tv or challenge it here:
| https://lichess.org/?user=PlayChessCoach#friend, and see its
| commentary in spectator chat. It only plays one game at a time,
| so you may need to wait a little bit. It's fairly strong (~3450
| rating, roughly on par with Stockfish 12 or SlowChess Blitz 2.7),
| but you can set up a position when challenging it so that it's
| missing a couple pawns or a piece (Variant: From Position).
|
| I ended up writing much more about it than I expected. If you're
| into the technical side of chess or machine learning, beyond the
| linked overview, there's:
|
| High-level explanation:
| https://chrisbutner.github.io/ChessCoach/high-level-explanat...
|
| Technical explanation:
| https://chrisbutner.github.io/ChessCoach/technical-explanati...
| (including code pointers)
|
| Development process:
| https://chrisbutner.github.io/ChessCoach/development-process...
| (including timelines, bugs and failures)
|
| Data: https://chrisbutner.github.io/ChessCoach/data.html
| (including raw measurements and tournament PGN files)
|
| And the code is here: https://github.com/chrisbutner/ChessCoach
| (C++ and Python, GPLv3 or later)
|
| Happy to answer any questions!
| evanhu_ wrote:
| I'm super impressed with what you've managed to create, do you
| have any further plans with this project? I'm curious now that
| it's finished and documented to such an extent will you try to
| bring it publicity and actual usage or was this just a passion
| project. Thanks
| cbutner wrote:
| Thank you! I do get that itch to jump in and improve things
| whenever I see it lose a game, but I don't have further plans
| (development or commercial) in the near-term. The goal
| originally was to see whether I liked ML, to decide on my
| next industry/career move, but there was a lot of "one more
| month".
|
| I'm actually hopeful that some search techniques such as
| SBLE-PUCT[1] or better derivations can make their way into
| other open source projects, but they've had big teams working
| for a while on similar, often better ideas, so we'll have to
| see.
|
| [1] https://chrisbutner.github.io/ChessCoach/high-level-
| explanat...
| mxwsn wrote:
| So, do you like ML?
| cbutner wrote:
| Haha - I dislike how much of a black box it is, despite
| the statistical basis (for example, the back and forth on
| batch normalization rationale). But lots of interesting
| problems and tech to dig into.
| iooi wrote:
| > It only plays one game at a time, so you may need to wait a
| little bit
|
| Why this limitation? Is it fairly computationally expensive to
| run?
| cbutner wrote:
| Yes, each bot uses a v3-8 Cloud TPU VM, and tries to be
| constantly playing a game. The search tree is also very
| memory-hungry. And right now it's also using the Python API
| for TensorFlow, which is likely wasting a lot of potential.
|
| Lots of room for improvement!
| egberts1 wrote:
| I look forward to reading things like "a classic Boris move" or
| Fisher or any of the great chessmasters.
| cbutner wrote:
| It does tend to name-drop: often famous names, but also just
| "Jeff".
|
| And if you spice up the commentary sampling parameters, it gets
| even more inventive, making up names, and saying that "the rook
| is pinning Fischer against the king".
| sobellian wrote:
| One of my pet peeves is when folks compare human elo ratings from
| FIDE to (presumably) engine elo ratings from CCRL, a league for
| chess engines. They're not the same! This is like comparing
| points between your local youth soccer league and La Liga. All we
| need to say is that this engine is only 100 elo behind Stockfish.
| That is akin to occasionally beating God in a game of chess.
| kthejoker2 wrote:
| Very cool - one bug, it seems to want to show line variations in
| the chat, but it only shows the last move, which is then linked
| to the actual move in the game.
|
| e.g. on move 22
|
| I don't know why black played this. As black, I would've played
| 27...Kf8 to protect f6 and e5.
|
| the 27..Kf8 is a link ... but the game hasn't proceeded to move
| 27 yet, so the link goes nowhere.
|
| Clearly it's trying to show some line that ends with 27....kf8 ?
|
| Anyway, the commentary itself is pretty excellent, great job!
| cbutner wrote:
| Yeah the auto-linking is just Lichess doing its best, even when
| the bot's talking nonsense sometimes.
|
| And thank you!
| prezjordan wrote:
| Remarkably cool. My only bit of feedback is that it's unfortunate
| so much of the commentary uses masculine pronouns (I found it
| distracting). Is that just a consequence of the training data?
| cbutner wrote:
| Yeah, that's a massive problem with the natural language domain
| all across machine learning.
|
| Unfortunately it's very difficult to track down training data
| for chess commentary in the first place, let alone trim down
| biases. For reference, I was able to gather about 1 million
| samples, but it really needs a billion.
|
| Hopefully through data augmentation and better general
| intelligence models we can make better progress on bias issues
| soon, as that's a huge problem when we start trusting AI models
| too much in life.
| prezjordan wrote:
| Appreciate the honesty here. Pretty wild how natural this
| model feels with 1 million samples.
| cbutner wrote:
| Sometimes it seems really accurate (like the cherry-picked
| GIF in the overview docs) and sometimes really off.
|
| I think for the most part, it knows more than it lets on,
| but finding the right sampling methods (or better yet,
| generalized search) to generate the best comments is a
| tough problem because it's difficult to evaluate quality.
|
| There's some info on the sampling methods here:
| https://chrisbutner.github.io/ChessCoach/high-level-
| explanat...
| EddieLomax wrote:
| If I click on "Challenge", and set up a game with unlimited time
| control (or correspondence), the challenge is declined with:
|
| > This time control is too fast for me, please challenge again
| with a slower game.
|
| Is that expected?
| cbutner wrote:
| Oh, that message is a little backwards, but the main bot only
| accepts challenges from 1+0 or 0+1 up to 15+10 time control.
|
| You can challenge
| https://lichess.org/?user=chesscoachclassical#friend to 30+20.
|
| Unfortunately, neither of them support correspondence.
| eeegnu wrote:
| Just tried it out, really interesting to read post game:
| https://lichess.org/dKbbqymG/
|
| I don't think this comment was accurate:
|
| > 23. c4: "White tries to get his pawns moving. I am still
| thinking that I have to move my bishop, but that is too slow. I
| think white should have moved his king back to c3 to prevent my
| pawn from becoming a passer."
|
| The idea behind moving the pawn was because black playing c4
| would have instantly lost both pawns since the bishop has nowhere
| safe to go where it still defends c2 (Bh7 leading to g6 losing
| the piece.) I don't think the king could have made it to the c
| file in time to stop that.
|
| > 34... Rxg4: "Now I can't stop him from advancing his pawn."
|
| Is this meant to be speaking from whites perspective (since black
| just got the unopposed h file pawn.)
|
| Interestingly only 10. O-O links with a move, I think it would be
| really helpful if they all linked to moves. Also i'm really
| excited for this kind of analysis! It would be really cool if
| request a computer analysis eventually generated such analyses
| trying to figure out each sides ideas.
___________________________________________________________________
(page generated 2021-10-05 23:00 UTC)