[HN Gopher] What happens when you make a move in lichess.org?
___________________________________________________________________
What happens when you make a move in lichess.org?
Author : dreis_sw
Score : 148 points
Date : 2024-10-23 08:13 UTC (14 hours ago)
(HTM) web link (www.davidreis.me)
(TXT) w3m dump (www.davidreis.me)
| perihelions wrote:
| - _" While these moves could be calculated client-side, providing
| them server-side ensures consistency - especially for complex or
| esoteric chess variants - and optimizes performance on clients
| with limited processing capabilities or energy restrictions."_
|
| Just a wild guess: might be intended to lower the implementation
| barrier for new open-source software clients on new platforms,
| and/or preempt them from implementing subtle logic bugs that only
| show up much later.
|
| The rules of chess are a bit tedious to implement, and you can
| easily get tired and code an edge-case bug that's almost
| invisible. Lichess itself did this--it once had a logic error
| that affected a very tiny number (exactly 7) of games,
|
| https://github.com/lichess-org/database/issues/23 ( _" Before
| 2015: Some games with illegal moves were recorded"_)
|
| (I apologize I couldn't find the specific patch that fixed this)
| epcoa wrote:
| > and/or preempt them from implementing subtle logic bugs that
| only show up much later.
|
| Validating a submitted move is distinct from listing valid
| moves. I assumed the server would need to validate regardless
| of providing a list to the client.
| perihelions wrote:
| It's still duplicated work, and clients are likely to get it
| wrong and create more work for both devs.
| xmprt wrote:
| For those curious about the illegal move, it seems like it's
| allowing queen side castling through the king side rook (or
| vice versa). eg. if this is the first rank, R _ _ R K _ _ _,
| then you could make the move O-O-O and end up with _ _ _ R K _
| _ _
|
| Naturally, it's not possible to view this move anymore, but
| this game (https://lichess.org/XDQeUk6j#48) has everything up
| until the last legal move right before the illegal castling
| happened.
| ARandumGuy wrote:
| I can see why that only appeared in 7 games. It's pretty rare
| to see a rook in between a king and another rook that are
| otherwise legally able to castle. Even rarer for someone to
| get into that position and actually try to castle.
|
| Also that linked game is pretty entertaining. It's not a good
| game, but it can be fun watching lower ranked players make
| moves that you'd never see in higher level games. Like, who
| plays Bb5+ against the Scandinavian? Amazing stuff.
| adamisom wrote:
| Wow it just ate the rook huh?
| complexworld wrote:
| Wouldn't the bug with queen side castling end up with _ _ K R
| _ _ _ _?
| ARandumGuy wrote:
| Another wild guess: Lichess could be pre-calculating and
| caching the legal moves for the most common chess positions.
| While pre-calculating every possible legal move for every
| position would be impossible, you could pre-calculate the most
| common openings and endgames, which could cover a lot of real-
| world positions. This cache could easily be larger then
| practical for the client, but a server could hold onto it no
| problem. This could save on the net processing time, compared
| to the client determining all legal moves for every position.
| pfedak wrote:
| This looks like the relevant fix: https://github.com/lichess-
| org/scalachess/pull/154
|
| (the broken code checked that the only pieces on the king's
| path to its new position were kings and rooks of the
| appropriate color)
| benediktwerner wrote:
| From what I remember, one of the main reason also was to avoid
| bloating the JS on the game page. That page is kept especially
| slim to maximize performance and load times for low-powered
| devices.
| hyperhopper wrote:
| I wish the article explained how it dealt with message loss from
| the at-most-once redis pub/sub channel
| benatkin wrote:
| Indeed, it does deal with the message loss. I was momentarily
| confused because in my many thousands of bullet chess games on
| Lichess I haven't had much of any message loss that can be
| attributed to Lichess's servers (but plenty when my Internet
| connection is down or unstable).
|
| I will have to take a look, because whatever it's doing, it
| works very well!
| DylanSp wrote:
| I was hoping for that too, that's the kind of interesting
| architectural question I wanted this article to answer.
| ilrwbwrkhv wrote:
| Beautiful architecture. Startups and companies like Netflix
| should learn from this instead of cargo culting microservices.
| ajkjk wrote:
| What? Do you have some reason to think Netflix's architecture
| is deficient?
| ilrwbwrkhv wrote:
| Overly complicated with microservices. Can be made 10x
| simpler.
| LinuxAmbulance wrote:
| Sometimes simplicity is not the best goal.
|
| Redundancy, scalability, decoupling, resilience, best
| possible handling of errors, cost optimization, etc. may be
| more important at the scale Netflix operates at.
| lcnPylGDnU4H9OF wrote:
| > Redundancy, scalability, decoupling, resilience, best
| possible handling of errors, cost optimization, etc. may
| be more important at the scale Netflix operates at.
|
| So much that they built a tool to intentionally make
| things difficult (read: it arbitrarily stops production
| system processes/containers/etc.) and help inform what
| decisions to make in favor of fault tolerance.
|
| > Exposing engineers to failures more frequently
| incentivizes them to build resilient services.
|
| https://github.com/Netflix/chaosmonkey
|
| https://en.wikipedia.org/wiki/Chaos_engineering
| renewiltord wrote:
| Embarrassing. I built 99% of Netflix functionality
| locally with VLC and a subdirectory of mkv files.
| ilrwbwrkhv wrote:
| For Netflix level of complexity. Pornhub has more traffic
| and serves more customer than Netflix with monolithic PHP
| and some services.
| paxys wrote:
| Because the top 5 comments on HN always say so, so it must be
| true.
| enneff wrote:
| And what exactly do you think lila, lila-ws, and redis are if
| not microservices (or as they should be called, "services")?
| Lichess could easily be implemented as a single monolithic
| process but it is not.
| immibis wrote:
| They are services, but not micro. lila-ws spun off of Lila
| for a good reason (fault isolation) and not because "let's
| make everything a service". And they don't follow any
| standard microservice pattern - a reverse proxy isn't a
| microservice.
| shironandon wrote:
| what happens to those websocket connections when the API is
| updated or redeployed?
| zazaulola wrote:
| It is to be expected that LLM will make a decision on its own
| if it suspects any changes to the API. In any case, there is no
| time to fix the code during the game.
| VoidWhisperer wrote:
| They werent talking about an LLM here
| paxys wrote:
| It's pretty easy to build auto reconnect capability in the
| client. The server will drop all its connections and go out of
| rotation, and the client will start a new connection and find
| the new one. If the switch happens fast enough then the user
| shouldn't even notice.
| sam0x17 wrote:
| 20 years later I still think "female lich" whenever I see the
| word lichess, even though I know it's li chess.
| Keyframe wrote:
| there are more of us then!
| Suppafly wrote:
| makes me think of the Asian fruit.
| AlienRobot wrote:
| When you promote a pawn to queen that's actually the lichess.
| d4rti wrote:
| I suspect the "l" parameter is for observed latency as the client
| displays observed latency from the server.
| lxgr wrote:
| Lichess also compensates for latency to some extent.
|
| To do that, the server needs some measure of "how long does the
| client think the player actually took to make a move", to later
| subtract latency not attributable to actual thinking from the
| clock.
| zxilly wrote:
| I wonder why this protocol needs an ack? a websocket wrapped in a
| tls should be perfectly capable of guaranteeing the integrity of
| the message
| andai wrote:
| You can verify this with ten lines of code and clumsy (a tool
| for simulating packet loss).
|
| I tried this and not all the messages I sent arrived.
| enneff wrote:
| What do you mean? If you open a web socket connection it
| should behave like a normal TCP connection. All sent data
| guaranteed to be delivered complete and in order, unless the
| connection fails.
| mananaysiempre wrote:
| Unless the connection fails, at which point you have no
| idea _when_ it failed. You know that the other side
| received all stream offsets within [initial, X] with X >=
| last received ACK, but other than that you have no idea
| what X is. Even getting the last received ACK value out of
| whatever API or upper-level protocol you're using could be
| nontrivial, because people rarely bother.
| enneff wrote:
| So that the client knows the message has been delivered and
| handled by the server, which can make the UI indicate the state
| of the connection.
| augusto-moura wrote:
| Maybe authorization, illegal moves? Don't know the full
| protocol to know how they handle edge cases. They might just
| return a NACK
| parl_match wrote:
| That just means that the message hit the TLS terminator. It
| doesn't mean that the backend logic received the state change.
| blastro wrote:
| lichess is one of the best sites on the internet. very happy to
| contribute my $5/mo
| trod123 wrote:
| If you consider this to be true, you would seem to have a
| rather low standard.
|
| There are many aspects in which they are not the best.
| dibyadarshan wrote:
| Like?
|
| Ad-free, compute intensive, non-CRUD, massively scaled,
| complex cheat moderation, infinite puzzles/analysis,
| educational (studies/tactics/openings explorer), etc. All
| this for free. I'm curious what's the best website in your
| opinion
| hilux wrote:
| Hello, fellow Patron!
|
| Even though nowadays I hardly have time to play, I'm still
| happy to support such a delightfully honorable and usable(!)
| open-source project.
| bobmcnamara wrote:
| nit: fen only encodes board state, not game state
|
| Edit: also includes move count but not repetition.
| xrisk wrote:
| How is the game state not just the board state? Move history
| doesn't matter in chess (FEN encodes the 50 move rule)
| andrewaylett wrote:
| Per Wikipedia, it doesn't encode the threefold repetition
| rule.
|
| https://en.wikipedia.org/wiki/Forsyth%E2%80%93Edwards_Notati.
| ..
| michaelmarkell wrote:
| Timing of moves
| anamexis wrote:
| Indeed, the 50 move rule, as well as castling rights, whose
| move it is, and whether any pawns are currently eligible for
| en passant.
| MobileVet wrote:
| I wish this discussed the timing arbitration of each move. Based
| on the packet information (if that is correct & complete) then
| the timing is done entirely on the clients. However, they show
| the time in seconds which can't be right so I am curious how
| accurate this packet schema is (or if those are float values).
|
| Regardless, one thing I find maddening about chess.com is the
| time architecture of the game. I haven't seen the underlying
| code, but it feels like the SERVER is tracking the time. This
| completely neglects transport time & latency meaning that 1s to
| move isn't really a second. Playing on the mobile client is an
| exercise in frustration if you are playing timed games and down
| to the wire. Even when you aren't, your clock will jump on normal
| moves and it is most obvious during the opening.
|
| This could also be due to general poor network code as well. The
| number of errors I get during puzzles is also frustrating. Do
| they really not retry a send automatically?? <breath>
|
| Chess.com has the brand and the names... but dang, the tech feels
| SO rough to me.
| mkagenius wrote:
| Vladimir Kramnik agrees with your observations about chesscom.
| chongli wrote:
| I'm surprised to see anyone bring him up here!
| nih wrote:
| Interesting
| pshc wrote:
| _> it feels like the SERVER is tracking the time_
|
| TBH this is what I expected for all online chess. How else to
| reconcile the two players' differing clocks and also prevent
| client-side cheating?
| MichaelZuo wrote:
| It hasn't been done client side in any pvp game I've heard
| of.
| stevage wrote:
| I'm pretty sure freechess.org did.
| bongodongobob wrote:
| Track the two clients pings? What client side cheating
| prevention would you need to do in chess? Afaik you can't
| cheat by clipping through walls or jumping around on the map.
| connicpu wrote:
| The client side cheating would by lying about when you
| received the packet in order to give yourself more time to
| think. Even if you only shifted it by 200ms per move, that
| could add up to a lot over the course of a long game.
| kaoD wrote:
| To give additional context: bullet chess can go down to 1
| minute per player. Lying about a few millisecond per move
| there is _huge_.
| HDThoreaun wrote:
| Cheat by giving yourself more time
| bongodongobob wrote:
| I can't play bullet on chess.com for this reason. Lost way too
| many games on "time" even though I had a second or two on the
| clock. Incredibly frustrating.
| pengowray wrote:
| > they show the time in seconds which can't be right
|
| Seems right.
|
| If you export/download games from lichess, they use the .pgn
| (Portable Game Notation) format, which is a standard plain-text
| format circa 1993, used by pretty much everyone for describing
| a chess game.
|
| Lichess follows the specification to the letter, and as it only
| technically allows one-second accuracy, lichess only record
| moves with one-second accuracy. It seems insane, but that's how
| they do it.
|
| Chess.com also exports PGN files, but they add a decimal place,
| allowing subsecond accuracy. No one has a problem with this.
| There is no software which cannot handle this. But Lichess
| refuses to "break" the spec.
|
| lichess PGN export example:
|
| > 1. d3 { [%eval -0.15] [%clk 0:01:00] } 1... g6 { [%eval 0.04]
| [%clk 0:01:00] }
|
| Chess.com PGN export example:
|
| > 1. d4 {[%clk 0:02:58.6]} 1... b6 {[%clk 0:02:59.2]}
| galkk wrote:
| So essentially lichess chose StackOverflow approach - (rather)
| beefy servers, instead of "treating them like a cattle".
|
| Interesting that they accumulate and periodically store game
| state. Unfortunately it is not very clear, where they store
| ongoing game state - in redis or on server itself. Also cost
| breakdown doesn't have server for redis, only for DB.
|
| BTW, their github has better architectural picture, than overly
| simplified one in the article:
| https://raw.githubusercontent.com/lichess-org/lila/master/pu....
| Unfortunately, I'm afraid, drawing something like that during
| interview may not land a job at faang =(
|
| Note that they have cost per game fairly low: $0.00027, 3,671
| games per dollar.
|
| Their cost breakdown, for ones who are curious
| https://docs.google.com/spreadsheets/d/1Si3PMUJGR9KrpE5lngSk...
|
| p.s. I'm not saying that Lichess's approach is the best or faang
| is the worst. Remember, lichess had 10 hours outage exactly
| because of the architecture chosen (single datacenter
| dependency). https://lichess.org/@/Lichess/blog/post-mortem-of-
| our-longes... . And outages like that are exactly the reasons why
| multi-datacenter and multi-region architectures are drilled down
| into faang engineers.
|
| My point is is that there are cases when this approach is legit,
| but typical interview is laser focused on different things, and
| most probably won't appreciate the "old style" approach to the
| problem. I'm sure that if Thibault will ever decide to land in
| faang he will neither do whiteboard coding nor system design.
| epolanski wrote:
| > Unfortunately, I'm afraid, drawing something like that during
| interview may not land a job at faang =(
|
| Yet another reason to be skeptical of the quality of hiring in
| faang if anything.
| immibis wrote:
| Why feel anything about it at all? You work at FAANG: be glad
| for the money or quit if there isn't any. You don't work at
| FAANG: bad hiring makes it easier for you to get hired and
| make money.
| epolanski wrote:
| You haven't considered the third option: couldn't care less
| about working at these companies because of different
| reasons (personal, financial, geography, cv or whatever).
|
| My criticism was mostly towards the very poor metrics these
| companies have introduced behind hiring, albeit I can
| understand that given the gigantic amount of applications
| they get a mechanism for removing false positives is
| acceptable even if missing on false negatives.
|
| And even more that it spread to companies that do not have
| their problems and can't afford false negatives.
| simplify wrote:
| This is a limited, self-centered way of thinking (not self-
| ish, just self in the neutral sense of the word).
|
| Looking at second-order effects, many companies look up to
| FAANG for "best practices", which often includes them
| blindly copying their hiring practices. Without feeling or
| calling out any healthy skepticism, the software hiring
| world becomes a worse place overall.
| benediktwerner wrote:
| Redid runs on the main server, where lila runs, as indicated in
| the diagram you linked. And moves are buffered in lila. Redis
| is only used for pub-sub.
| juujian wrote:
| I remember Meta having a few outages of their own. And outlook
| as well. So I'm not sure what to think now. But sure, on paper
| FAANG is redundant and hence better.
| immibis wrote:
| As I understand, the separation between Lila and Lila-ws is
| primarily for fault isolation rather than independent scaling.
| Maybe independent scaling becomes useful if websocket overhead
| exceeds what one machine can handle.
| huins wrote:
| > - l: Probably some length?
|
| I don't understand why the author didn't just look this up in the
| source code. Lichess is open source and we can see exactly what
| this field is here, it's the average lag:
|
| https://github.com/lichess-org/lila/blob/45b5f0cfbbf6c045ad7...
| send = (t: string, d: any, o: any = {}, noRetry = false): void =>
| { const msg: Partial<MsgOut> = { t }; if (d !==
| undefined) { if (o.withLag) d.l =
| Math.round(this.averageLag); if (o.millis >= 0) d.s =
| Math.round(o.millis * 0.1).toString(36); msg.d = d;
| } if (o.ackable) { msg.d = msg.d || {}; //
| can't ack message without data this.ackable.register(t,
| msg.d); // adds d.a, the ack ID we expect to get back }
| const message = JSON.stringify(msg); ...
|
| Which is calculated from how long the server takes to respond to
| ping messages that the client sends: private
| schedulePing = (delay: number): void => {
| clearTimeout(this.pingSchedule); this.pingSchedule =
| setTimeout(this.pingNow, delay); }; private
| pingNow = (): void => { clearTimeout(this.pingSchedule);
| clearTimeout(this.connectSchedule); const pingData =
| this.options.isAuth && this.pongCount % 10 == 2 ?
| JSON.stringify({ t: 'p', l:
| Math.round(0.1 * this.averageLag), }) :
| 'null'; try { this.ws!.send(pingData);
| this.lastPingTime = performance.now(); } catch (e) {
| this.debug(e, true); } this.scheduleConnect();
| }; private computePingDelay = (): number =>
| this.options.pingDelay + (this.options.idle ? 1000 : 0);
| private pong = (): void => {
| clearTimeout(this.connectSchedule);
| this.schedulePing(this.computePingDelay()); const
| currentLag = Math.min(performance.now() - this.lastPingTime,
| 10000); this.pongCount++; // Average first 4
| pings, then switch to decaying average. const mix =
| this.pongCount > 4 ? 0.1 : 1 / this.pongCount;
| this.averageLag += mix * (currentLag - this.averageLag);
| pubsub.emit('socket.lag', this.averageLag);
| this.updateStats(currentLag); };
| burgerquizz wrote:
| how would you protect your websocket server? I am building a
| game, but when I put the domain behind (free plan) cloudflare, I
| get latency delay (3x slower) on the players events.
|
| Saw CF had some paying solution, but was wondering about a free
| solution
___________________________________________________________________
(page generated 2024-10-23 23:00 UTC)