[HN Gopher] How to Animate Multiplayer Cursors
___________________________________________________________________
How to Animate Multiplayer Cursors
Author : stevenfabre
Score : 172 points
Date : 2022-07-05 13:04 UTC (9 hours ago)
(HTM) web link (liveblocks.io)
(TXT) w3m dump (liveblocks.io)
| dwallin wrote:
| I think it's a mistake to just drop most of the data and then try
| to rebuild the lost data afterwards.
|
| Nothing requires you send just a set of points across. A better
| approach is to transform and compress the data BEFORE sending it
| across the wire. This way you get the benefit of using all the
| available data to create a more accurate simplification.
|
| For example, each update take your cursor point set and construct
| a bezier curve that best fits the data.
| somehnacct3757 wrote:
| This is analogous to player position in any multiplayer game.
|
| One additional parameter to keep in mind is how real-time does
| your simulation of remote players need to be? If you don't need
| real-time positioning there's a whole other dimension of
| shortcuts you can take by introducing what amounts to a
| 'streaming delay'.
|
| In a lot of these apps the cursors can't interact with each other
| so you have no need for real-time positioning and its
| accompanying smoothing techniques. Cheat cheat cheat! That's how
| games get their performance, way more often than being smart
| they're clever opportunistic cheaters!
| capableweb wrote:
| > This is analogous to player position in any multiplayer game.
|
| Not sure. Multiplayer games are easier to predict player
| movement. Once someone starts moving forward, you can predict
| that they'll move forward for a little while, start turning,
| continue forward and so on. Add in physics (like the
| motions/movements of a car, or the running of a human) and
| there will be constraints the player can't break (when you stop
| moving forward, you'll move forward slower and slower until you
| stop, maybe over 100ms or so)
|
| But mouse movement is highly erratic. It'll be short of
| impossible to add any sort of prediction, as it'll be incorrect
| most of the time, instead of being mostly correct but sometimes
| not.
| tomovo wrote:
| The real fun starts when online players start shooting at
| each other. How to decide if a hit was fair AND make it look
| believable for both parties to minimize outrage?
| EllenChisa wrote:
| Loved the deep dive into the difference between timing functions
| and spring/spline animations (ctrl-f comparison to jump to the
| animation).
| AnhTho_FR wrote:
| Awesome deep dive!
| bluelightning2k wrote:
| Can we just take a moment to appreciate how well done this
| article is.
|
| It's like the AAA studio equivalent of a tech article.
| Stampo00 wrote:
| The spline stuff is super useful to me for another non-cursor
| idea I've had simmering.
|
| But I'm torn by this. The spline approach seems to look the most
| accurate. But when all three approaches are shown at the same
| time at the end, I think the spring animation might look more
| visually pleasing. But then, if the spring approach is only
| degrees better than CSS transitions, is it really worth all the
| extra code?
| DonHopkins wrote:
| I love this article! Great analysis and solutions. It's nice when
| user interface designers care enough to think through, implement,
| try out, and refine such important details.
|
| I developed a multi player version of SimCity for X11 that I
| released in 1993 and demonstrated at the InterCHI '93 Interactive
| Experience, which showed you other player's cursors moving around
| and editing the map, but of course it required a fast network to
| run on and updated all the clients synchronously, due to the
| limitations of X-Windows, so there were no interpolation issues.
| (X-Windows clients aren't even capable of performing local
| computation and feedback the way NeWS clients could and web
| browser clients now can, so it was a moot point.)
|
| https://www.donhopkins.com/home/catalog/simcity/simcity-anno...
|
| https://www.donhopkins.com/home/catalog/simcity/simcitynet.h...
|
| The multi player demo showing different kinds of voting dialogs,
| multiple cursors, the tool palette and pie menus, and the voting
| "yes" shortcut of building the same thing in the same place,
| starts at 5m45s:
|
| https://www.youtube.com/watch?v=_fVl4dGwUrA&t=5m45s
|
| One interesting thing about the SimCity cursor was that it was
| color and shape coded to show which tool was selected. The tool
| palette (and also the pie menus which had the same icons and
| layout as the tool palette) served as a legend for the cursor by
| showing the same color coded outline around the icons as the
| cursor used. So you could tell which tool other users had
| selected. You could hide the tool palette to make the map bigger,
| and use the pie menus instead, which were much more efficient.
|
| Multi Player SimCityNet for X11 on Linux: Demo of the latest
| optimized Linux version of Multi Player SimCity for X11. Ported
| to Unix, optimized for Linux and demonstrated by Don Hopkins:
|
| https://www.youtube.com/watch?v=_fVl4dGwUrA
|
| Micropolis Online (SimCity) Web Demo: A demo of the open source
| Micropolis Online game (based on the original SimCity Classic
| source code from Maxis), running on a web server, written in C++
| and Python, and displaying in a web browser, written in
| OpenLaszlo and JavaScript, running in the Flash player. Developed
| by Don Hopkins:
|
| https://www.youtube.com/watch?v=8snnqQSI0GE
|
| Source Code:
|
| https://github.com/SimHacker/micropolis
|
| HAR 2009 talk: Constructionist Educational Open Source SimCity:
|
| https://donhopkins.medium.com/har-2009-lightning-talk-transc...
| blt wrote:
| Would be interested to hear a signal processing perspective on
| this problem. I feel like splines might be susceptible to
| creating false information (e.g. ringing/overshoot), but I don't
| know enough about it.
| jewel wrote:
| There's another approach that might work better.
|
| Instead of sampling the mouse position every 100ms, you'd save
| off all the mouse positions, and then send the latest batch every
| 100ms. The other side would then replay the exact positions, just
| delayed by 100ms. It'll end up with the same latency as these
| motion smoothing approaches, while only using slightly more
| bandwidth.
| thomasahle wrote:
| What about a mix:
|
| Every 100ms you fit the best Beizer curve to the last batch of
| mouse positions and send that.
|
| It seems like that would give a more precise reconstruction
| than fitting a Beizer curve only to one sample every 100ms on
| the server.
| javajosh wrote:
| Agreed. This would be only slightly annoying if you're talking
| to the person. But 100ms or even 200ms is an acceptable
| latency, especially if it's constant. This solution is not just
| simpler, but it's also more efficient as you can bundle the
| data up efficiently, and include state changes as well.
|
| There is a lot of prior art in this space, btw. I remember
| Meteor.js having a great real-time demo over websockets that
| actually used predictive techniques to keep things
| (imperfectly, but still impressively) 0 latency.
| gernb wrote:
| Am I missing how that would work? That's an exponential
| increase in bandwidth needed to send all the mouse positions. 1
| position per 100ms per player vs 10-30 per 100 ms per player.
| Those positions all have to be propagated to other players. so
| in the first case, 10 players = 10 positions per 100ms. In the
| 2nd case is 100-300 positions per 100ms.
| acomms wrote:
| I like this solution but would latency create issues? You'd
| only be sending 100ms worth of motion every ~120ms. Would you
| just drift 200ms behind with every passing second (20ms behind
| after each 100ms batch)? I think I may be missing something
| though.
| AnotherGoodName wrote:
| You actually intentionally start with more playback delay
| with this method and sync playback with the other user
| intentionally 300ms behind (assuming a 100ms latency)
|
| So the remote user packets up 100ms of mouse movement with
| timestamps, sends it with ~100ms latency. Your side now has a
| buffer of ~100ms to start playing the positions back.
|
| This also removes all jitter in the playback from varying
| latency (up to the point the jitter stays under 100ms).
|
| All of the above numbers are made up for this example. You
| can adjust the playback delay as much as needed for smooth
| playback.
| HWR_14 wrote:
| Honestly, I don't see why you'd need to batch it ever 100ms.
| Sure, you don't want to send a mouse movement every time an
| event triggers, but surely 30fps looks smooth and won't
| overload the system.
| LeifCarrotson wrote:
| You can't reliably send, fail, retry, and confirm receipt of
| a TCP packet in 33ms over arbitrary Internet connections. 8ms
| is right out. I can get 200us over a local EtherCAT realtime
| industrial IO network, but that's with careful management of
| well-isolated single-machine network conditions and it just
| doesn't work with a cellular modem for download and
| oversubscribed residential cable for upload. Assuming latency
| of 120ms (as used in the defaults in the linked tutorial) is
| much more realistic.
|
| And you also can't set up your system with a 5 second delay
| to send data every 5 seconds, because any jitter will result
| in hiccups.
|
| You could set up your system to send out the new data each
| time the previous buffer is acknowledged, but that's kind of
| pointless, if you get lucky with a good connection and can
| send data to be rendered 4.970 to 5.000 seconds from now,
| what's the difference for the user between doing that versus
| reducing the network load by approximately a factor of 3 and
| waiting until you have data for 4.900 to 5.000 seconds?
|
| I think 100ms is a reasonable minimum batch size.
| jfim wrote:
| The biggest problem is TCP. Ping times for multiplayer
| games on decent Internet connections can be on the order of
| 10-20ms or even lower.
| layer8 wrote:
| Just for comparison, the default USB mouse polling rate is
| 125 Hz, so 8 ms. If that is too often, 16 or 32 ms would make
| sense, which is close to 60 or 30 Hz/fps, respectively.
| cousin_it wrote:
| Yeah, wanted to say the same thing. If latency is the only
| problem, just batch up the updates and replay on the other end.
| If bandwidth also becomes a problem, only then start
| compressing the data. But to be honest, if we can stream video
| over the internet, we surely have enough bandwidth to stream
| cursor positions.
| stevenfabre wrote:
| Good point, it's also important to consider this in the
| context of the application that will use those multiplayer
| cursors. It's important that the state of the document
| matches the cursors, so it makes sense to have both presence
| (cursors, selection) and storage (document data) be perfectly
| in sync even if that means having a slight 80-200ms delay.
| sghosh2 wrote:
| The animations on the blog itself are incredible. Would love to
| read something in the future about how you guys put this together
| (i.e. what frameworks, components, tooling is used to generate
| these blogs).
| stevenfabre wrote:
| That would be pretty meta! "how to create how to interactive
| articles" :)
| cableshaft wrote:
| I'm super interested in this. I have a few interactive
| articles I'd like to make, but I'm not sure what tech to use
| for it to make it relatively quickly and without taxing the
| page too much.
|
| Was thinking maybe the Phaser game framework, but that might
| be too heavy, especially if there needs to be 7 or 8 of them
| on the same page.
| stevenfabre wrote:
| We build our interactive visuals with React and embed those
| into the mdx file for the blog post.
|
| I made note for us to write a blog post about it. Stay
| tuned :)
| steventey wrote:
| The interactive animations in this post are mind-blowingly
| good!!!
| JamesSwift wrote:
| I opened the article just to check and you're right. This is
| A+, top-tier stuff. Sets a standard for many other technical
| articles.
| vincentriemer wrote:
| Another potential approach could be using the Web Animations API
| with an additive/relative animation approach via the `composite:
| 'add'` option. I say potential as it really only fully works in
| Chrome atm -- Firefox has it but has weird rendering bugs and
| Safari says it has experimental support but I've never managed to
| get it working.
|
| I like the potential of this approach as it lets you get smoother
| results than just CSS transitions yet doesn't require you to use
| a RAF loop-based animation library.
| fullstackwife wrote:
| It depends on the app, but in some cases you should also include
| content identifier to the usual X,Y coordinates, as from the user
| perspective it is more important to see the cursor hovering on
| top of something instead of being "very close". The meaning is
| different in such case.
| stevenfabre wrote:
| Exactly! With Liveblocks, you can also store anything presence
| related. You can for instance store a selectedId and use that
| on the other side to highlight the selected element.
| agitator wrote:
| I was hoping that this article would go into deeper techniques
| than just interpolation and splines.
|
| I would be cool to see an example of how a Kalman filter approach
| would compare in terms of precision and latency. My expectation
| is that it would be the best of both worlds.
| mcphage wrote:
| Or even a physics simulation--if the mouse is moving with a
| given velocity, that velocity won't change super fast. So even
| if your data is a bit behind, you can use physics to estimate
| where they probably are _now_. And if you get new data, and
| your estimation is too far off, then move the mouse to the
| right spot. If it 's mostly correct, just base your future
| simulation off of the new information, so that it moves
| smoothly towards the correct value.
| paulryanrogers wrote:
| These are pointers and not cursors, unless I misunderstand.
| Cursor made me think of sharing tmux or IDE text sessions.
| [deleted]
| schneegansmarie wrote:
| I always felt like using a CSS ease transition between points
| would be enough. Love the spline approach with perfect-cursors,
| cursors really feel like they're being moved by real people.
___________________________________________________________________
(page generated 2022-07-05 23:01 UTC)