[HN Gopher] G9.js: Automatically Interactive Graphics
___________________________________________________________________
G9.js: Automatically Interactive Graphics
Author : nnx
Score : 559 points
Date : 2023-08-09 04:24 UTC (18 hours ago)
(HTM) web link (omrelli.ug)
(TXT) w3m dump (omrelli.ug)
| jsunderland323 wrote:
| This is lovely
| c0brac0bra wrote:
| The animations don't seem to run on mobile
| xiaoxiae wrote:
| They do for me (Android, Firefox).
| srush wrote:
| I got excited about this a couple years ago and ported it to
| Pyodide in to run in the browser with a pytorch like library:
|
| https://srush.github.io/g9py/
|
| It's a little choppier than javascript, but crazy to me that it
| works at all.
| aboodman wrote:
| Not sure what I would use this for, but it's crazy clever.
| aboodman wrote:
| I guess it could potentially be used for any drag and drop
| interface? It's such a different way to think of the problem,
| I'm going to have to try it for something just to wrap my head
| around it.
| jrmiii wrote:
| Well, that was 8 hours ago - you get anywhere noodling over
| this?
|
| I'm trying to go through the same thought process - this is
| neat, how do i translate into more practical applications. It
| seems like such a powerful paradigm if one can figure out
| that mapping.
| aboodman wrote:
| No :).
|
| The interesting thing is that drag and drop interfaces are
| not typically purely directly mapping the drag operation to
| moving the object selected. There's a lot more going on --
| there's hover and activation effects, limits, often
| animation.
|
| This approach gets rid of the annoying leg work of mapping
| pointer movement to changes, but I'm not sure how to fit
| the rest of drag drop UI into it (I do see examples of
| limits in the doc).
|
| I think it's going to have to be something in the back of
| my head for awhile, and when I see interesting dragdrop
| experiences I'll think about how it could be achieved with
| this approach.
| joeld42 wrote:
| this is pure magic. Amazing stuff.
| ronyfadel wrote:
| This is simply stunning!
| sph wrote:
| I would love an explanation on how it works because this feels
| like black magic to me.
|
| If you know you have two points, one at (0, 10) and (10, 0) and
| drag the latter to (20, 0), how does the system tell that the
| first needs to be mirrored by the same distance?
|
| Again, this is magic, so very well done to the author. Great
| stuff
| fouronnes3 wrote:
| Define a function that measures the error between the position
| of the drawn element and the new position required by the drag
| action. Compute partial derivatives of this function with
| respect to the data variables. Minimize this function to find
| the new data values. Re-render everything with the new optimal
| data values.
|
| It's a bit confusing, because typically what's called 'data'
| here would be called "parameters" in mathematical optimization,
| or "weights" in statistics / machine learning. Also ML would
| call "data" what's fixed, so here the position of the dragged
| element.
| pohl wrote:
| Any sufficiently advanced mathematics is indistinguishable from
| magic, where "sufficently advanced" turns out to be a low bar.
| indigoabstract wrote:
| Looks pretty cool!
|
| One nitpick is, there are no markers/selection, so it's not very
| obvious that you can drag the objects, unless you stopped to read
| the description first.
|
| I'm also bit puzzled on how readjusting the object's position
| works. Usually, you'd define a pointer handler and transform
| based on the input, but if I understand correctly, the engine
| tries its best guess to automate this for you?
| devmor wrote:
| This feels like witchcraft. I'm incredibly impressed.
| nine_k wrote:
| The fact that you can pick a node which is rendered as a fifth or
| a sixth level of a recursive tree, move it around, and the rest
| of the tree would react, looks highly magical to me. It works
| without a function that explicitly infers the initial parameters
| of the tree from an arbitrary leaf node position.
|
| I wonder how is it achieved, and how well does it work when
| positions of points are not connected by reasonably smooth
| functions.
| Chabsff wrote:
| If I had to guess: 1. Get partial derivatives of each
| parameter's influence on the moved object by sampling the
| function twice. 2. Perform a bog standard gradient descent. 3.
| go back to 1 until convergence is achieved.
|
| Or you could be lazy and just throw an off-the-shelf optimizer
| at it.
| HammadB wrote:
| https://github.com/bijection/g9/blob/master/src/minimize.js
| Chabsff wrote:
| They do quite a bit more sampling in order to dynamically
| determine a good delta value. But apart from that, I dare
| say that I got it broadly right.
|
| It feels to me like there's also quite a bit of temporal
| coherence that could be leveraged in order to accelerate
| the process.
| throwaway4good wrote:
| Requires the relationship from the screen coordinates to
| the variables in the program to be differentiable.
|
| It is a very nice idea though.
| jaen wrote:
| It requires the final function to have a numerically
| reasonable finite difference gradient, which is somewhat
| different from what is commonly referred to as
| "differentiable" - eg. the insides of that function could
| still use non-differentiable/non-analytic functions.
|
| It seems to be based on numeric.js, which is based on the
| classic Fortran UNCMIN [1] optimizer.
|
| [1]: https://perception.lab.uiowa.edu/UNCMIN
| [deleted]
| Kalabasa wrote:
| Reminds me of the game Euclidea, where you manipulate geometric
| relationships (exactly like the circle example)
| jamessb wrote:
| There were some interesting comments when this was last discussed
| on HN (7 years ago):
| https://news.ycombinator.com/item?id=12572412
| anagri wrote:
| very impressive, the last demo of a working crank-shaft-engine
| shows you can use it to make complicated models as well.
|
| great start, best wishes with the project, looking forward to
| more amazing demos.
| MrLeap wrote:
| First thought was cool, clean simple API. I like it better than
| d3 to do the equivalent. -- Then I read this:
|
| "When someone interacts with the graphics, for example by trying
| to drag an element to a new position, g9 optimizes over the space
| of possible values for your data to find a set of values that
| comes closest to creating the change. "
|
| That is some highly magical DOM jimmying. High science weirdness.
| The applications I can think of where you'd need to manipulate a
| chart to modify data are all pretty niche (except for fun, which
| is universal). When the use case does come around this is going
| to be a delight.
| nine_k wrote:
| Because it's not for charts. It's for interactive demos; have
| you tried playing with them on the rest of the page?
| MrLeap wrote:
| Yeah I did. They're fun.
| sainez wrote:
| As far applications go, there are a lot of STEM concepts that
| can be communicated effectively by allowing interactivity.
| MrLeap wrote:
| This is a good point.
| noduerme wrote:
| I wouldn't call that high science. Average the velocity and
| vector of a drag event over the first few frames, keep
| averaging with more frames, and start moving everything to the
| predicted end-point with a nice ease-in-out, you're most of the
| way to "optimizing over the space of possible values".
| WhitneyLand wrote:
| I don't think so. It looks like it's using gradient descent
| optimization which does not require quotation marks around it
| and requires college level stem math for most people at least
| to really understand.
| noduerme wrote:
| You're right, it turns out to be a gradient descent
| function adapted from numeric.js. Seems like a long way to
| go to interpolate mouse events into a range of values.
| Still suffers from gimbal lock in the cube demo. I guess
| since it doesn't allow you to manage the draw routine, you
| couldn't use quaternion to Euler conversion. I'm not sure I
| see the purpose of it. But what do I know, I don't have a
| stem degree.
| injidup wrote:
| It seems to have things in common with 2D CAD constraint based
| sketching tools?
| [deleted]
| arihantparsoya wrote:
| How is this different than p5.js?
| JackFr wrote:
| Dumb question -- I know what d3 stands for. What are the 5 p's
| in p5 and the 9 g's in g9?
| kayge wrote:
| I was curious too, took a little bit of digging :)
|
| "the original domain of [P]rocessing was proce55ing.net, so
| people used to sometimes refer to processing as proce55ing or
| P5 or p5 for short. they still do sometimes. p5.js is a
| reference to that."
|
| from https://github.com/processing/p5.js/issues/2443
| nine_k wrote:
| I'm not familiar with p5. Can you pick an arbitrary rendered
| point of a p5 picture, and manipulate it?
| qbasic_forever wrote:
| Yes, it has full interactivity support:
| https://p5js.org/learn/interactivity.html
|
| P5 has an imperative/immediate mode style of graphics API so
| you have to code the result of your interactivity (moving a
| point, etc.) yourself. Something like three.js, which has a
| full scene graph for rendering and interactivity, might be a
| better choice if you want a lot of interactivity out of the
| box.
|
| I'm kinda scratching my head at g9 as it seems to be a mish
| mash of scene graph that's implicitly generated and
| imperative rendering APIs. It's not exactly clear to me why
| you'd want this mix except for very specific scenarios the
| author created support for with their implicit scene graph
| generation for interactivity.
|
| In my mind you either want something that's like three.js
| with a full scene graph style of rendering and interactivity,
| or a p5 style imperative API for both. This seems to be some
| in-between thing to scratch some itches.
| sph wrote:
| Have you tried dragging one of points?
| qbasic_forever wrote:
| Yes, I understand what is happening here with
| interactivity. Like I said I don't follow why I want some
| implicit generation of an interactivity scene graph (i.e.
| what points are connected or move relative to other
| points) when I can just be explicit about the scene graph
| interactivity with other similar libraries.
| [deleted]
| monsieurbanana wrote:
| Because it's a fast way to get that interactivity, rather
| than having to explicitly code it yourself.
|
| A good example are modern math videos on youtube, like
| this one (not saying that they're using G9.js in
| particular): https://www.youtube.com/watch?v=spUNpyF58BY
|
| It's 20 minutes of changing graphics. You could code all
| of that with P5, and it would be an excruciatingly long
| process. G9.js would get you there in a fraction of the
| time.
| tkzed49 wrote:
| "When someone interacts with the graphics, for example by
| trying to drag an element to a new position, g9 optimizes over
| the space of possible values for your data to find a set of
| values that comes closest to creating the change."
|
| This clearly isn't trying to be Processing. Obviously it's not
| revolutionary in terms of being productive, but it's a super
| cool concept and seems unique to me! The dragon example is
| particularly telling. You can grab any node and it tries to fit
| the fractal to wherever you move it.
| awirick wrote:
| This could be extremely helpful for conveying trig or other math
| concepts in an education setting. I know a lot of visual learners
| that struggle with mapping the math to the visual space.
| Sprinkling in interactivity is a great way to help cement
| understanding.
___________________________________________________________________
(page generated 2023-08-09 23:02 UTC)