[HN Gopher] D3 in Depth
___________________________________________________________________
D3 in Depth
Author : lobo_tuerto
Score : 209 points
Date : 2024-05-16 14:18 UTC (8 hours ago)
(HTM) web link (www.d3indepth.com)
(TXT) w3m dump (www.d3indepth.com)
| sbarre wrote:
| If you like the idea of using D3 for data visualization but you
| want something a bit higher level, check out Observable Plot[1],
| a library by the D3 team that adds a lot of conventions and
| conveniences on top of D3, specifically for building plots,
| charts and graphs.
|
| And even beyond that, there's Observable Framework[2], a static
| site generator for building visualization dashboards (that
| supports Plot, D3 and lots of other libraries).
|
| 1: https://observablehq.com/plot/
|
| 2: https://observablehq.com/framework/
| gaws wrote:
| Plot and Framework are locked into the Observable ecosystem,
| which has its own learning curve. Learning D3 offers the best
| flexibility and control.
| mbostock wrote:
| In what way are these free open-source tools "locked-in" to
| Observable? Observable Plot is a vanilla JavaScript library
| that is released under the ISC license and can be used with
| any style of web development (e.g., React, Svelte, whatever).
| And likewise Observable Framework is released under the ISC
| license and you're able to self-host projects anywhere and
| develop locally?
| toasted-subs wrote:
| Can you add the library as a html link.
|
| Anything else would be unwise to invest time in learning.
| ghostly_s wrote:
| Yes? Examples here:
| https://observablehq.com/plot/getting-started
| jeffbee wrote:
| This is like saying that a computer without an operating
| system offers the best flexibility, or that a pile of sand is
| more flexible than an assembled computer.
| locallost wrote:
| D3 is not an empty computer. It's a high level library
| built in a high level language that runs in a browser. You
| are very very very far away from "metal". Being able to do
| something in a few clicks or little typing is nice until
| you need to do something more complicated. Sometimes it's
| worth actually learning how something basically works.
| Sometimes it isn't, it's everybody's choice.
| wisemang wrote:
| So D3 is not what I'd consider a high-level library. The
| point of it is to be able to do all kinds of stuff not
| supported by your average charting solution's out of the
| box histograms and whatnot. It's a great way to make more
| bespoke visualizations.
|
| Of course sometimes you just want to make a damn
| histogram without reinventing the wheel which is where
| some of the niceties discussed above come in.
| locallost wrote:
| D3 runs in the browser and it manipulates DOM nodes, a
| very high level abstraction. You can accomplish something
| with it using SVG, which is again a high level
| abstraction. You can give it data and it will spit out
| numbers that you can use to draw something on the screen.
| You can bind that data to elements and it will keep track
| of what's entering and what's exiting. You can apply
| fancy transitions with a line of code! You can make it
| change an appearance of e.g. a graph by changing the
| interpolation method. And so on.
|
| Sure you have to write code, but it does a lot of things
| in the background that would take you s lot longer to do
| yourself. High level is not just "here's a library on top
| of D3 that you just give data to and you're done, but
| then you have to cry for days because you need to do
| something it didn't think of - 80% of use cases". When I
| used it I didn't get it, but then I took some time, a
| couple days, to actually understand it and I got a lot
| more productive. It's a great feeling when instead of
| thinking in workarounds for a missing configuration
| option you can think in solutions for the actual task. A
| lot quicker and more rewarding.
|
| I didn't do much computer graphics in my life, some
| mandatory university classes, but I'd say doing it on the
| simplest beginner level it was a much different
| experience.
| wisemang wrote:
| Fair enough. I meant specifically in the context of data
| visualization libraries, having done a fair bit of that
| sort of work in my own career. It's harder to make a
| basic chart in d3, compared to many out of the box
| solutions, but it's possible to do pretty much whatever
| you want, thanks to (what I consider) the low-ish level
| of abstraction it provides. I.e. DOM manipulation vs
| direct specification of standard chart attributes.
| __MatrixMan__ wrote:
| I've used D3 in nodejs to generate a png. Is it really
| that different than using something like Python or Java
| to do the same?
| addicted wrote:
| D3 is low level for doing what it does. Visualizing data.
|
| It's hard to think of a lower level abstraction for that
| purpose.
| simonw wrote:
| Plot isn't locked into the Observable ecosystem at all - it's
| a JavaScript library that you can use independently of
| everything else there.
| bilekas wrote:
| Yeah, I can say from experience of D3 at least, it's very far
| from locked in. In fact I would ask what library you're
| thinking of could be less.
|
| The community also has helped with some bizarre integrations
| I was trying to do, into libraries like highcharts for
| strange reasons etc.
| sbarre wrote:
| I'll ignore the part where you're wrong about the lock-in,
| but to the rest of what you said...
|
| Yes? I guess I didn't think it needed to be said so
| explicitly, but if you are opting into Plot and Framework,
| you are accepting convention over configuration as they say.
|
| And for some situations, that is what you want.
|
| If you value flexibility and control above all else, then I
| agree they may not be for you.
|
| But if you value other things, like speed of iteration, or
| just getting some good data visualizations built without a
| deep investment in learning underlying concepts, then Plot
| and/or Framework have definite advantages over doing it all
| with D3.
| forrestthewoods wrote:
| > but you want something a bit higher level
|
| These days I kinda want the opposite. Most of the time I know
| _exactly_ what I want my output to look like. But frameworks
| inherently provide a large bag of presets and if you want
| something slightly different from the preset it 's a pain in
| the ass.
|
| For many types of plots I find it's actually kinda easier to
| just manipulate SVG data directly. It's not that hard!
|
| I'd like to see a library/framework that focuses on making it
| easier to manipulate SVG or canvas output. It should be trivial
| to make tweaks. But the higher level a framework goes the
| harder it is to step off the golden path.
| cess11 wrote:
| Have you tried p5js?
| sbarre wrote:
| Yeah I hear you, if you know _exactly_ what you want, Plot is
| probably not for you..
|
| But I've found lots of value in using Plot to explore data,
| and quickly throw something together and look at it, and
| decide if it's a good way to present the data or not.
|
| Although I will say that you can "drop down" to the SVG
| pretty easily with Plot. Lots of examples even on their site
| where they start with Plot marks and then heavily customize
| the SVG directly.
| contravariant wrote:
| Why does d3 not work for you if you just want to manipulate
| the svg directly? You can use it to generate or update
| arbitrary xml as far as I know.
| zdragnar wrote:
| The higher level in this case is observable plot, not d3.
| screye wrote:
| Does the website feel janky to anyone else ? I have routinely
| found content-rich webpages to be janky.
|
| edit: nvm, turned on hardware acceleration. Smoother, but still
| jankier than I'd like on a Ryzen 9.
| toasted-subs wrote:
| As much as I hate to even ask for but is there a Matlab like
| drag and drop interface for web visualization.
|
| I try to avoid graphs with html5 canvas like the plague maybe a
| new way to design them would be nice.
| jo-m wrote:
| Observable Plot is amazing. It is very flexible and intuitive
| to use. I now prefer it over Plotly.js which I used to use
| before.
|
| The docs are pretty OK, the only thing missing is a usable API
| reference section, so discoverability is not great.
| hajile wrote:
| Learn D3 then (assuming you're doing something chart-like as most
| of us are) move to something like VisX or some non-React
| equivalent and never look back.
|
| I view D3 and jQuery as being somewhat similar and both are very
| prone to spaghetti. Something like VisX adds a lot of structure
| over the D3 you're using to keep things sane.
| laurels-marts wrote:
| I've been using D3 on and off for quite some time. I'm just
| starting now with react and visx definitely looks very
| promising as it bridges the gap nicely. Do you find it limiting
| though (like you have to fight the tool) to achieve something
| more custom vs. using plain D3?
| hajile wrote:
| VisX starts with low-level primitive components that match
| very closely to D3 primitives. It then builds some chart
| primitives on top of those and then builds some charts on top
| of that. You can build pretty much anything using the lowest-
| level primitives and still have a better developer experience
| than D3 IMO.
|
| If you don't need any real customizability (and know you
| won't need it in the future), I'd stick with something with
| something less customizable, but faster to setup. If you know
| you need custom, take the 10-20% more time to setup and be
| able to change in the future as that'll be faster than
| porting your charts over.
| solatic wrote:
| Even if you never end up using it professionally, I think it's
| worth it to learn D3. The core point that the library makes is
| that your projection of the data (say, a 1080p screen, 1920x1080)
| is always different from the dimensions of the data itself (say,
| grading from 0-100). At some point you need to decide how to
| scale the data (0-100) so that it draws correctly according to
| the projection (1920x1080). You can do the math yourself, or you
| can use a library like D3 to do the math for you.
|
| Even if you don't draw data on the web, you'll probably
| eventually draw data somewhere, and you'll be glad to look for an
| equivalent in that ecosystem.
| codetrotter wrote:
| One of the first things I ever programmed in PHP, of my own
| creation was a binomial distribution graph png generator.
|
| Pretty sure I had to scale the rectangles I was drawing for the
| graph according to the dimensions of the png I was making.
|
| It was of course very trivial, and nothing like the amazing
| things D3 can do. But it was a neat experience, making
| something of my own for very first time that produced an image
| on the computer.
| groggo wrote:
| That's a good way of framing the core concept.
|
| D3 was never intuitive for me. Maybe the problem is that it
| tries to do too much, at too many different level of
| abstraction? If it's just about projecting data, that's easy
| enough to understand. But I guess projection is also closely
| tied to axis and other pieces.
| treflop wrote:
| I like d3 and have built a lot of things in it but every time I
| go back to it after not using it for a few years, I can never
| remember much, the docs just confuse me more, and I feel like I'm
| back at square one.
|
| Whereas you could show me some partial differential equations to
| solve that I haven't touched in 10 years and that somehow comes
| back quicker.
|
| But I also think the d3 reference docs are absolutely horrible
| too.
| lucisferre wrote:
| I have used D3 extensively to the point of building
| integrations for AngularJS and Angular for it. I have always
| found it to be exceptionally unintuitive even if it is quite
| powerful. It is easy to get wrong, hard to debug and hard to
| grok.
|
| Not saying this is an easy problem to solve either. If I needed
| to generate some very specific data driven graphics it would
| probably still be my go to.
| gedy wrote:
| Yes same, and I'm curious of what mental modal it uses as it
| does not register with me, who otherwise has no problem
| programming.
| asdman123 wrote:
| As far as I remember it is inspired by the Grammar of
| Graphics, as is ggplot2 in the R ecosystem:
| https://link.springer.com/book/10.1007/0-387-28695-0
|
| For me it also always had this moment when it ,,clicked"
| just to disappear after not using it for a couple of
| months.
| ethbr1 wrote:
| Sounds like it's based on a good model for the problem
| space, but a model that's alien to other day-to-day ones.
|
| Which is an interesting problem!
|
| Is it better to optimally model ones problem space?
|
| Or suboptimally model it, but create a model that's
| closer to developer/user expectations?
| groggo wrote:
| So happy to read this in a few different comments. I took a
| data-viz class in college and just felt stupid because d3
| didn't click for me.
| vvpan wrote:
| I have had the same experience. As a tech lead these days I
| insist that nobody uses libraries that require a high level of
| domain knowledge. Basically code that is incomprehensible even
| for the person that wrote it should be avoided as hard as
| possible. D3 is a prime example of such a library.
| PaulHoule wrote:
| I feel it is that way much less than other plotting libs.
|
| Maybe 80% of what I do is applications, 10% systems and 10%
| data analysis. Often I will spend two weeks making Jupyter
| notebooks and then not look at it for two months.
|
| I find most plotting libs have a lot of arbitrary things to
| remember, they usually have several nano DSLs (strings that get
| interpreted in ways more complex than atoi.). On top of that
| most plotting libs have something wrong from my point of view
| in various areas like deciding the range of the axes or
| handling huge numbers of points or something.
|
| D3 makes you do a lot yourself but the interfaces it provides
| to do that are conceptually straightforward. Instead of using
| someone else's bloated and buggy general purpose code though an
| APi that feels like assembling a ship in a bottle you can just
| _do it right_ with D3.js.
| treflop wrote:
| I agree and I think that's why I end up re-learning d3 over
| and over.
|
| But having to do it yourself is a lot of work so you have to
| make an executive decision between shoehorning some plotting
| library that can't do a chart feature that honestly should be
| supported versus building an entire chart from scratch.
| uxcolumbo wrote:
| Which other library would you recommend?
| treflop wrote:
| I haven't found anything else that comes close and that's why
| I haze myself over and over.
| locallost wrote:
| I only did one bigger project with D3 and had the same
| experience! I was thinking about the same thing before I read
| your post. Not really sure why, but when I was doing it
| regularly it felt smooth, but now I can't remember anything.
| ska wrote:
| > hereas you could show me some partial differential equations
| to solve that I haven't touched in 10 years and that somehow
| comes back quicker.
|
| The HCI (human calculation interface) for calculus has been
| lovingly polished for many, many generations. A quick read over
| Newtons original texts will quickly reveal that wasn't always
| so.
|
| It seems unfair to hold an API to the standard, really.
| qwertox wrote:
| Happens to me as well, with the exception of the equation
| thing.
| hantusk wrote:
| I also have written a lot d3, between versions 2 and 7, and the
| refactoring that has happened meant a lot of examples online
| that were hard to comprehend were even harder to update.
|
| I feel like its more stable now though. Something clicks for me
| since ive started writing it in more imperative style with
| svelte+d3 rather than d3 alone. The generated elements are
| easier for me to reason about, rather than otherwise relying on
| inspecting the generated elements with dev-tools after the
| generation.
|
| This site was helpful to me, to combine d3 and svelte:
| https://svelte.recipes/
| pphysch wrote:
| D3 has an unnecessarily non-intuitive API design, and I think
| that kills it for a lot of people.
|
| For example, why must a data join be performed like
| `selection.data(array).join(element-type)` instead of something
| like `selection.join(element-type, array)`. Looking from a pure
| API-design perspective, not getting bogged down in the technical
| details.
|
| Lazy eval method chaining is a natural fit for applying+branching
| large amounts of HTML/CSS state updates but I feel that the
| designers took it a bit too far with the core D3 operations.
| hantusk wrote:
| I don't think it was for the technical fit or performance
| reasons, but more a philosophy about everything starts with
| data, and graphics are just visualizations anchored to the data
| points (or a functionally derived property of the data points).
|
| That also means the d3-* libraries compose really well, since
| the data is the common binding, and not some conceptual class
| or custom element.
| blackhaj7 wrote:
| This was the only resource that made D3 make sense for me. The
| author is responsive and helpful too.
| PaulHoule wrote:
| D3 is the first visualization toolkit that I really liked.
| darknavi wrote:
| I just picked up D3 + React for a VS Code webview project. Pretty
| easy to pick up but I find jumping in with examples very black
| box (as someone who doesn't have a ton of web experience). That
| plus going from JS to TS had some quirks that I didn't realize
| when in JS land.
|
| I could really use a "start from the basics" tutorial.
|
| I will say Github Copilot has been excellent at helping me with
| random TS errors when porting my graphs from JS to TS. It's nice
| how typed D3 can be when you finally opt into it.
| bilater wrote:
| I love the visualizations created but something about D3 syntax
| has just never clicked for me. It seems overly convoluted,
| verbose and from an old era. We've thankfully moved to a more
| declarative way of doing things.
| laurels-marts wrote:
| D3 is from jQuery era so there are a lot of methods for
| manually manipulating the DOM like d3.select() and then
| selection.attr() etc. but those patterns tend to lead to
| components that are lengthy, hard to read and hard to maintain.
| Arguably a better approach is to group each D3 module based on
| whether it "modifies DOM" or "doesn't modify DOM". Then use
| React/Svelte to declaratively render the DOM and only use the
| D3 modules from the latter group for their math utilities.
| lloydatkinson wrote:
| I keep searching for entirely not-web based diagramming/charting
| solutions. That can be driven via CLI or using some library, and
| output SVG or PNG.
|
| All of these things usually depend on browser functionality or
| use some janky as fuck SSR solution using a headless browser to
| generate them server side.
|
| Of particular note, I'd also like to be able to programmatically
| generate Venn diagrams. It seems there is, bizarrely, a complete
| dearth of anything in this space. A couple of JavaScript chart
| libraries support them, but generally speaking there seems to be
| hardly any support for them.
|
| gnuplot seems good - just no Venn diagram support.
| wbeckler wrote:
| I've heard great things about Vega [1], which sits on top of D3.
| It's a dependency of OpenSearch Dashboards, allowing users to
| create custom dashboards on log and observability data [2]. The
| vega library might alleviate some of the concerns others are
| expressing about the learnability of D3.
|
| [1] https://vega.github.io/vega/docs/ [2]
| https://opensearch.org/docs/latest/dashboards/visualize/vega...
| javajosh wrote:
| The real problem with D3 is that, as a library, it doesn't really
| do visualizations. It gives you functions and exmaples to compose
| your own visualizations, whose key requirement is an expert-level
| understanding of JavaScript (particularly closures and functional
| programming). What mbostock says is the core "abstraction" of D3,
| the select and select diffing, is only an affordance for writing
| reentrant "paint()", with the flourish that your paint statements
| are expressed as builder statements, which themselves are scoped
| to select and can be broken up for various effects. Especially in
| animated examples, D3 does suprisingly little for you, as it's up
| to you to write the outer loop and track state over time. Nor
| does D3 give you any abstraction for encapsulating a
| visualization, as a function or otherwise. This is left as an
| exercise for the reader. All of this comes as a big surprise, or
| shock, to the new D3 user. It would be like adopting a new
| database and finding only a library for file IO and BTree indices
| and query optimizers, that you need to wire up yourself.
| stared wrote:
| Well, D3 is not a set of tools for charts (alike matplotlib,
| ggplot or so), but a low-level library to create your data
| visualization from scratch. Depending on your goal, it may or
| may not be what you need.
| stared wrote:
| I've been in love with D3.js and have used it as the key
| technology for dozens of data visualization projects, both
| commercial and personal. However, its rendering syntax now feels
| dated, reminiscent of the jQuery era. While I still use various
| auxiliary functions (e.g., for ease or scaling), I now create
| actual components in a modern framework (in my case, Vue 3 and
| TypeScript).
|
| Despite this shift, I am grateful to D3.js for the beautiful
| years it gave me and for teaching me various good practices in
| data visualization. The defaults and demos by Mike Bostock have
| been both an inspiration and an example of good standards,
| aligned with Tufte's minimalism.
| williamcotton wrote:
| D3 is inspired by the Grammar of Graphics by Wilkinson.
|
| Another tool also inspired by GG is aptly named ggplot2 and for
| the R language. It offers a much more terse and functional
| inspired interface that also works well with an uncommon tool
| named Rush, R one-liners for the Shell, for quickly creating
| graphics from CSVs piped into the tool.
| breadwinner wrote:
| D3 is a terrific toolkit. The fact that it is a low-level toolkit
| means you have to write more code, but on the other hand, you
| have more power. One of the negatives of D3 is that D3 charts are
| not responsive by default. Whether the chart fills the screen, or
| fills a small area in a dashboard, the grid lines are spaced the
| same and fonts are the same size. You can add responsiveness
| manually. It would be nice if a future version was responsive out
| of the box.
| amelius wrote:
| Can anyone recommend a good plotting library for Python, that is
| not matplotlib?
| HereBeBeasties wrote:
| https://opensource.com/article/20/4/plot-data-python gives some
| common options. What kind of plots are you trying to achieve?
| Interactive? Jupyter notebooks? Reporting? SVG or HTML output?
| You might also like to look at things like https://evidence.dev
| tompark wrote:
| Vega/Vega-lite has a python interface called Altair.
|
| I haven't used Altair but I wanted something higher level than
| D3 (which I've used but it feels fairly low level) so I went
| through the learning curve for Vega.js, and it could handle
| every usecase that I tried, some of which were pretty complex.
___________________________________________________________________
(page generated 2024-05-16 23:00 UTC)