[HN Gopher] Debugging GraphQL N+1 Issues with Open Source Tracin...
___________________________________________________________________
Debugging GraphQL N+1 Issues with Open Source Tracing Tools
Author : soaringmonchi
Score : 27 points
Date : 2023-07-04 13:53 UTC (9 hours ago)
(HTM) web link (kickstage.com)
(TXT) w3m dump (kickstage.com)
| rawling wrote:
| When my tech lead decided we were going to move to GraphQL, one
| of the reasons I was told it was great was because it
| solves/avoids the n+1 problem.
|
| But from this it sounds like that's something you have to detect
| and code for directly... which we'd have happily done in the
| existing code anyway?
| andrewingram wrote:
| It doesn't solve it, but it lets you build a system that solves
| it for all future queries going forward. Think of it as a
| platform for taking endpoint specifications (ie GraphQL
| queries) and satisfying them efficiently. You have to build the
| part which makes it efficient, but once that's done, it's done.
| SkyPuncher wrote:
| N+1s are a fundamentally a data modeling and querying issue.
|
| People who complain about GraphQL having a lot of N+1s are
| complaining about their lack of fundamental knowledge. You can
| build a GraphQL with minimal N+1s and you can build a
| "standard" API with loads of N+1s
| latchkey wrote:
| I made this related comment about N+1 on a netflix graphql post
| a few days ago... some of the responses might be interesting to
| you.
|
| https://news.ycombinator.com/item?id=36382212
| throwaway2990 wrote:
| Sorry to hear you have an ill informed tech lead :(
| gavinray wrote:
| > one of the reasons I was told it was great was because it
| solves/avoids the n+1 problem
|
| So, the thing about GraphQL is that it's a _specification_, in
| the same way that REST or gRPC are specifications.
|
| There's nothing inherent in either GraphQL or REST that admits
| or prevents the N+1 problem. You have endpoints with business
| logic (or in GQL's case, "resolvers") and there's no telling
| what's inside of those methods code.
|
| It's true that GraphQL was built to integrate with tools like a
| "Dataloader" to solve "N+1", but you could just as validly be
| using a Dataloader in your REST or gRPC API endpoints.
| e1g wrote:
| GraphQL eliminates the n+1 problem _for the client_ by pushing
| it to the server. Generally, the client <->backend connection
| is slow and spiky, and the backend<->db connection is fast and
| stable, so this trade-off made sense for Facebook as they had
| more and more mobile users connecting via 3G networks in
| developing countries.
| swid wrote:
| I have used Dataloaders in the past in both Go and Typescript
| and it just about solves the issue automatically. Dataloader is
| a specific library that is copied for different languages that
| allows you to write code that looks like N+1, but it will group
| the N part into batches, so it becomes just 1+1. It's kind of
| weird to explain, but it should be easy to understand once you
| start working with them.
| troupo wrote:
| Dataloaders replace N+1 with E queries (where E is the number
| of separate entity types to fetch) and an in-memory merging
| of all datasets (huge if you're not careful).
|
| They are _a_ solution, but not necessarily a good one. And
| GraphQL doesn 't lend itself to goos solutions.
| swid wrote:
| N+1 only makes sense for E=2. For each parent in the list,
| fetch the child's data. If there were multiple children, or
| if the children had their own children fetched 1 at a time,
| it would no longer be N+1.
|
| So I think 1+1 (or 1+N/batch size) is still correct.
| andrewingram wrote:
| For the first production-scale GraphQL servers I built, I just
| had it dump ascii art representations of the downstream request
| waterfalls during dev. This was enough to visualise and fix any
| n+1 issues.
| jensneuse wrote:
| It's great that we're talking about tracing / OTEL (Open
| Telemetry) in the context of GraphQL. We've recently added OTEL
| support to WunderGraph, allowing our users to trace through the
| Gateway and their services. Some more info on our OSS
| implementation can be found here:
| https://docs.wundergraph.com/docs/guides/monitoring-and-obse...
|
| In addition, we've deployed all the required infrastructure like
| an OTEL Collector and OTEL backend using Clickhouse to offer a
| managed end-to-end solution including dashboard for traces and a
| trace overview. A live demo can be found here (sign up for free
| into the demo org): https://cloud.wundergraph.com/wundergraph-
| demo/apollo-federa...
___________________________________________________________________
(page generated 2023-07-04 23:03 UTC)