[HN Gopher] How to Design Better APIs
___________________________________________________________________
How to Design Better APIs
Author : adrianomartins
Score : 437 points
Date : 2022-03-12 00:25 UTC (22 hours ago)
(HTM) web link (r.bluethl.net)
(TXT) w3m dump (r.bluethl.net)
| dimitrios1 wrote:
| For #2, better yet, prefer RFC 3339.
| zgiber wrote:
| Please don't specify ISO8601 as the date format. That standard
| embodies dozens of formats. Use one specific format instead like
| RFC3339.
| drdrey wrote:
| Why would one prefer ISO 8601 dates over POSIX timestamps?
| paulryanrogers wrote:
| Human readability most likely
| mulmboy wrote:
| They can include a local timezone. Sometimes there's a big
| difference between 12am in UTC+0 and 3am in UTC+3 despite
| representing the same instant in time.
| inopinatus wrote:
| True enough, but I would still recommend that API responses
| normalize to UTC (Z suffix) in the general case and document
| as much, and if actually returning a timestamp with a
| specific timezone, document the intended meaning.
| guhidalg wrote:
| +1, if your application cares about time then you should
| just tell your users all dates will be normalized to UTC
| and they are responsible for displaying them in a preferred
| time zone.
| codebje wrote:
| Time zone rules change relatively frequently - future
| dates may be better stored in the appropriate time zone,
| optionally along with the UTC offset at the time of
| recording, so you don't report incorrect information when
| the rules do change on you.
|
| Past dates should always be in UTC, for the same reason -
| timezone rule changes are sometimes even retroactive.
| hn_throwaway_99 wrote:
| 1. Human readability
|
| 2. The ISO date format is "big endian" by default, which makes
| it trivial to chunk and compare just specific date parts. Want
| all events that occurred on the same (UTC) date? Just do
| dateField.substring(0, 10). Everything in the same year?
| dateField.substring(0, 4).
| seelmobile wrote:
| I prefer them because they're human readable and can preserve
| time zones if that's important
| djbusby wrote:
| 8601 has a large pattern space - RFC 3339 is a narrower subset
| of ISO. Somewhere I saw a diagram that convinced me. I link if
| I can find again.
|
| Edit: a relevant link
|
| https://news.ycombinator.com/item?id=28976526
| worik wrote:
| https://ijmacd.github.io/rfc3339-iso8601/
|
| A ven diagram very cool
| djbusby wrote:
| That's the one! Thanks!
| inopinatus wrote:
| * Human readable
|
| * Supports birthdays for people older than 52
|
| * Reliable after 2038
|
| * Supports leap seconds
|
| * HTML date/time spec is a subset
|
| * String collation order matches temporal order
|
| * Excel
| latch wrote:
| > * Human readable
|
| Computers are the main consumers of APIs, and ISO 8601 is far
| from machine-readable.
|
| For example, in Elixir, DateTime.from_iso8601/1 won't
| recognize "2022-03-12T07:36:08" even though it's valid. I had
| to rewrite a chunk of Python's radidjson wrapper to 1-9 digit
| fractional seconds (1).
|
| I'm willing to bet 99% of ISO8601 will fail to handle all
| aspects of the spec. So when you say "ISO8601" what you're
| really saying is "our [probably undocumented, and possibly
| different depending on what system you're hitting] version of
| the ISO-86001 spec."
|
| (1) https://github.com/python-rapidjson/python-
| rapidjson/pull/13...
| inopinatus wrote:
| > what you're really saying
|
| No.
|
| The specific profile of ISO8601 that should be used to
| express timestamps in an API is that defined in RFC3339.
|
| Choosing to use a half-baked parser is a separate matter.
| wokwokwok wrote:
| I'm gonna say it:
|
| Many rest apis are lazy and developer friendly, not consumer
| friendly.
|
| If you have related resources, let's say, product and product
| options as two distinct endpoints:
|
| - /api/product
|
| - /api/options
|
| Then, and I want to be clear here, it is _impossible_ for a
| client to perform an atomic operation on multiple distinct
| objects types.
|
| Let's say the client needs to add a product with a single option
| or fail.
|
| You can create a product.
|
| You can create an option.
|
| You can add an option to a product.
|
| ...but, at some point in time, a product will exist with no
| options on it.
|
| This kind of "pure" rest api is simply convenient to the
| _developer_ because they push the problem of object consistency
| to the client.
|
| ...but that's not a client concern.
|
| If a product needs to be associated with an option on creation,
| then your api should offer that as an endpoint.
|
| It doesn't meet the "standard" for a rest api?
|
| Too bad.
|
| Write APIs that do what the customer / consumer needs _not_ lazy
| APIs that just make your life easier.
|
| I've worked with a lot of APIs and you know what I do not give
| the tiniest moment of care about?
|
| Consistency.
|
| I do. Not. Care. If POST /api/foo creates an object, or if the
| endpoint is /api/foo/create.
|
| Messy? Inconsistent?
|
| I don't care. Just put it in the documentation and I'll find it
| and use it.
|
| ...but if your api forces object relational consistency onto me
| as an api consumer, you have no _idea_ how much pain and hassle
| you've caused me having to implement my own set of fake
| transactions over your stupid api.
|
| Please, write APIs for consumers, not according to "the rules" of
| rest APIs.
|
| ...and provide documentation. :)
| bmn__ wrote:
| > it is _impossible_ for a client to perform an atomic
| operation on multiple distinct objects types.
|
| > It doesn't meet the "standard" for a rest api? Too bad.
|
| It is perfectly fine to model the creation of a product with
| options via a new resource. Another possibility is to model
| this as a transaction, a series of requests. Both meet the
| goals of atomicity and also following restful constraints.
|
| But since you disallowed those possibilities as the premise,
| there is no constructive way forward from there. Nice
| construction of a straw-man, must feel satisfying to see it
| topple over, but who do you hope to impress with that?
| 1123581321 wrote:
| Agree that the tendency exists but don't see the tradeoff
| between transactional and messy in this example--you should be
| able to create a new product with options [...] in the same
| request, perhaps using /options to ascertain what's available
| before posting to /products, depending on the situation.
| shoo wrote:
| maybe a more illustrative example of where this can be messy
| is where a customer has placed an order for a bundle of
| products A B and C, and there's no "order bundle" API that
| can do that operation atomically, instead the client code
| trying to place the order has to call a bunch of random
| stateful create-product APIs to attempt to construct each
| product in the bundle, and deal with the mess if suddenly
| some bundle items error while others succeed. bonus points if
| some of the create-product APIs are not idempotent.
|
| "congratulations, we've provisioned the bundle A, C, C, C
| that you ordered! sorry about B!"
| parksy wrote:
| Everyone has different experiences but my view of the problem
| is that it's not so much down to developer laziness as it is
| technological ignorance and reactive direction from higher up -
| the people who actually fund and approve the work.
|
| Outside of the bigger providers I don't think most APIs are
| designed with general consumption in mind. The impetus usually
| seems to be reactive - "we've got this idea for a mobile app
| using data from our legacy WMS/CRM/whatever, and it needs to be
| finished yesterday!"
|
| So generally (not always but usually) what I've seen is
| whatever the underlying system supports and requires gets
| reflected into the API, and there's minimal proper engineering
| or sensible abstraction, partly because of time limits and also
| because the requirements are discovered mid-flight.
|
| "Oh... Products have options in a separate entity? Oops. Uh,
| we'll add an options endpoint. Easy."
|
| Several years later, the app is mildly successful, the business
| decides to whitelabel the API for some of their B2B clients,
| there's no budget for a redesign, "It works doesn't it? If it
| ain't broke don't fix it..."
|
| Where possible I try to design and build APIs with some
| forethought and push for as much information on requirements
| and capabilities of existing systems beforehand but quite often
| the stakeholders don't really know and don't want to pay for a
| few weeks of in-depth analysis.
|
| The endless facepalming of hearing the cycle of "Don't
| overengineer it" when I ask too many questions people can't
| answer, through to "Why didn't we pick up on this requirement"
| and having to listen to "we can all learn from this" when
| something fails because of some if/else statement buried in the
| legacy business logic, right back to "Don't overengineer it"
| when you propose the clean way of encapsulating that in the
| API, has left a permanent indent on my skull. So much leakage
| of black-box logic into the clients which bloats frontend logic
| and provides plenty of double handling and scope for unknown
| business logic to be missed, and no one in charge really gives
| a shit other than the developers and engineers, and that's fine
| because we can deal with it and it's their fault for not
| thinking of it anyway... they're the smart ones right?
|
| You'd think this is just a problem with SMEs but some of the
| biggest national and multinationals I've worked with have a ton
| of horrid shit in their API designs. One industry-wide
| government-mandated reporting spec you'd post a SOAP envelope
| that wraps an undocumented XML document (they had an example
| document and the envelope was documented, but no doctype or
| anything to go off for the contents), a major real estate
| interchange format has you polling a REST API for an update
| flag, then loading and saving CSVs over FTP. Some godawful
| affiliate marketing API essentially just exposed their nth
| normal form relational database as an API. One government
| department just used HTSQL for their API using a python script
| to reflect some of their silos into PGSQL and called it a day
| (even though whether or not your interactions would work or
| made sense depended entirely on their internal application
| logic which wasn't represented or documented).
|
| And too many projects where it turns out the promised APIs
| didn't even exist until the app was well into development.
| "Don't worry about it, that's their problem and gives us an
| excuse when the inevitable delays occur..."
|
| No wonder we're hearing of massive breaches daily, from top to
| bottom the entire industry is barely held together by glue and
| sticky tape. It seems like an engineering problem but I think
| it goes much higher than that, it's a cross-industry failure in
| planning and management. Engineers are not given the time and
| budget to do the job that the data deserves. The market self
| corrects though, massive breaches have closed more than one
| business down for good.
|
| It feels futile but I do support and encourage people to work
| towards just reasonable and well-documented endpoints, I doubt
| there's one true standard, but just some level above slapping
| it together at the last minute would be nice. I don't care if
| it's JSON or XML, or whether paths follow some semantics or
| another. As long as the relationships, types, and mandatory vs
| optional fields are explained clearly and succinctly, and the
| auth mechanism isn't a nightmare, and I can work with it.
|
| Sorry for the rant this is just one area that triggers me. I've
| seen too much bad practice, and fighting against it for decades
| and still hearing the same lack of concern in the architecture
| phase has left me more than a little jaded.
| alkonaut wrote:
| If options are children of products and not many-to-many then
| the only logical way to have two endpoints in the API is if the
| parent (Aggregate root in DDD speak) items are
| readable/writable as a consistent tree, while the query for
| child items in /api/options is a read only api.
|
| There is nothing nonstandard about such a REST api. If a
| product must be associated with an option then the
| /product/create endpoint should only accept a product with an
| option already attached.
|
| > This kind of "pure" rest api is simply convenient to the
| developer because they push the problem of object consistency
| to the client.
|
| It's not like REST means that to be "pure" or "standard" you
| end up exposing all your database tables with read/write and
| just let callers CRUD anything they want, including creating
| inconsistent data?
|
| > "the rules" of rest APIs.
|
| The rules are very simple: the REST api should expose and
| enforce the consistency rules of the business logic!
| underwater wrote:
| Generally internal APIs are developed alongside one or two
| apps. They don't need a pure, resource-oriented, API that
| perfectly represents the domain models but ignores how the API
| is used in practice.
|
| A good example is the tip on PUT vs PATCH to update objects.
| That seems to be missing the point. Why are you forcing the
| clients to calculate the correct resource fields to PATCH to
| the server? This is supposed to be an API, not a database. Just
| expose methods that correspond to the actions that the users
| will perform.
|
| Sure, HTTP only has 5 verbs, but that doesn't mean your API
| should solely consist of five possible interactions on a
| resource.
| shoo wrote:
| structured programming and rpc demonstrates you can get
| pretty far if your only verb is CALL
| kortex wrote:
| > Sure, HTTP only has 5 verbs
|
| More like 40.
|
| You can also just like...make up your own verbs, the HTTP
| police won't arrest you, though this rustles the jimmies of
| HTTP purists. No guarantees any middleware will support it,
| and this is probably a bad idea if you actually do this in
| public apis, it's more so "hey, these verbs aren't magic
| incantations, it's just a string which gets switch-cased".
|
| Same with HTTP status codes. There's obviously some well
| known ones but you can just like...make up your own.
|
| https://www.iana.org/assignments/http-methods/http-
| methods.x...
| shoo wrote:
| i was in the process of writing a snarky reply describing how
| the next level enterprise approach is to host api/product in
| microservice A and api/options in microservice B, each with
| their own internal data store using completely different tech
| stacks.
|
| but, if you've already suffered through the pain of attempting
| to implement some kind of best-effort ad-hoc distributed commit
| logic in your client code, then needing to call two completely
| different services for api/product and api/options doesn't
| really make anything _worse_.
|
| on another hand, it doesn't make anything better.
| janaagaard wrote:
| My recommendation: Use PUT everywhere instead of POST. PUT has to
| be idempotent, so if the request fails, the client can simply
| post it again. This solves issue like worrying about creating a
| second copy of an item if a POST timed out.
|
| Using PUT to create elements means that the client has to supply
| the ID, but this is easily solved by using GUIDs as IDs. Most
| languages have a package for create a unique GUIDs.
| systemvoltage wrote:
| What are the use cases where idempotency is not needed or even
| harmful?
| gls2ro wrote:
| I don't think we should strive to remove non-idempotent cases.
| If something is not idempotent does not mean it is bad. It just
| means that request should be handled differently.
|
| In your example (and I ask this as I remained confused after
| also reading SO):
|
| Let's say that you need the client to provide the ID in the
| request body.
|
| In this case, how is using PUT when creating a new resource
| idempotent if the ID should be unique and you have a constraint
| on the DB level for example?
|
| What happens when the second call will be made with the same
| ID?
|
| If I execute the following sequence:
|
| Call 1: PUT resource + request.body {id: 1} => a new record is
| created so the state of the system changes
|
| Call 2: PUT resource + request.body {id: 1} => no record is
| created, maybe the existing one is updated
|
| IMO this is not idempotent nor should it be. Creating a new
| resource is not an idempotent operation.
|
| I also don't like that depending on the state of the DB two
| requests with the same attributes will have different impacts
| on the system.
|
| In my mind as a consumer of an API it is simpler: POST it
| creates a new record so I know what to expect, PUT updates an
| existing one.
| egberts1 wrote:
| For those in Security Theatre of HTTP APIs, I found this also
| useful:
|
| https://github.com/yosriady/api-development-tools
| 11235813213455 wrote:
| Something I wonder is how to design search/list endpoints where
| the query can be long (a list of IDs, ex: /users?ids=123e4567-e89
| b-12d3-a456-426614174000,123e4567-e89b-12d3-a456-426614174001,123
| e4567-e89b-12d3-a456-426614174002,...), so long that it can
| exceed the url max length (2048), after 50 UUIDs, you can quickly
| exceed that length, so GET is not ideal, so which method, SEARCH
| with a body? POST with a body?
| flowerbreeze wrote:
| I would go with POST with a body in that case, where I
| interpret is as a "new search item" and use GET to scan through
| results, if there are many results available. I don't think
| I've needed something like this more than once or twice though.
| From users perspective, asking information on specific 50 items
| at once is not something commonly done.
| 11235813213455 wrote:
| it's used for an export feature, where we join data from the
| client-side, not ideal, but for now it's a good compromise in
| terms of complexity for the API and client
| daniel-thompson wrote:
| There was a new RFC published a few months ago to address this
| use case. It defines a new HTTP method QUERY, which is defined
| to be safe & idempotent like GET and explicitly allows a
| request body like POST. See https://www.ietf.org/id/draft-ietf-
| httpbis-safe-method-w-bod...
| 11235813213455 wrote:
| great, thanks for the info!
| athrowaway3z wrote:
| The type signature:
|
| ` list of uuids -> list of results `
|
| is usually bad design to begin with.
| antifa wrote:
| If QUERY is too new, I was always a fan of base62 for URLs, but
| base64 and straight binary encoding could do well for
| compacting UUID lists, they are essentially just giant
| verbosely written integers.
| trinovantes wrote:
| Do you guys use plural or singular terms in your API endpoints or
| both? /books /books/:id /book
| /book/:id
|
| It gets harder to keep consistent when there are a lot of nouns
| that have the same singular/plural form like "clothing"
| akvadrako wrote:
| I always prefer singular just because it's easier to spell if
| you know the type.
|
| It's stupid to have /geese return a Goose.
|
| If the APIs were in Esperanto it would be different.
| imnitishng wrote:
| Both /books /book/:id
| systemvoltage wrote:
| /clothes/:clothing_id
|
| The reason for using plural is because without the
| `:clothing_id`, `/clothes` endpoint would return a collection
| of clothing.
| barefeg wrote:
| We use the form that represents a collection of objects. For
| example books and clothing both refer to a collection of things
| so I take them as valid forms
| ramraj07 wrote:
| To resurface the debate, is including related endpoint urls pre-
| filled in a "misc" field a good idea?
| softfalcon wrote:
| Reading this makes me feel like a take all the magic GraphQL does
| for granted
| cyb_ wrote:
| Google published a similar recommendations:
| https://google.aip.dev/
| akavel wrote:
| AIPs are very deep and comprehensive, I would say "similar" is
| a diplomatic understatement here :)
| pokoleo wrote:
| The error messages could be better yet.
|
| The example uses a different code per issue, for instance:
| "user/email_required". Most integrators will build their UI to
| highlight the input fields that contain an error. Making them
| parse the `code` field (or special-case each possible code) is
| pretty toilsome. // from blog post {
| "code": "user/email_required", "message": "The
| parameter [email] is required." }
|
| Make it parseable: // improved {
| "message": "An email is required.", "error":
| "missing_parameter", "parameter": "user.email"
| }
|
| In addition, I:
|
| * rewrote `message` to be an acceptable error message displayed
| to (non-technical) end-users
|
| * moved message to be the first field: some developer tools will
| truncate the JSON response body when presenting it in the stack
| trace.
|
| ---
|
| As an added bonus, structured data allows you to analyze `error`
| frequencies and improve frontend validation or write better error
| messages:
| https://twitter.com/VicVijayakumar/status/149509216182142976...
| hinkley wrote:
| Localization has entered the chat.
|
| You need codes because the field isn't going to be 'email' for
| much longer than it takes for your management to realize that
| people outside of the US also have wallets.
| chaz6 wrote:
| My view is that apis should simply return a number when an
| error occurs. The vendor should supply a list of error codes
| to its consumers, translated into as many languages as
| necessary. The developers who are consuming the api can then
| determine how best to present the error to its end users. A
| set of error numbers is tied to the version of the api that
| is being consumed so there should be no surprises.
| codys wrote:
| Field ids are not (necessarily, especially when doing
| localization) something shown in the UI. The point made by
| the original commenter is that a field in the error should
| refer directly to which field has an issue. It does, via an
| id that happens to be "email". It's still up to the clients
| to decide how to represent that to the user, but they're
| given a distinct field rather than needing to infer which
| field an error code refers to.
|
| (While the comment I replied to can be read differently, I
| assume we all know that changing actual field names (in APIs)
| depending on localization is nuts)
| aidos wrote:
| No doubt they exist, but I've never seen an api that
| localised identifiers.
| moltar wrote:
| Agreed. But rather than reinvent, let's just use JSON API
| standard?
|
| https://jsonapi.org/format/
|
| (Scroll to the very bottom)
| pokoleo wrote:
| The article is about REST API design, not JSON APIs! It's a
| whole different ballpark.
| pan69 wrote:
| It might be to formal for your use-case, but there is a
| standard defined for error responses in RFC 7807:
|
| https://datatracker.ietf.org/doc/html/rfc7807
| AtNightWeCode wrote:
| That is stupid. Most of our failed requests are logged and
| logs are only read by dashboards and alarms. Sure, you can
| have a friendly message too but formalizing the errors in a
| structured way simplifies things and also improves the
| performance when scanning through large amount of logs.
| aeontech wrote:
| Wow, I had never seen an API with errors at this level of
| detail... I feel lucky when they at least use sane status
| codes instead of always giving back 200 and a maybe-json-
| maybe-plaintext-maybe-empty body...
|
| I'd love to hear from anyone who has encountered APIs in the
| wild that actually implement this standard!
| amjd wrote:
| I used to work at Akamai and Problem Details is used in
| most of their APIs. It might have something to do with the
| fact that one of the RFC authors (Mark Nottingham / @mnot
| on HN) worked there for a while.
| JimDabell wrote:
| I use Problem Details in most APIs I build. It's as simple
| to generate a Problem Detail as it is to generate ad-hoc
| errors, but you can re-use code.
| abdusco wrote:
| ASP.NET Core uses ProblemDetails ~~by default~~.
|
| https://docs.microsoft.com/en-us/aspnet/core/web-
| api/handle-...
|
| https://docs.microsoft.com/en-
| us/dotnet/api/microsoft.aspnet...
| id02009 wrote:
| I think stripe is close to having decent API in this regard
| morelisp wrote:
| I've used both Problem Details and JSONAPI errors[0] which
| are basically the same idea (and I've used them plenty
| outside of JSONAPI-proper APIs). In both cases if you have
| a decent error-handling middleware there should be not much
| difference than outputting any other kind of errors.
|
| One thing to keep in mind re. "maybe-json-maybe-plaintext-
| maybe-empty" responses is that the more complex your
| errors, the more likely the error handling encounters an
| error. If you're trying to send back some JSON or XML but
| it fails it's usually better to at least shove out a line
| of plain text and hope it reaches a human than to mask the
| real error with a second fallback in the "right format" but
| with unhelpful fixed content.
|
| [0] https://jsonapi.org/format/#error-objects
| b-pagis wrote:
| Such simple approach is limited only to errors without
| arguments.
|
| For more complex use cases, where we would want an error
| message to indicate that field value was too long and in
| addition provide maximum field length, we would need to
| introduce new field in the error response.
|
| While it is solvable by adding this information to client
| application side. It would create a situation where the logic
| is duplicated in two places (backend and client application)...
|
| Also if we would want better UX, then we would need to display
| all errors at the same time in the form that is incorrectly
| filled. This would require changing error structure to return
| array of errors and it potentially create a breaking change in
| the API or would result in confusing structure that supports
| both, legacy and new format...
|
| Some years ago, I wrote an article sharing the ideas on how
| REST API error structuring could be done depending on the
| complexity of application or service:
| https://link.medium.com/ObW78jhDkob
| pokoleo wrote:
| Interesting! Do you find that returning an array of errors
| works in practice?
|
| Most validation I've seen looks like: raise
| error if foo raise other_error if bar
|
| This pattern turns into one exception per response, and some
| foresight in architecting exceptions would be needed
| smoyer wrote:
| It also doesn't hurt to repeat the HTTP status code in the JSON
| body - when you receive a response, the status code and entity
| body are coupled but even if the server logs the status code,
| they're often decoupled in the logging system - having both in
| one log entry is way easier!
| solatic wrote:
| a) Use standardized error codes, not standardized error messages.
| Clients are responsible for internationalization, which includes
| presenting error messages in the user's language. If you document
| a set of error codes as an enum, the client can present a user-
| friendly error message in the user's language based on the error
| code. If there are dynamic parts of the error message, i.e. "404:
| There is no user with ID 123456 in the system", then the user ID
| should be extracted into the error response body, so that it can
| be provided correctly to the user in the user's language.
|
| b) Pagination is the devil. The state of the server can change
| while the user is paginating, leading to fragile clients. Don't
| paginate your API. If you think you have a need to paginate, have
| one API call return a list of IDs, and have a separate API call
| return a list of resources for a given list of IDs, where the
| second API call accepts some maximum number of IDs as a query
| parameter. This ensures consistency from one API call to the
| next.
| nuttingd wrote:
| Pagination is harder than it seems to get right.
|
| I think pagination is only predictable under these conditions:
|
| 1) The offset used for the next fetch must be based on a
| pointer to a unique key. We can't rely on the number of rows
| previously seen.
|
| With this rule, deletes which occur to rows in previous pages
| will not cause unpredictable contractions.
|
| 2) The paged result set must be sorted based on a monotonically
| increasing field, like created_at, plus enough other fields to
| create a unique key. You could lean on the PK for this, i.e.:
| ORDER BY (created_at, id) ASC.
|
| With this rule, new inserts which occur during page enumeration
| will only affect unseen pages (and we'll see them eventually)
|
| The API call looks roughly like this: /orders/?
| region=US&offset=(2022-03-12T07:05:58Z&ord_1234)&limit=100
|
| The DB query looks roughly like this: SELECT *
| FROM orders WHERE (created_at, id) > (:offset_created_at,
| :offset_id) OR ( :offset_created_at IS NULL
| AND :offset_id IS NULL ) ORDER BY (created_at, id)
| ASC LIMIT :page_size
|
| EDIT: formatting
| alkonaut wrote:
| Any pagination is brittle. Regardless of whether its by page,
| cursor or something else. You can't have equal sized pages if
| there is a risk that there are deletions on a previous page or
| at the exact index you use as a cursor etc.
|
| The solution is usually simple: assume it doesn't matter. Write
| a spec for your feature and explicitly state that "In the
| solution we assume it doesn't matter if the same record is
| ocassionally reported twice or a record is missing from the
| pagination in some cases". Done.
| JimDabell wrote:
| > Pagination is the devil. The state of the server can change
| while the user is paginating, leading to fragile clients. Don't
| paginate your API.
|
| The problem is not pagination, and the solution is not to avoid
| pagination. The problem is offset-based pagination, and the
| solution is to use cursor-based pagination.
|
| > If you think you have a need to paginate, have one API call
| return a list of IDs, and have a separate API call return a
| list of resources for a given list of IDs, where the second API
| call accepts some maximum number of IDs as a query parameter.
|
| This has the same problem as you described above. The state of
| the server can change between fetching the list of IDs and
| operating on them.
| solatic wrote:
| Cursor-based pagination doesn't solve your state issue, it
| forces the server to create a copy of state for the cursor
| request. This is complex to implement correctly - for
| example, if the user does not actually paginate through all
| the entries, when do you dump the unused cursor? If the user
| issues the same request over and over, do you return a cached
| cursor or re-copy the state into a new cursor? If you re-copy
| the state, are you defended from a malicious actor who sends
| 1,000 new requests? Of course, all these concerns can be
| mitigated, but it's easier to just design without pagination
| in the first place if you can.
|
| > The state of the server can change between fetching the
| list of IDs and operating on them.
|
| Right, for example, a returned ID may have been deleted
| before the user can issue a query for that resource. But this
| is usually far more comprehensible to clients, particularly
| if the resource requires authorization such that only the
| client is permitted to delete that resource.
| sigmaml wrote:
| You appear to be referring to a database cursor.
|
| It is quite simple to implement pagination in the
| application layer using a unique record identifier (primary
| key or ULID or ...) as an anchor for the navigation. From
| that unique ID, we can then fetch the previous `n` or the
| next `n` records, depending on the direction of the
| navigation.
|
| This way, the server remains stateless, since the anchor
| (possibly sent as an encoded / obfuscated token, which can
| include some other parameters such as page size) is
| supplied by the client with each pagination request.
|
| Unless I am missing something in your argument.
| borrow wrote:
| What if that particular unique ID is deleted right before
| the client requests the next page?
| sigmaml wrote:
| 1. This kind of pagination can be done for any key as
| long as its data type admits total ordering.
|
| 2. The WHERE condition typically uses `>` or `<`, so it
| doesn't fail even when that record is deleted before the
| next client request referring to it.
| JimDabell wrote:
| Pagination only makes sense in the context of an ordered
| collection; if there is no stable sort order then you
| can't paginate. So you identify the last record seen with
| whatever fields you are ordering by, and if the last
| record has been deleted, then it doesn't matter because
| you are only fetching the items greater than those values
| according to the sort order.
|
| Anyway, there is plenty of documentation out there for
| cursor-based pagination; Hacker News comments isn't the
| right place to explain the implementation details.
| pronik wrote:
| If the IDs are monotonous, it doesn't matter.
| JimDabell wrote:
| Cursor-based pagination doesn't require anything at all
| like you describe. Are you sure you aren't mistaking it for
| something else?
| akvadrako wrote:
| You should use stable page boundaries instead of cursors. If
| you are returning results by timestamps, the page boundary
| should be the timestamp and secondary ordering keys of the
| last row returned.
|
| Cursors take too many server resources and require client
| affinity.
| JimDabell wrote:
| You're describing cursor-based pagination. It's got nothing
| to do with the SQL concept of cursors.
| eyelidlessness wrote:
| Pagination _by offset_ can be bad. Pagination _by cursor_ which
| is specifically designed for your list of resources is
| perfectly reasonable. Use stable pagination queries, not ?*
| CrimsonRain wrote:
| b -> What happens if the "list of IDs" is 10k or 100k or 1M+?
| solatic wrote:
| There's typically business-logic ways to require a range to
| be specified, and then you specify a maximum range. For
| example, if the resource in question is audit events, you
| require the user to specify a time range, up to a maximum of
| a day, or seven days, or whatever performance / budget
| constraints allow for.
| CrimsonRain wrote:
| So if I specify a range and something changes between in
| that range while I'm interacting with those, sounds like
| same problem is back? Just same issue; different package :)
|
| If I do a search, a paginated search result can have an ID
| so I can paginate between the data without a new data
| messing up my pagination.
|
| But for normal entities, simple pagination is (mostly) more
| than enough.
|
| The solution you are describing is overkill and almost no
| benefit at all.
| akvadrako wrote:
| That's just like paging by day or whatever.
| knighthack wrote:
| A good checklist of things to mind when designing APIs. Sometimes
| you forget the good things.
| hardwaresofton wrote:
| A few things I see rarely discussed that I often do:
|
| - Always use a response envelope
|
| - HTTP status is not the same as your application status. I
| always include a "status" field in the response envelope (OP
| recommends standardizing errors but I think standardize all of
| it)
|
| - Always have an unique error code for every error your API can
| return (they should also be URL safe ("some-thing-went-wrong"),
| basically an enum of these should exist somewhere, the more
| specific the better.
|
| - Offer OpenAPI whenever you can.
|
| - There is no such thing as a publicly exposed "private" API. If
| it can be hit (and is not protected), it eventually will be.
|
| - Do blackbox testing of your API, E2E tests are the most
| important kind of test you could have.
|
| - (controversial) build in special test/debug endpoints -- these
| help with blackbox testing.
| scurvy_steve wrote:
| - Always use a response envelope
|
| I would mostly agree except in 1 case, streaming data is easier
| without an envelope. Making some array inside an envelope
| stream is usually more code and messier than just getting some
| metadata out of header. So if you have something like data
| integration endpoints and you expect someone could pull many
| megs of records, consider no envelope.
| metadat wrote:
| Are you able to elaborate on what is a response envelope?
|
| Edit:
|
| Nevermind, see [0]. It's the simple and intuitive concept of
| encasing the payload in an consistently structured object which
| includes metadata, e.g. {"Error": null, Data:
| ...}
|
| [0] https://stackoverflow.com/questions/9989135/when-in-my-
| rest-...
|
| Makes intuitive sense, as it's then easier to develop a nice,
| generic REST client.
| hardwaresofton wrote:
| Here's an example straight from code I've rewritten at least
| 5 times because I'm allergic to saving myself time:
| export enum ErrorCode { InvalidEntity = 1,
| NotSupported = 2, UnexpectedServerError = 3,
| InvalidRequest = 4, Validation = 5, //
| ... } export enum ResponseStatus {
| Success = "success", Error = "error", }
| export class ResponseEnvelope<T> { public readonly
| status: ResponseStatus = ResponseStatus.Success;
| public readonly data: T | Pruned<T> | null = null;
| public readonly error?: { code: ErrorCode,
| message: string, details: object | string,
| status: ResponseStatus, };
| constructor(opts?: Partial<ResponseEnvelope<T>>) {
| if (opts) { if (opts.status) { this.status =
| opts.status; } if (opts.data) { this.data =
| opts.data; } if (opts.error) { this.error =
| opts.error; } } if (!this.data)
| { return; } // Prune if the thing is
| prunable this.data = prune(this.data);
| }
|
| }
|
| Hopefully this isn't too hard to grok, it's Typescript.
|
| BTW, If you're cringing at the number scheme for the
| ErrorCode enum, don't worry I am too. This is why I prefer to
| use strings rather than numbers, and it basically just ends
| up being stuff like "invalid-entity", etc.
| JackFr wrote:
| I completely disagree.
|
| I find envelopes to be unnecessary cruft that just junk up
| otherwise clear code. And I think packing metadata into an
| envelope along with the actual data keeps people from
| thinking clearly about their own APIs, mostly with respect
| to ambiguity surrounding the word "error". Validation
| errors are errors and network failures are errors, but
| they're very different animals and should never be
| conflated.
|
| I don't want to check for a status code in metadata ever,
| when an HTTP status is provided. I don't want to read a
| time stamp if that time stamp is simply a property of the
| request. However if the time stamp is a property of the
| data, then I care - but then it shouldn't be part of the
| metadata.
| hardwaresofton wrote:
| Well I imagine this is why sages like uncle bob and sam
| newman will always be needed.
|
| > Validation errors are errors and network failures are
| errors, but they're very different animals and should
| never be conflated.
|
| > I don't want to check for a status code in metadata
| ever, when an HTTP status is provided.
|
| These seem like conflicting views. If the HTTP status is
| all you check you _must_ be conflating application
| /operation status and network status.
|
| > I don't want to read a time stamp if that time stamp is
| simply a property of the request. However if the time
| stamp is a property of the data, then I care - but then
| it shouldn't be part of the metadata.
|
| Sure -- I'd like to say that I think the disagreement
| here boils down to this question: should metadata be
| represented in data responses.
|
| Correct me if I'm reading you incorrectly, but it seems
| like you're arguing that metadata should not be
| represented in the response, only data. I'd argue that
| the in-practice and academic standards dictate the
| opposite, with standards like hypermedia, json schema,
| HAL, JSON:API, OpenAPI, etc.
|
| If it's just a question about the degree of metadata that
| should be present then that's one thing, but it sounds
| like you're against it in general. Once you have any
| metadata you must have an envelope of some sort, by
| definition (whether it's a good or bad one), as far as I
| can see.
| [deleted]
| kortex wrote:
| I think I'm with Jack on this one, at least when it comes
| to REST interfaces. When I query GET /bar/123, I want an
| object with a Bar protocol/content-type. I don't want a
| Envelope[Bar]. What if it's any data type other than
| json-isomorphic, e.g. image or video? Is /videos/123.mp4
| going to return a json object like
| {"data": <base64_encoded_video>, "status": "whatever"}
|
| Of course not!
|
| You already have an envelope, it's the HTTP protocol. The
| real trick is generically mapping data structures to HTTP
| responses with headers. In fact HTTP-response-shaped-
| objects make halfway decent general purpose envelopes in
| the business logic itself. They are basically Result[T,
| E] but with even more metadata available.
| bcrosby95 wrote:
| We wrap the response body because it's important to give
| clients of an API endpoint, something that they will use
| to query/modify data, a clear indicator of the
| application response status. We do this in the response
| body because http status codes aren't good enough, and
| people tend to miss headers. It's hard to miss it when
| it's part of the body.
|
| And no, we don't do that for static content for a simple
| reason: static content isn't served from our API servers.
| [deleted]
| [deleted]
| [deleted]
| tommiegannert wrote:
| > 6. Accept API key authentication ... using a custom HTTP header
| (such as Api-Key).
|
| Wouldn't a bearer token [1] make more sense? Defined for use by
| OAuth2, but I don't see why it couldn't be the general mechanism
| for... bearer tokens.
|
| > 11. Return created resources upon POST
|
| Especially important if your database+caching layers use eventual
| consistency.
|
| [1] https://datatracker.ietf.org/doc/html/rfc6750
| jamietanna wrote:
| Authentication and Authorization are two subtly different
| things. In this case, you may want an API key (Authentication)
| to be required to ensure things like rate limiting is enforced,
| but then want proof that the call is operating on a user, or is
| a machine-to-machine interaction which OAuth2 Bearer tokens
| work nicely for (Authorization)
| bmn__ wrote:
| The author has a poor understanding of HTTP and adjacent
| standards. I find it is so insufficient that he should not
| dispense advice yet, he must learn much more, especially about
| the essential yet completely unmentioned parts of a Web system:
| media types, hyperlinks, Link relations, cacheability. Critique:
|
| 2. ISO 8601 is a shit show of a standard. Last time I surveyed,
| there was not a single compliant implementation in the world.
| Recommend "Date and Time on the Internet: Timestamps"
| <http://rfc-editor.org/rfc/rfc3339> instead. This standard is
| public/not encumbered by stupid copy fees and is restricted to a
| much more reasonable profile that can actually be implemented
| fully and in an interoperable fashion.
|
| 4. Use `OPTIONS _` instead. <http://rfc-
| editor.org/rfc/rfc7231#section-4.3.7>
|
| 5. This goes against the design of the Web. Do not version URIs.
| If the representation changes and is not backward compatible,
| change the media type instead, e.g. by adding a profile
| <http://rfc-editor.org/rfc/rfc6906>. You can serve multiple
| representations on the same URI and vary on the request headers
| (e.g. Accept-_).
|
| 6. HTTP already contains a customisable mechanism. Use the
| standard header <http://rfc-editor.org/rfc/rfc7235#section-4.2>,
| not the custom header `Api-Key` which is not interoperable.
|
| 7. "Don't use too many [HTTP status codes]": why? This opinion is
| not backed up with an explanation. The correct advice is: use as
| many status codes as arise from the requirements. `422
| Unprocessable Entity` is useful in nearly every Web system.
|
| 8. What does "reasonable" mean? This lacks an explanation.
|
| 10. Use application/problem+json <http://rfc-
| editor.org/rfc/rfc7807> instead.
|
| 11. "It's a good idea to return the created resource after
| creating it with a POST request": why? This opinion is not backed
| up with an explanation. If you follow this advice, the user agent
| cannot programmatically distinguish between a representation
| reporting on the requested action's status, and the
| representation of the newly created resource itself. Use the
| Content-Location header <http://rfc-
| editor.org/rfc/rfc7231#section-3.1.4.2> to make that possible,
| use the Prefer header <http://rfc-editor.org/rfc/rfc7240> to give
| the user agent some control over which representation to respond
| with.
|
| 12. "Prefer PATCH over PUT": disagree, ideally you offer both
| since there is a trade-off involved here. The downsides of PATCH
| are not mentioned: the method is not (required to be) idempotent
| meaning it becomes moderately tricky to keep track of state, and
| the client is required to implement some diff operation according
| to the semantics of the accepted media type in the Accept-Patch
| header which can be difficult to get right.
|
| 13. Missed opportunity to advertise the OPTIONS method and
| related Accept-Post <https://datatracker.ietf.org/doc/html/draft-
| wilde-accept-pos...> / Accept-Patch <http://rfc-
| editor.org/rfc/rfc5789#section-3.1> headers. A representation of
| a specific media type can self describe with the "type" Link
| relation <http://rfc-editor.org/rfc/rfc6903.html#section-6>.
|
| 14. Don't mix data with metadata. Use the Range header
| <http://rfc-editor.org/rfc/rfc7233> and next/prev Link relations
| <https://webconcepts.info/concepts/link-relation/next> instead.
|
| 15. Instead of complicating both server and client, the better
| idea is to simply link to related resources. We are not in the
| 1990s any more. A well written Web server offers HTTP persistent
| connections, HTTP pipelining, all of which make round-trips cheap
| or in the case of HTTP/2 server push, even unnecessary. Benchmark
| this.
|
| 1. + 9. betrays a weird obsession with naming. The advice is _not
| wrong_ , but it shifts attention away from the genuinely useful
| areas that benefit much more from a careful design: the media
| types, the hyperlinks between and other hypermedia mechanisms for
| traversing resources (forms, URI templates). If an inexperienced
| programmer follows the advice from the article, he will the idea
| to spend lots of time mapping out resources in the identifier
| space and methods and possible responses for documentation
| purposes. This is both a waste of time because the single entry
| point is enough and the rest of the resources can be reached by
| querying the server via OPTIONS and traversing hyperlinks etc.,
| and dangerously brittle because of strong coupling between the
| server and the client.
| [deleted]
| AtNightWeCode wrote:
| I agree with most things.
|
| I really hate when APIs use different api-key headers depending
| on the role of the consumer.
|
| It is very annoying when you get dates that are not in the ISO
| format. There are reasons to not use UTC everywhere. One should
| make that decision.
|
| The reason why many APIs use POST instead of DELETE is that POST
| is said to be more secure.
|
| Many APIs that I use do not have neither of PATCH, PUT or DELETE.
| An order for instance will have an order status resource that one
| just keeps adding status entities to. In general, well-designed
| systems minimize the need for changing data.
| ChrisMarshallNY wrote:
| It's a good posting. I do many of these things, but not all.
|
| I've been designing SDKs for a long time; APIs, for a somewhat
| shorter time.
|
| I don't like "pure" RESTful APIs, because I feel as if they are
| "human-hostile." I tend to follow the patterns set by companies
| like Google, where the stimulus is a fairly straightforward URI
| (with parameters, as opposed to a block of XML or JSON --if
| possible, as I know that we often still need to send big data as
| transaction data, which can be a pain, depending on the method),
| and the response is a block of structured data. That also makes
| it a lot easier for API clients to build requests, and allows the
| server to "genericize" the processing of requests.
|
| _> 10. Use standardized error responses_
|
| Is something I do, along with a text header payload, describing
| internal information about the error. It is my experience that
| this text payload is never transferred, and I am forced to send a
| text response via the body, if I want to know internal
| information. That can be a pain, as it often breaks the
| transaction, so I have to play with the server and client,
| frequently, using a cloned server instance.
|
| I should note that my forte is not backend development, so the
| chances are good that I am not familiar with all the tools at my
| disposal.
| thatwasunusual wrote:
| > I tend to follow the patterns [...]
|
| Can you elaborate on this? Give examples?
| ChrisMarshallNY wrote:
| I don't feel like doing that in a comment, but feel free to
| check out some of my work. The BAOBAB server[0] is one of my
| more recent examples. I'm using a modified version as the
| backend for the app I'm developing, now. It works great.
|
| I should note that the BASALT layer uses "plugins," that can
| be extended to include "pure" REST APIs. I just haven't found
| these practical, for my purposes.
|
| [0] https://riftvalleysoftware.com/work/open-source-
| projects/#ba...
| kumarvvr wrote:
| Sometimes, I feel that we ought to have a simple protocol, on top
| of HTTP, to simply do remote procedure calls and throw out all
| this HTTP verbs crap. Every request is a http POST, with or
| without any body and the data transfer is in binary. So that
| objects can be passed back and forth between client and server.
|
| Sure, there is gRPC, but it requires another API specification
| (the proto files).
|
| There I said it. HTTP Verbs constrained REST APIS are the worst
| thing ever. I hate them.
|
| They introduce un-necessary complexity, un-necessary granularity
| and they almost always stray away from the "REST principles". To
| hell with "Hypermedia" stuff.
|
| I find it such a joy to program in server rendered pages. No
| cognitive overhead of thinking in "REST".
|
| But, of course, all this is only where the client and server are
| developed by the same person / company.
|
| For publishing data and creating API for third party use, we have
| no serious, better alternative to REST.
| bluefirebrand wrote:
| My current company has settled on "We use GET to retrieve data
| from the server and POST to send data to the server, nothing
| else" because it was causing quite a lot of bikeshedding style
| discussions where people were fussing over "Should this be a
| post, put or patch"?
|
| It all came to a head when someone wrote an endpoint using a
| PATCH verb that some people were adamant should have been a
| PUT.
|
| It was among the most silly nonsense I have ever been a part of
| and these discussions have thankfully gone to zero since we
| decided on only GET and POST
| tobyjsullivan wrote:
| As someone who has spent a decade working with APIs, I 100%
| agree. The use cases that are a good fit for "RESTful" APIs
| pale in comparison to those that would benefit from RPC.
|
| What is the point of having your client translate an action to
| some operation on a document (read or write), only to then have
| your server try to infer what action was intended by said
| document operation.
|
| It pains me that this article doesn't mention any of the trade
| offs of each suggestion (POST vs PUT vs PATCH and expandable
| objects, especially) or of using REST APIs generally.
| laurent92 wrote:
| +1, each time I return 404 for an object which is not found
| in the DB, the customer gets a red error message in their UI
| as if something failed more severely than an object being
| unavailable, and the metrics believe something is
| unavailable.
|
| I bit my fingers every time I have mapped HTTP verbs and
| neither return codes, to REST verbs and codes.
|
| Also, error codes at the API level often need a remapping
| when used in user context, for example if the OAuth token
| expires, we don't say it the same way for an action of the
| user (then it's mandatory) than when displaying data
| passively (in which case it shouldn't be too red because the
| user may not care).
| lkrubner wrote:
| I feel like the whole of the 1990s was devoted to this. How to
| serialize an object and then what network protocol should be
| used? But increasingly over time, between 2000 to 2005,
| developers found it was easier to simply tunnel over port
| 80/443. In 2006 Pete Lacey wrote a satire about SOAP, which is
| funny but also accurate, and look at how late people are to
| discover that you can tunnel over HTTP:
|
| http://harmful.cat-v.org/software/xml/soap/simple
|
| I was puzzled, at the time, why the industry was cluttering
| HTTP in this way. Why not establish a clean protocol for this?
|
| But people kept getting distracted by something that seemed
| like maybe it would solve the problem.
|
| Dave Winer used to be a very big deal, having created crucial
| technologies for Apple back in the 1980s and 1990s, and he was
| initially horrified by JSON. This post is somewhat infamous:
|
| http://scripting.com/2006/12/20.html#godBlessTheReinventers
|
| "... and damn, IT'S NOT EVEN XML!"
|
| He was very angry that anyone would try to introduce a new
| serialization language, other than XML.
|
| My point is, the need for a clear a clean RPG protocol, but the
| industry has failed, again and again, to figure out how to do
| this. Over and over again, when the industry gets serious about
| it, they come up with something too complex and too burdensome.
|
| Partly, the goal was often too ambitious. In particular, the
| idea of having a universal process for serializing an object,
| and then deserializing it in any language, so you can serialize
| an object in C# and then deserialize it in Java and the whole
| process is invisible to you because it happens automatically --
| this turned out to be beyond the ability of the tech industry,
| partly because the major tech players didn't want to cooperate,
| but also because it is a very difficult problem.
| moltenguardian wrote:
| gRPC is encoding agnostic, and requires _no_ Protobuf at all.
|
| See: https://grpc.io/blog/grpc-with-json/
| kortex wrote:
| In practice though, the tooling is cumbersome enough that you
| can't readily sub in some other protocol besides protobuf,
| json, and allegedly flatbuf. I've had little success finding
| ways to e.g. use msgpack as the serde. Maybe it's out there
| but I haven't found it.
| T-J-L wrote:
| Have you seen: https://github.com/twitchtv/twirp
| cies wrote:
| While I totally agree with the overkill that REST can be, I
| really do NOT agree with your statement:
|
| > but it requires another API specification
|
| This implies API-specs are part of the problem; and I think
| they are not.
|
| Specs that have generators for client-libs (and sometimes even
| sever-stubs) are verrrrry important. They allow us to get some
| form of type-safety over the API barrier which greatly reduces
| bugs.
|
| One big reason for me to go with REST is OpenAPIv3: it allows
| me to completely spec my API and generate clients-libs for sooo
| many languages, and server-stub for sooo many BE frameworks.
| This, to me, may weight up to the downsides of REST.
|
| GraphQL is also picking up steam and has these generators.
|
| JSON-RPC (while great in terms of less-overkill-than-REST) does
| not have so much of this.
| bruhboribhe wrote:
| We do, it's called JSON-RPC.
| kumarvvr wrote:
| Whoa, completely missed that boat.
|
| Is it in active use? Wikipedia page says last spec update was
| in 2010. No other details online. I could not find any
| specific implementations.
| c-cube wrote:
| It's used at least as the foundation of the LSP protocol.
| So basically, it's deployed daily in millions of editors.
| bruhboribhe wrote:
| Active use is really irrelevant if you only plan on using
| it inside a company, because you can implement a client and
| server within 30 mins to an hour, no external tools needed.
| The spec is clear and readable. It's excellent. We use it
| with a TypeScript codebase and just share the interfaces
| for the services in a monorepo. The spec is so simple it
| doesn't really need an update.
|
| If you want a more advanced implementation with type
| inference for Typescript you can use this:
| https://github.com/shekohex/jsonrpc-ts
|
| I'd still recommend implementing your own, my own
| implementation is based off the one above.
|
| The only caveat compared to GRPC is you lose out on field
| validation due to it not being protobuf/and the obvious
| json decode overhead
|
| Edit: if you want to know some current users of it, I
| believe the Ethereum protocol uses it heavily
| er4hn wrote:
| Arista Networking uses it in their eAPI protocol. It
| let's you have machine parsable outputs and avoid ye olde
| days of screen scraping network device outputs to view
| interface status and other details.
|
| I believe most users make use of it via an open source
| json-rpc python lib. You can find a few examples online
| if you'd like to know more.
| incrudible wrote:
| Arguably, the vast majority of REST APIs are "JSON RPC" with
| a more convoluted calling convention based on URLs, HTTP
| verbs and status codes.
| asadawadia wrote:
| >Sometimes, I feel that we ought to have a simple protocol, on
| top of HTTP, to simply do remote procedure calls and throw out
| all this HTTP verbs crap. Every request is a http POST, with or
| without any body and the data transfer is in binary. So that
| objects can be passed back and forth between client and server.
|
| https://github.com/asad-awadia/indie-rpc
| lewisjoe wrote:
| Agreed 100%. Slapping a REST api over a software is like
| reducing that software to a set of resources and attribute
| updates over those resources. And that never feels like the
| right way to talk with a software. That could be convenient for
| the majority of crud apps out there, but not everything we
| build is a crud system. For example how would you design
| operations on a cloud word processor as REST apis?
|
| A better perspective would be, most softwares can be viewed as
| a set of domain specific objects and the set of operations
| (verbs) that can happen to those objects. These operations may
| not be a single attribute update, but a more complex dynamic
| set of updates over a variety of business objects. If you try
| to model this with a REST api, it either quickly becomes chatty
| or you end up compromising on REST principles.
|
| GraphQL seems to make much more sense than REST, IMO.
| FearlessNebula wrote:
| I'm not sure what issue the verbs are creating, can someone
| help me get through my thick skull what this persons issue with
| them is? I don't see how they add much complexity, just check
| the API docs and see what verb you need to use to perform a
| certain action.
| mobjack wrote:
| They are the wrong layer of abstraction outside of simple
| CRUD apps.
|
| If you have to check the API docs anyways, I rather define
| custom domain specific verbs than debate whether something
| should be a PUT or a PATCH.
| spikej wrote:
| I don't think I've ever come across any third party actually
| implementing HATEOAS (https://en.wikipedia.org/wiki/HATEOAS)
| zja wrote:
| That's because no one knows what it is.
| bluefirebrand wrote:
| I had a new hire on my team criticize the API we built for
| our product because we don't use put or patch, and we don't
| allow a GET and POST to share the same path. He said "it's
| not very RESTful"
|
| I pointed him at HATEOAS and suggested if he wasn't familiar
| with it, he probably hasn't ever seen a truly RESTful API.
|
| I don't think I convinced him that our approach is good (I'm
| not sure I am convinced either, but it works well enough for
| our purposes)
|
| I do think I convinced him that "it doesn't correctly conform
| to a standard" isn't necessarily a useful critique , though.
| So that's a win.
| JimDabell wrote:
| Chrome, Safari, Firefox, Edge, and Opera are all third-party
| clients for the HATEOAS-based API known as the World-Wide
| Web.
| darkstar999 wrote:
| What?
| brigandish wrote:
| The classic book on the subject is RESTful Web APIs[1],
| and it spends a while explaining HATEOAS by using the
| example of the web as we've come to expect it as the
| exemplar REST API using HATEOAS. I also have this
| essay[2] on HATEOAS in my open tabs, and it uses the
| example of a web browser fetching a web page.
|
| [1] https://www.oreilly.com/library/view/restful-web-
| apis/978144...
|
| [2] https://htmx.org/essays/hateoas/
| incrudible wrote:
| This should come with a big warning for people looking to
| do real work. This is not what most REST APIs are like in
| practice, nor what they should be. The vast majority of
| REST APIs are RPC-like, because that's the pragmatic way
| to deal with the problem 99% of the time. The "REST"
| branding is just for buzzword compliance.
| incrudible wrote:
| This answer is correct, but lacks context. REST wasn't
| conceived with APIs in mind. In fact, it's an awful fit
| for APIs, as many of the other comments point out.
| Rather, REST today is a buzzword that took on a life of
| its own, bearing only superficial resemblance to the
| original ideas.
|
| HATEOAS is a generalization of how something like a
| website would let a client navigate resources (through
| hyperlinks). It requires an _intelligent_ agent (the
| user) to make sense. Without HATEOAS, according to Roy
| Fielding, it 's not _real_ REST. Some poor misguided API
| designers thought this meant they should add URL
| indirections to their JSON responses, making everything
| more painful to use for those _unintelligent_ clients
| (the code that is consuming the API). Don 't do this.
|
| If you must do REST at all - which _should_ be up for
| debate - you should keep it simple and pragmatic. Your
| users will not applaud you for exhausting HTTP verbs and
| status codes. The designers of HTTP did not think of your
| API. You will likely end up adding extra information in
| the response body, which means I end up with two levels
| (status code _and_ response) of matching your response to
| whatever I need to do.
|
| If something doesn't quite fit and it looks ugly or out-
| of-place, that's normal, because REST _wasn 't conceived
| with APIs in mind_. Don't go down the rabbit hole of
| attempting to do "real REST". There is no pot of gold
| waiting for you, just pointless debates and annoyed
| users.
| bluefirebrand wrote:
| Absolutely agreed on all points.
|
| The best APIs I've ever used or built have been, at best,
| REST-ish.
|
| And generally the parts where they deviate from REST make
| them more usable, not less.
| progval wrote:
| The web (HTTP + HTML + JS) intentionally fits the
| definition of a REST API. https://oleb.net/2018/rest/
|
| In particular:
|
| > The central idea behind HATEOAS is that RESTful servers
| and clients shouldn't rely on a hardcoded interface (that
| they agreed upon through a separate channel). Instead,
| the server is supposed to send the set of URIs
| representing possible state transitions with each
| response, from which the client can select the one it
| wants to transition to. This is exactly how web browsers
| work
| askme2 wrote:
| At OSIsoft, they implement HATEOAS religiously.
| https://docs.osisoft.com/bundle/pi-web-api-
| reference/page/he...
| lowboy wrote:
| HATEOAS have always sounded like a delicious part of a
| healthy breakfast
| bluefirebrand wrote:
| My brain drops the A, so I always read it HATEOS which
| makes me thinks it is a joke Linux distro of some kind.
| pmontra wrote:
| I used some API recently that returns URLs in the response
| body. It's really useful because they maintain those URLs and
| we don't have to rewrite our URL building code whenever the
| server side rules change. Actually we don't even have to
| write that code. It saves time, bugs, money.
|
| I don't remember which API was that, I'll update the comment
| if I do.
| berkes wrote:
| Better yet, those URLs communicate _what_ you may do.
|
| Instead of building the logic to determine if, say, a
| Payment can be cancelled, based on its attributes, you
| simply check 'is the cancel link there'.
|
| I find this a critical feature. Because between the
| backend, and various mobile clients, react, some admin, and
| several versions thereof, clients will implement such
| businesslogic wrong. Much better to make the backend
| responsible for communicating abilities. Because that
| backend has to do this anyway already.
| crdrost wrote:
| So the problem with "Data transfer is in binary" is that it
| really requires both the source and the recipients to be
| running the same executable, otherwise you run into some really
| weird problems. If you just embrace parsing you of course don't
| have those problems, but that's what you are saying not to
| do... Another great idea is for a binary blob to begin with the
| program necessary to interrogate it and get your values out,
| this has existed on CDs and DVDs and floppies and tape forever
| but the problem is that those media have a separate chain of
| trust, the internet does not, so webassembly (plus, say, a
| distributed hash table) really has a chance to shine here, as
| the language which allows the web to do this quickly and
| safely. But it hasn't been mature.
|
| The basic reason you need binary identicality is the problem
| that a parser gives you an error state, by foregoing a parser
| you lose the ability to detect errors. And like you think you
| have the ability to detect those errors because you both depend
| on a shared library or something, and then you get hit by it
| anyway because you both depend on different versions of that
| shared library to interpret the thing. So you implement a
| version string or something, and that turns out to not play
| well with rollbacks, so the first time you roll back everything
| breaks... You finally solve this problem, then someone finds a
| way to route a Foo object to the Bar service via the Baz
| service, which (because Baz doesn't parse it) downgrades the
| version number but does not change the rest of the blob, due to
| library mismatches... Turns out when they do this they can get
| RCE in Bar service. There's just a lot of side cases. If you're
| not a fan of Whack-a-Mole it becomes easier to bundle all your
| services into one binary plus a flag, "I should operate as a
| Bar service," to solve these problems once and for all.
| azornathogron wrote:
| > So the problem with "Data transfer is in binary" is that it
| really requires both the source and the recipients to be
| running the same executable, otherwise you run into some
| really weird problems.
|
| I think you're misinterpreting "data transfer is in binary"
| with something like "a raw memory dump of an object in your
| program, without any serialisation or parsing step".
| loup-vaillant wrote:
| For a second there I thought this was about _non-web_ APIs:
| https://caseymuratori.com/blog_0024
| ceeker wrote:
| Can someone share how they handle versioning in their API when it
| comes to data model changes? For example `POST /users` now takes
| a required field `avatar_url` but it was not part of `v1`.
|
| Since this field is validated in the DB, merely having `v1` `v2`
| distinction at the API layer is not sufficient. So I was thinking
| we will have to either 1) disable DB validations and rely on app
| validations or 2) run two separate systems (e.g., one DB per
| version) and let people 'upgrade' to the new version (once you
| upgrade you cannot go back).
|
| Even though people refer to Stripe's API versioning blog, I don't
| recall any mention of actual data model changes and how it is
| actually managed
| longnguyen wrote:
| We're using event sourcing so the "projection" (db snapshot)
| have 2 different tables for v1 and v2. Think users_v1,
| users_v2.
|
| Obviously there will be always challenges with eventually
| consistency but that is another topic altogether.
| theteapot wrote:
| Have a default value for v1. Don't maintain two DBs just for
| this.
| ComputerGuru wrote:
| I upvoted because I'm curious to hear what others are doing. We
| typically make sure to only make such breaking changes where
| either the now-required value or a sane filler value could be
| used. If it's the same API for the same purpose, it's usually
| not a stretch to assume the values for a new field are derived
| from some combination of an old field or else are primitive
| components of an old field such that they can be deduced and
| stubbed in your transition layer (or calculated/looked-
| up/whatever one-by-one as part of a bulk migration script
| during the transition). If your v2 is so drastic of a breaking
| upgrade that it bears no relationship to v1, I imagine your SOL
| and probably should have thought out your v1 or your v1-to-v2
| story better, if only for the sake of the poor devs using your
| API (and you probably need separate tables at that point).
|
| For other fields like your example of `avatar_url` I would use
| a placeholder avatar for all legacy users (the grey anonymous
| snowman profile comes to mind).
| ceeker wrote:
| Thanks. This is a fair point. I made up the example only to
| illustrate the idea. Since Stripe is considered some sort of
| benchmark here I was curious to see how they tackle all the
| learnings they will have over time...I feel it is very hard
| to think through all the future cases especially when you are
| just about starting out with your product.
|
| For example, in financial services and insurance, regs change
| and what data we need to collect change and sometimes their
| dependency will change. I am curious what's companies that
| have grown substantially had to do to their APIs.
| christophilus wrote:
| I think Stripe was originally built on Rails (can't find
| anything to confirm that at the moment). But my guess is
| they enforce things at the app layer, since Rails didn't
| really provide a good way to enforce things at the DB layer
| originally. They support very old API versions by
| transforming requests backwards and forward through a list
| of API version transforms, which also suggests to me that
| this sort of thing is enforced at the app layer rather than
| the DB.
| ComputerGuru wrote:
| No worries, I understood it was a throwaway example that
| shouldn't be looked at too closely. You just have to
| remember that your DB isn't a model of what you want to
| require from your customers but rather a model of what you
| actually necessarily have and don't have. A field like the
| ones you're talking about shouldn't be marked non-nullable
| in the database if there's a chance you actually don't have
| that data (and when you are suddenly required to collect
| something you didn't have before, you're not going to have
| it).
|
| Coming at this from a strongly-typed background, you
| acknowledge the fact that despite new regulations requiring
| a scan of the user's birth certificate in order to get an
| API token, _that field can 't be marked as non-null if you
| don't in fact have all those birth certificates_. You are
| then forced to handle both the null and not-null cases when
| retrieving the value from the database.
|
| So your API v2 can absolutely (in its MVC or whatever
| model) have that field marked as non-null but since your
| API v1 will still be proxying code to the same database,
| your db model would have that field marked as nullable
| (until the day when you have collected that field for all
| your customers).
|
| If a downstream operation is contingent on the field being
| non-null, you are forced to grapple with the reality that
| you don't have said field for all your users (because of
| APIv1 users) and so you need to throw some sort of 400 Bad
| Request or similar error because (due to regulations) this
| operation is no longer allowed past some sunset date for
| users that haven't complied with regulation XYZ. In this
| case, it's a _benefit_ that your db model has the field
| marked as null because it forces you to handle the cases
| where you don 't have that field.
|
| I guess what I'm saying is the db model isn't what you wish
| your data were like but rather what your data actually is,
| whether you like it or not.
| martypitt wrote:
| Hey
|
| We're working in this space at the moment, (eliminating the
| pain from breaking changes in APIs) and looking to get
| feedback on what we're building.
|
| We're all from banking backgrounds, so understand the reg
| headaches you're talking about.
|
| Can we chat?
| bityard wrote:
| Those are possible, but ugly solutions. Two cleaner ones are
| either depracate and remove the v1 api altogether, or when
| inserting a record to the database from the v1 api, use a
| default dummy value for avatar_url.
| blowski wrote:
| Yes, and definitely favour the deprecation. Treat web APIs
| like any interface - have minor and major versions,
| deprecating then dropping old versions.
| RamblingCTO wrote:
| Although I agree with the comments above, adding a field is
| also a breaking change, even with a default value.
| Especially prone to this is any openApi client (speaking
| from experience ...). Maybe filtering it out would be an
| actual solution without breaking anything. An API update
| shouldn't need to update my implementation because it's not
| working anymore, in that case it's a breaking change and a
| major version bump.
| kortex wrote:
| I'm not saying you _should_ do it this way, this is just how
| our startup (still very much in the "move fast and discover
| product fit" stage) does it. We have separate API models
| (pydantic and fastapi) and DB models (sqlalchemy). Basically
| everything not in the original db schema ends up nullable when
| we first add a field. The API model handles validation.
|
| Then if we absolutely do need a field non-null in the db, we
| run a backfill with either derived or dummy data. Then we can
| make the column non-null.
|
| We use alembic to manage migration versions.
|
| But we aren't even out of v0 endpoint and our stack is small
| enough that we have a lot of wiggle room. No idea how scalable
| this approach is.
|
| The downside is maintaining separate api and db models, but the
| upside is decoupling things that really aren't the same. We
| tried an ORM which has a single model for both (Ormar) and it
| just wasn't mature, but also explicit conversions from wire
| format to db format are nice.
| mtoddsmith wrote:
| That's what salesforce does. In our app we're on version 47.0
| of their API
|
| https://test.salesforce.com/services/Soap/c/47.0
|
| And in the latest version of the API docs they have details
| regarding old versions. Example:
|
| https://developer.salesforce.com/docs/atlas.en-
| us.api.meta/a...
|
| Type reference Properties Create, Filter, Group, Nillable,
| Sort Description The ID of the parent object record that
| relates to this action plan.
|
| For API version 48 and later, supported parent objects are
| Account, AssetsAndLiabilities, BusinessMilestone, Campaign,
| Card, Case, Claim, Contact, Contract, Financial Account,
| Financial Goal, Financial Holding, InsurancePolicy,
| InsurancePolicyCoverage, Lead, Opportunity, PersonLifeEvent,
| ResidentialLoanApplication, and Visit as well as custom
| objects with activities enabled.
|
| For API version 47 and later, supported parent objects are
| Account, BusinessMilestone, Campaign, Case, Claim, Contact,
| Contract, InsurancePolicy, InsurancePolicyCoverage, Lead,
| Opportunity, PersonLifeEvent, and Visit as well as custom
| objects with activities enabled.
|
| For API version 46 and later, supported parent objects are
| Account, Campaign, Case, Contact, Contract, Lead, and
| Opportunity as well as custom objects with activities
| enabled.
|
| For API version 45 and earlier: the only supported parent
| object is Account.
| hummingn3rd wrote:
| I really like this way of versioning
| https://medium.com/@XenoSnowFox/youre-thinking-about-api-ver...
|
| It uses Accept and Content-Type to version resources:
| application/vnd.company.article-v1+json
| BossingAround wrote:
| Interesting! Personally, if I had to go to the lengths of the
| article, I might as well use a schema registry like Avro.
| kortex wrote:
| That's...really clever, but at the same time, I feel like
| there's a lot of assumptions baked into how Content-Types are
| used, and making your own content-type for each data model
| when it's all just application/json seems...wrong to me on an
| intuitive level, but I can't quite annunciate why.
|
| I only half agree with the sentiment that /api/v1 violates
| REST patterns. I don't think there's any guarantee that
| /api/v1/bars/123 _can 't_ be the same object as
| /api/v2/bars/123.
| sandreas wrote:
| While I agree with the most aspects of this very good blog post,
| there is a minor detail that I would like to note:
|
| While pagination is important, there is another possibility of
| pure size - it is using cursors, like mentioned in the JSONAPI
| specification[1] (containing many of the hints in the topics'
| post) and in this blog post[1]
|
| [1] https://jsonapi.org/format/#fetching-pagination
|
| [2] https://dev.to/jackmarchant/offset-and-cursor-pagination-
| exp...
| BulgarianIdiot wrote:
| Mostly common-sense things, but I can't wait for the community to
| stop trying to use PUT, PATCH, DELETE and the like. There's a
| reason that in 2022 web forms only support GET and POST (and
| implicitly, HEAD).
| jahewson wrote:
| What is the reason?
| runarberg wrote:
| I don't know what OP thinks is the reason, but the actual
| reason is CORS.
|
| Web Devs (such as me) have asked for e.g. DELETE as an
| acceptable formmethod (e.g.
| https://github.com/whatwg/html/issues/3577) however WHATWG
| always pushes back citing security concerns such as CORS.
|
| I suspect this is not what OP had in mind since it is trivial
| to send a DELETE request with a simple JavaScript:
| <button onclick="fetch('/api/resource/42', { method: 'DELETE'
| })"> Delete </button>
| egberts1 wrote:
| Right.
|
| If CORS can be weakened in any simple way with that HTRP-
| DELETE method, then your database could simply disappeared
| via HTTP-DELETE method.
|
| Besides, webmasters' HTTP DELETE method is a different
| domain scoping issue than the web developers'
| HTML/JavaScript FORM deleteThis row-entry approach.
|
| I marvel at designers trying to flatten the scoping/nesting
| of abstractions without factoring apart the disparate error
| and protocol handling.
| jimbob45 wrote:
| Are you saying dump DELETE because you should instead logically
| delete it with an IsDeleted column passed in via POST?
| djbusby wrote:
| Or POST to '/resource/id/delete'? That's my least favourite
| pattern
| [deleted]
| boredtofears wrote:
| Huh? There's full browser support for all of those verbs.
|
| What is the argument for not supporting them?
| BulgarianIdiot wrote:
| The HTTP request APIs pass through any method name you write.
|
| The HTML forms only support GET and POST. Try it.
| inopinatus wrote:
| There is no mention of HTML in these recommendations.
|
| The recommendation is perfectly good for any API using HTTP
| as the substrate. The wisdom of using HTTP as a protocol
| substrate is questionable, but having made that decision
| the verbs it supplies work perfectly well.
|
| Incidentally, the HTML living standard supports three form
| methods, not two: get, post, and dialog. Which rather
| reinforces the point that HTML != HTTP.
| remoquete wrote:
| Extra points: document the API / have an API technical writer in
| the team.
|
| Part of a good API design is documenting it. If you use an API
| specification format, such as OpenAPI, design and documentation
| overlap nicely.
| BossingAround wrote:
| Ah yes, the classic:
|
| _POST /document/:id?params_
|
| _Creates a document with parameters_
| kuon wrote:
| ISO 8601 is bad, use RFC 3339.
|
| A lot of implementation are actually based on an ISO draft which
| changed in final release. But most dev do not have access to the
| spec. For example, timezone can only be specified as offset in
| the standard, while implementations accept name.
|
| Globally avoid ISO for software, it is non free crap.
|
| Also, do not use those standard for dates with timezone in the
| future. Use wall time/date and location. As timezone can change
| way more often than you think.
| idoubtit wrote:
| My own experience is that (unix) timestamps are much less
| error-prone than textual representations like ISO 8601 and
| such. A field like `update_time_seconds` is clear and easy to
| convert into any representation. This is what the Google _API
| Improvement Proposals_ recommends in most cases, though civil
| timestamps are also described. https://google.aip.dev/142
|
| Of course, when preparing queries against the API, you may need
| a helper to build valid timestamps. But this is mostly relevant
| to the discovery phase which isn't automated. And textual dates
| also have drawbacks, for instance the need to encode it when
| used a an URL parameter.
| kuon wrote:
| I agree, for timestamp, I often use ms or sec offsets. But I
| do not dare say it because I pass for an old fool.
| grishka wrote:
| Why use string-based timestamps at all? Use unixtime. It's much
| easier to parse and it literally can't be malformed.
| arendtio wrote:
| Sometimes timezones or the like change. So for future dates,
| those textual represations are probably better.
|
| Example: in November you create an appointment in 6 months at
| 15h, but then your government decides, to not use summer time
| next year.
|
| If you use timestamps your appointment will be wrong by one
| hour (humans tend to keep the 15h).
|
| In general, I am a huge fan of timestamps, but I think it is
| good to know where they have their limits.
| grishka wrote:
| > Sometimes timezones or the like change.
|
| Why would you ever want anything to do with timezones in an
| API? Even for appointments, it's still a timestamp.
| Timezone should be applied the very last moment, as part of
| date formatting for output on the client.
|
| If you allow your users to create appointments this much in
| advance and then they miss them, it's a UX problem, not an
| API design problem.
| gigatexal wrote:
| lol this could have been a single line blog post: "Do whatever
| stripe does with their APIs."
|
| I kid. There's some good stuff in here.
| pan69 wrote:
| Some nice tips in here. However, tip 15, I strongly disagree
| with:
|
| > 15. Allow expanding resources
|
| I would suggest the opposite. A REST API should not return nested
| resources at all. Instead, and to stay with the example provided
| on the website, to obtain the "orders", the /users/:id/orders
| endpoint should be called.
|
| It might be tempting to return nested resources, because clients
| would only have to make a single call.Technically this is true
| but once the domain of your API starts to grow, you will find
| that the interface will become increasingly muddled.
|
| The suggestion provided (use query parameters) is basically a
| work-around. If you want to offer this to your clients, front
| your REST API with a GraphQL API instead. It is literally the
| problem that GraphQL solves. Keep your REST API clean and dumb
| and focused around CRUD.
| magicalhippo wrote:
| > to obtain the "orders", the /users/:id/orders endpoint should
| be called
|
| Ok, but an order has an array of order lines, and each order
| line has sub-arrays as well, like details of the packages it
| was shipped in etc.
|
| So that might be 5-10 calls to get an order line, and for a few
| thousand lines we're looking at several tens of thousand calls
| to get a full order.
|
| Secondly, you then make some changes and want to replace the
| order with the data you got. You then have to produce a delta
| and upload that. And those thousands of calls better be done in
| a transaction of sorts, otherwise you'll have big issues.
|
| Seems easier to me to just be able to GET or PUT an entire
| order with all the details in one go.
| BeefWellington wrote:
| > Ok, but an order has an array of order lines, and each
| order line has sub-arrays as well, like details of the
| packages it was shipped in etc.
|
| > So that might be 5-10 calls to get an order line, and for a
| few thousand lines we're looking at several tens of thousand
| calls to get a full order.
|
| You definitely need to pick and choose the granularity you
| offer. There's a level of normalization in data structuring
| that approaches absurdity and this would be a good example of
| an absurd case.
|
| It is however an excellent demonstration of why making
| overbroad "you must never do X" rules is dangerous.
|
| I think a decent test is to ask yourself the question: "How
| much data is here that I don't need?"
|
| In the example referenced, if I'm looking at the full order
| history, I probably want to see only summary/header data with
| the option to view the full order details, so it wouldn't
| make sense to return to the user potentially a hundred
| thousand order lines' worth of data just because they're
| trying to view their history.
|
| If I then want to view the /orders/:id for one specific
| order, at that point it _does_ make sense to return all of
| the related lines, shipment details, etc.
| metadat wrote:
| Dear PAM69, this is great advice if you want to end up with a
| slow-to-load, low-performing web application that your
| customers complain about and hate using. But at least it'll
| adhere to a specific notion of architectural "purity", right?
| /s
|
| Anytime clients need to make 15 async calls before the UI can
| be displayed, you're headed up the creek. Generally speaking,
| this is an anti-pattern. There are exceptions, but they're not
| the rule.
|
| It's better to weigh the tradeoffs in any given situation and
| make a decision about bundling sub-resource references based on
| what you're optimizing for.
|
| A few quick examples:
|
| * Dev speed: Unbundled
|
| * Quick-loading UI: Bundled
|
| * Sub-resources are computationally-expensive to query?
| Unbundled
|
| This has been my experience consistently throughout 20 years of
| web-tier development.
| BulgarianIdiot wrote:
| Well they said use GraphQL, which is a solid way to avoid
| REST's clumsiness once and for all.
| nerdponx wrote:
| Sort of, but is it any better if the GraphQL layer still
| has to make 15 requests in order to serve a single useful
| response?
| pan69 wrote:
| But those 15 requests are then all occurring over a local
| network (in the data center), not over the Internet.
|
| The true power with GraphQL is that it might not even
| make all 15 calls because it will entirely depend on what
| you are querying for. E.g. if you query for a User but
| not the Orders for that User, then the request to
| retrieve the orders is simply skipped by GraphQL.
| nerdponx wrote:
| Great point.
|
| I have only used Apollo for GraphQL, and I found a few
| things about it offputting (e.g. I need a 3rd-party
| library to figure out what fields the client actually
| requested). What GraphQL server do you use? Or is Apollo
| + Express generally a good "default" option for basic
| setups?
| doctor_eval wrote:
| Also, of course, those 15 calls are occurring in
| parallel. I love how GraphQL makes all the complexity of
| marshalling data go away. Even when a GraphQL server is
| directly fronting an SQL database, I found the latency to
| be better than what I'd probably get if I was to code the
| calls manually.
| pan69 wrote:
| Like I suggested, use GraphQL to solve this problem. I know
| front-end teams that run their own GraphQL server to abstract
| away clumsy APIs and to optimize client/server requests.
| doctor_eval wrote:
| I think this is an unnecessarily harsh and sarcastic tone to
| take here.
|
| The comment you're replying to set out specific reasons why
| they disagree with expanding/bundling sub-resources. It
| obviously depends on your use case - and in fact they say use
| GraphQL, which I heartily agree with - but the point is that
| you don't always know how the API is going to evolve over
| time, and keeping things unbundled tends to be a "no regrets"
| path, while bundling resources by default, in my experience,
| can lead to trouble later.
|
| When the API evolves - as it probably will - using bundled
| resources ends up running the risk of either an inconsistent
| API (where some stuff is bundled and some isn't, and the
| consumer has to work out which), a slow API (because over
| time the bundled data becomes a performance burden), or a
| complicated API (where you need to embed multiple, backward-
| compatible options for bundling and/or pagination in a single
| operation). In addition, the bundling of resources commits
| you to I/O and backend resource consumption based only on an
| assumption that the data is required. None of this makes
| sense to me.
|
| In practice, if you can keep your API latency reasonably low
| and take a little bit of care on the client side, there's no
| reason a user should notice the few milliseconds of
| additional latency caused by a couple more API calls during
| the page draw of an app.
|
| It's not about architectural purity, it's about decomposing
| your application in a way that balances multiple conflicting
| needs. I agree with pan69, after many years of doing this in
| multiple contexts, my default has become to not bundle
| resources when responding to an API request.
| arnorhs wrote:
| I agree.
|
| One thing to add, is that there's nothing preventing you to
| invent a new noun for a particular rest resource that
| returns bundles of content. eg. /user/:id/dashboard - it
| makes it so this endpoint is not tied to eg. user/:id ..
| making that endpoint harder to change in the future, but
| also solves the issue mentioned by the rude comment above
| re: needing to perform a lot of separate rest calls.
| berkes wrote:
| Another two reasons to avoid nested resources, are performance
| and coupling.
|
| A nested resource is hard to optimize. Simple, atomic, flat
| resources can be cached (by clients, proxies or server) much
| more efficient. Once you allow nested resources, there's no
| going back, as clients will depend on them. So you've
| effectively disabled lots of performance improvement options.
|
| A nested resource implies data relations. Tightly coupled to a
| data model. One that will change over time, yet the api is hard
| to change. If you have a Project nested in your Users, and the
| business now needs multiple projects per user, this is hard to
| change with nested resources, but much easier with endpoints,
| the /users/:id/project can be kept and return e.g. the first
| project, next to a new /users/:id/projects.
| CipherThrowaway wrote:
| > you will find that the interface will become increasingly
| muddled.
|
| Will I? For example The Stripe API uses expand parameters and I
| prefer that approach to "atomic" REST or being forced to use
| GraphQL. There is a missed standardization opportunity for
| incremental graph APIs built on REST.
| JaggedJax wrote:
| I've consumed the kinds of APIs you're referencing and they are
| my least favorite. I would prefer a poorly documented API over
| one that returns me 15 IDs that I must look up in separate
| calls. I think there's a reason those APIs also tend to have
| rate limits that are way too low to be useful.
| synthmeat wrote:
| We're querying dozens, if not into hundreds, of GraphQL APIs
| for a couple of years now. Terrible DX, terrible performance,
| terrible uptimes. Everyone on the team, with no exception,
| hates them. Even a lot of those who produce them cobble
| together a bad REST implementation for parts of their own
| products.
|
| Agreed even at rate limits comment - frequently, probably in
| moments of desperation, they rate limit according to Host
| header to keep their own products up and working.
| RedShift1 wrote:
| > Terrible DX
|
| Can you elaborate what's terrible about the developer
| experience? If anything it's much better than REST, even if
| the developer of the API doesn't bother with documentation,
| the GraphQL schema is fully usable as documentation. Plus
| the way to input and output data into the API is
| standardized and the same across all GraphQL API's.
|
| > terrible performance
|
| How? Is the API slow to respond? Or are you making too many
| calls (which you shouldn't do) increasing the round trip
| time?
|
| > terrible uptimes
|
| I fail to see how that uptime is related to GraphQL. REST
| API's can be just as unreliable, if the server is down, the
| server is down and no technology can fix that.
| iratewizard wrote:
| runarberg wrote:
| I would rather you not assigning hate to the entirety of
| autistic developers. There are plenty of autistic
| developers who are fully capable of designing great APIs
| with awesome usability. Being autistic has nothing to do
| with how APIs are developed.
| iratewizard wrote:
| 8note wrote:
| That's not a rest-y API.
|
| You want
|
| GET /orders?user=Id
|
| Orders can be searched on many dimensions, not inherent to
| users
| pan69 wrote:
| True. But it sort of depends on the kind of relationship
| "orders" has. E.g. "orders" is a good example to use your
| suggested GET /orders?user_id=:id and that is probably
| because an order has a many-to-many relationship, e.g. with
| users and products (i.e. an order doesn't belong to neither
| user nor product). However, take something like an "address"
| which might belong to a user (one-to-many), i.e. the user has
| ownership of the relationship with an address, in that
| scenario you probably want to use GET /users/:id/addresses
|
| But then again, when it comes to API's, domain modelling is
| the hard part and it is therefore the reason why you don't
| want to return nested results/objects.
| Too wrote:
| Once your api query parameters reach a certain level of
| complexity, like such nested lookups, one should just consider
| giving direct proxied read access to the DB for clients that
| need it. Why reinvent your own query language. I Don't know
| enough about graphql to compare it. Databases have access
| controls also.
| CipherThrowaway wrote:
| > 14. Use pagination
|
| Generally agree with everything else but strongly disagree with
| pagination envelopes and offset pagination. Allow keyset
| pagination using ordering and limiting query parameters.
| barefeg wrote:
| The PUT vs PATCH is debatable in different levels. One simple
| issue is how to resolve complex merges in a PATCH. For example if
| we patch a key that contains a list, what will be the expected
| result?
| ajcp wrote:
| Even beyond PUT vs PATCH I found this statement to be rather
| naive:
|
| "From my experience, there barely exist any use cases in
| practice where a full update on a resource would make sense"
|
| Just in finance I can think of *dozens* of use cases around
| invoicing and purchasing *alone*. A lot of times thousands of
| resources may need just one field "corrected", but hundreds
| others "reset" to ensure the proper retriggering of routing,
| workflows, and allocations. That the resources need to exist as
| originals is incredibly important for these kinds of things.
| myshkin5 wrote:
| > 5xx for internal errors (these should be avoided at all costs)
|
| An anti pattern I've often seen has devs avoiding 5xx errors in
| bizarre ways. I would change the above to make to have monitoring
| in place to address 5xx errors. By all means, let your code throw
| a 500 if things go off the rails.
| polishdude20 wrote:
| We don't use PATCH but use a PUT for partial objects. We have
| validator code at every endpoint and we validate both creates and
| updates. When a PUT comes in, the validator knows what can and
| can't be changed. Depending on your role, the validator lets you
| change certain things can be updated as well. A PATCH would need
| these too and now you have more code to deal with. Also, it
| requires the developer to now worry that they have all the fields
| for a complete object or not.
| BulgarianIdiot wrote:
| Based on that description, you may be using PUT in conflict
| with its semantics (namely, idempotent way to replace an entire
| resource).
|
| This is one reason why I don't bother with these methods and
| stick to GET and POST.
| Supermancho wrote:
| > I don't bother with these methods and stick to GET and
| POST.
|
| Most people don't bother with them. If you need caching or
| want to be able to manipulate params in the browser/link to
| resources, use GET.
|
| This idea that you need additional verbs for web services is
| a classic case of in-theory vs in-practice. Introducing non-
| trivial complication for very tiny benefit is a strange
| tradeoff.
| [deleted]
| bcrosby95 wrote:
| PUT can be useful if you have a client that tries to handle
| errors. For example, our mobile application will
| automatically retry a PUT.
| perlwle wrote:
| https://www.vinaysahni.com/best-practices-for-a-pragmatic-re...
|
| Helped me get started with API design in my early career.
| Learning from other existing APIs helps too. such as stripe,
| github, shopify. Any others?
|
| Something that We do at my current job:
|
| * set a standard and stick with it. tweak it if needed. we even
| have naming standard on some of the json key. for example, use
| XXX_count for counting. when it doesn't make sense, use
| total_XXX.
|
| * Document your API, we use postman and code review API changes
| too.
| tlarkworthy wrote:
| A good resource, some others I consult with
|
| https://cloud.google.com/apis/design
|
| https://opensource.zalando.com/restful-api-guidelines/
___________________________________________________________________
(page generated 2022-03-12 23:01 UTC)