[HN Gopher] Implementing Microsoft REST API Filter
___________________________________________________________________
Implementing Microsoft REST API Filter
Author : skibish
Score : 50 points
Date : 2022-07-12 17:08 UTC (5 hours ago)
(HTM) web link (sergeykibish.com)
(TXT) w3m dump (sergeykibish.com)
| recursivedoubts wrote:
| don't make me tap the sign:
|
| https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
|
| _> I am getting frustrated by the number of people calling any
| HTTP-based interface a REST API. Today's example is the
| SocialSite REST API. That is RPC. It screams RPC. There is so
| much coupling on display that it should be given an X rating.
|
| > What needs to be done to make the REST architectural style
| clear on the notion that hypertext is a constraint?_
| vxNsr wrote:
| Hey, I'm kinda new to this area, can you show with examples
| what the differences are? I read the article, he does a lot of
| explaining but doesn't do much showing. Like why is a specific
| api more RPC and why a different one is a perfectly constructed
| REST api?
| recursivedoubts wrote:
| REST was the term Roy Fielding coined as description of the
| original web architecture. The big distinguishing feature of
| REST-ful systems vs. non-REST-ful systems was something
| called the uniform interface, where clients had zero
| knowledge about a given server's API surface beyond a URL
| entry point. Everything after that was encoded in hypermedia,
| giving us the term "Hypermedia As The Engine of Application
| State" (HATEOAS).
|
| Here is an article I wrote on it:
|
| https://htmx.org/essays/hateoas/
|
| RPC-style APIs require shared knowledge about a given end
| point: what arguments it expects, what data it returns, etc.
| Unfortunately, for historical reasons, we've come to call all
| HTTP JSON APIs "REST APIs". It's actually pretty funny.
| voidfunc wrote:
| You can tap the sign as much as you want, that battle was lost
| a long time ago. REST is just the common term people use for
| HTTP+JSON RPC.
| lkxijlewlf wrote:
| "Ugh... REST is too much work! We'll just change the meaning
| to suit us."
| jgalt212 wrote:
| True that. I use it incorrectly with non-technical folk as
| they may understand what a REST API is, but when I say "it's
| more like an RPC interface" their eyes glaze over.
| rafale wrote:
| You gotta say RESTful if you want to emphasis RESTfulness.
| But I personally don't care. I don't mind JSON RPC.
| masklinn wrote:
| Then say that?
|
| Rest pedants don't care if you don't use rest, as long as
| you don't call it rest. It's hardly a difficult concept.
| voidfunc wrote:
| Yea, but nobody cares what REST pedants think.
| recursivedoubts wrote:
| DONT THINK I WONT
| deathanatos wrote:
| ... and it's not _just_ the hyperlinking that 's problematic.
| (Yeah, MS's unRESTful "RESTful" APIs do a huge amount of
| coupling in the form of URL building.)
|
| E.g., in Azure, which is also a "RESTful" API that has no idea
| what REST is about, MS completely misses Fielding points that
| most of the effort of definition should be spent defining the
| content / data's format, not things like URL structure. That
| way we can speak about MIME types / content-types, and know
| what structure we're describing. But Azure will happily
| describe in JSONSchema a single type, and declare that it is
| used for both PUT/GET, and ... it's _not_. And discovering the
| additional constraints that exist on the type in the PUT is
| gleaned only through calling the API, certainly not through
| Azure 's docs. And that's assuming you get a usable error in
| response.
|
| JSONSchema is also a bit of a disappointment. On the one hand
| -- yay, a spec? But on the other hand, it fails to capture so,
| so much. Half the fields in the type will be required ... and
| the schema will say they're optional. Sum types of any kind are
| particularly badly handled, and half the time are just "string"
| though I think this is more of a failing on MS/Azure than
| JSONSchema, for simple string-like enums; but more complicated
| sum types, IDK if JSONSchema can't cut it or if MS just doesn't
| get it or _what_. For example, to instantiate a VM, the request
| body looks something like: body: required
| struct { properties: optional struct {
| storageProfile: optional struct { imageReference:
| optional struct { communityGalleryImageId:
| optional string, exactVersion: optional string,
| id: optional string, offer: optional string,
| publisher: optional string, sharedGalleryImageId:
| optional string, sku: optional string,
| version: optional string, } osDisk:
| optional struct { createOption: optional enum {
| "Attach", "Empty", "FromImage" } image: optional
| struct { uri: optional string, }
| managedDisk: optionalStruct { id: optional
| string, // omitted fields }
| vhd: optional struct { uri: optional string,
| } } // omitted fields }
| // omitted fields } // omitted fields }
|
| I've listed _only_ the fields used in determining where to
| source the VM 's OS disk from. And it's nuts! "properties" and
| "osDisk" are actually required; if you specify "imageReference"
| or "image" or probably "vhd" (but I've never used that myself),
| "createOption" must be "FromImage", if you specify
| "managedDisk" it must be "Attach", and the docs don't describe
| what meaning "Empty" has. You can specify only one of those,
| because otherwise, you're saying to source the image from two
| things which would be nonsense (but is permitted by
| schema/docs?).
|
| "imageReference" itself is really a sum type; you must specify
| (offer, publisher, sku, version[, exactVersion]), or
| communityGalleryImageId, or sharedGalleryImageId. You could
| image it being, enum ImageReference {
| FromMarketplace { offer: String, publisher: String, sku:
| String, version: String, exactVersion: Option<String> },
| SharedGallery(String), CommunityGallery(String),
| }
|
| And we've not even touched VHDs, managed disks, or VM images
| yet! And you don't need createOption.
|
| I _think_ , again, I'm going off what I've learned the hard way
| about how Azure works. I shudder to think what the validation
| logic looks like. (I'm also reading the docs. Reading
| JSONSchema is ... painful to start with, but Azure's schema's
| directory layout structure makes it triply painful.)
|
| But even that sum type is to miss the point of REST entirely.
| The RESTful definition would be: image: URI-
| reference
|
| and _that 's it_. The Content-Type of the content at the
| provided URI provides the type of image that it is.
|
| Oh and while I'm here: don't choose a boneheaded page size if
| you paginate an API call. Half of Azure's services will
| trickle-feed you 100 records at a time, and so the response
| body is like 60 KiB. Since the payload also has the next page's
| URI, your calls get decimated by latency. Some bad offenders:
| listing images in a repo in ACR gets ~ a phone modems worth of
| overall throughput. It takes minutes to download single-digit
| megabytes of image metadata. The Azure pricing APIs are
| similar: it's ~58KiB per page. The entire VM pricing data is
| something like 131 MiB, and that requires 2,235 HTTP calls to
| fetch.
| XVincentX wrote:
| Disclaimer: I work for Microsoft as an API Architect.
|
| I am not working on this specific API, so I am not going to
| comment on anyway. I hear your complains about Microsoft API
| guidelines (which is an entire different conversation) but I
| wanted to add my two cents with regards to JSON Schema.
|
| The problem that I have been having with JSON Schema since
| forever - is that the data that is being modeled is
| complected with contextuality of its usage. For instance, if
| I have
|
| type user = { name: string, surname: string, password: string
| }
|
| IN JSON Schema it is very hard to give contextuality on it,
| and most of the times involves having two separate types.
|
| Here is an example:
|
| If I am creating a new user, then name, surname are
| mandatory, while password is not because the system is
| autogenerating it. If I a logging in - then I want ALL of the
| fields.
|
| As of today, it is very hard in JSON Schema to express this.
|
| Basically speaking, I am arguing that the data structure is a
| thing, another one is its usage in a context, where there can
| be requirements and complicated validation logic involving
| even other fields
|
| In my experience, the only thing that has been very very
| close to what I have been looking for when modelling systems
| is Clojure. Most of the people laugh to my face when I say
| that primarily because it is a LISP 2 and yet... In
| particular, spec (and even better spec2) have the tooling to
| express data structure as sophisticated as we want without a
| type system and with the contextuality constraints that are
| fundamental for a real type reuse.
| AlphaSite wrote:
| I've debated back and forth on whether it's a good idea to
| have separate input and output models for each endpoint,
| because trying to have a generic structure that's usable
| everywhere makes it really easy to pipe output back to
| input for a PUT, but it's difficult to express constraints
| like, this field cannot be updated, only created or these
| fields are required but only for create and update supports
| a different subset of the fields again.
|
| I think ideally you want seperate structures but you need
| tooling which helps you map between output/input structure
| automatically (in strongly typed languages, it's easy in
| Python or JavaScript) and that's just lacking currently.
| deathanatos wrote:
| I'm not sure what you mean by "it is very hard to give
| contextuality on it"; OAS does supports referring to a type
| by reference, so that higher level types can reuse the
| definition of structs they might contain.
|
| But even so, here the problem is that the APIs aren't
| actual PUT/GETs: they payload types aren't the same going
| up as they are coming down. It is really two separate
| types, one for PUT, one for GET.
|
| Some of that is to be expected (there will be some
| information after the create that is only added by the VM
| coming into being) but how Kubernetes handles this with a
| separate "status" for the item I think ends up letting the
| rest of the type (spec, in k8s's case) be the same type.
| (... ish. K8s has variants of this problem, too.)
|
| To expand a bit, I'm largely relegated to the API docs
| themselves. Browsing the actual schema is hard:
| Start at: https://github.com/Azure/azure-rest-api-specs
| Descend into specification. Descend into ... so many
| choices ... compute. Descend into resource-manager.
| Descend into Microsoft.Compute Descend into stable
| Descend into -- and this is tricky! the latest
| version isn't the latest version. The latest
| version is 2022-04-04, but for VM creation it's 2022-03-01.
| The only way I know to determine this is to seek backwards,
| or find it in the docs. Descend into ComputeRP
| Descend into virtualMachine.json.
|
| And it's 3.3k LoC! Some of this verbosity is JSONSchema, to
| be sure... but still. And then you might have to wade back
| up to common.json, though I forget what circumstances cause
| me to need to look there.
| nightpool wrote:
| I'm not sure how you're blaming JSONSchema for "Half the
| fields in the type will be required ... and the schema will
| say they're optional". But JSONSchema has plenty of ways to
| define sum types: http://json-schema.org/understanding-json-
| schema/reference/c...
| ben_jones wrote:
| Am I right to call out most of this problem is caused by
| incessantly reusing types (because defining an additional
| type for i.e. CREATE vs PUT actions is soooo hard.
| deathanatos wrote:
| Hmm. Thank you for that, I seem to be mistaken there. I
| wonder if that was added after I acquired my knowledge of
| JSONSchema? Or if it was some weird thing w/ Swagger where
| it wasn't supported there.
| XVincentX wrote:
| Swagger (OpenAPI 2) and OpenAPI 3 never supported JSON
| Schema. They support a subset of the draft 4 with custom
| additions. OpenAPI 3.1 added support for additional
| schemas where you can use whichever draft you want.
| masklinn wrote:
| Might just be confusion with swagger / openapi, which is
| a complete mess of procedural constraints reminiscent of
| XML Schemas.
| derefr wrote:
| A true REpresentational State Transfer API would give each
| version of each of the API's L7-protocol request and response
| messages its own media-(sub)type; and would then rely on HTTP
| content negotiation to allow clients to specify which messages
| they're expecting to receive in response to a request; where
| the difference between throwing/nonthrowing, sync/async,
| new/old variant, etc., would all come down to which response
| message types the client is declaring themselves to accept.
|
| Don't think I've literally _ever_ seen this done in practice.
| Don 't think I've ever seen a "REST client" library that even
| offers the possibility of negotiating with a backend that's
| attempting to do this -- let alone doing it in a streamlined
| manner.
| recursivedoubts wrote:
| no, the whole media type thing was a moderately-interesting-
| idea-turned-ridiculous-navel-gazing turn that, among many
| other pedantic blind alleys, turned everyone away from the
| deeply innovative aspect of REST: the uniform interface, and,
| in particular, HATEOAS
|
| https://htmx.org/essays/hateoas/
|
| any idiot (such as myself) who has ever made a web 1.0 app
| has created a better REST API than 99.9% of all REST API
| engineers, so called
| Lazare wrote:
| One suggestion I've seen (eg here:
| https://cloud.google.com/blog/products/api-
| management/unders...) is to distinguish between REST and
| OpenAPI designs.
|
| Because you're right, REST is a specific thing, but we do need
| _some_ name for what everyone is talking about, and just saying
| "RPC" is much too vague.
| qxmat wrote:
| I did the same a while back.
|
| I used ANTLR4 to generate a .NET tree walker that would build up
| an IQueryProvider expression. IQueryProvider would then compile
| to Expression<T> and self-optimise (simplifying boolean algebra
| and removing redundant expressions). We hooked this up to npgsql
| and viola - a sane, typesafe, query string DSL without a single
| line of SQL.
|
| Nowadays I hack together Terraform and Python ML- I miss .NET
| dearly... it was a simpler time.
| revskill wrote:
| To me, REST focus on Resource first. RPC is Producedure first.
___________________________________________________________________
(page generated 2022-07-12 23:01 UTC)