[HN Gopher] Write OpenAPI with TypeSpec
___________________________________________________________________
Write OpenAPI with TypeSpec
Author : bterlson
Score : 85 points
Date : 2024-03-26 21:56 UTC (1 days ago)
(HTM) web link (blog.trl.sn)
(TXT) w3m dump (blog.trl.sn)
| tyayers wrote:
| This is typical Microsoft - overly complicated solution in search
| of a problem. Yaml and Json are beloved for a reason - they're
| simple and effective.
| karmajunkie wrote:
| I think "beloved" is a bit of a stretch... tolerated perhaps,
| for their simplicity and versatility, in that you _can_ make
| them work for most kinds of data transfer and configuration if
| you don 't mind shoehorning your needs into that spare box. But
| beloved? hardly...
| eyelidlessness wrote:
| Just because you don't have the problem it's solving doesn't
| mean the problem doesn't exist.
| dz0707 wrote:
| Having written quite a bit of open API specs, I don't agree
| with you. Json is hard to read, yaml has own quirks, especially
| when you try to spilt it into parts. Amazon also tries to
| invent own language for describing apis, so I guess they are
| not happy with open API too. Anyway, without ability to
| generate code from spec, there is not much use from it. Code
| gen/nswagger/open API generator and others produce terrible
| code, at least for java/c#/typescript(there's 4.1k open issues
| for open API generator), using custom generators for codegen
| make problem less painful, but that is additional burden, I'm
| looking for better alternative, would be very interesting to
| see what they will do with code generation.
| bterlson wrote:
| Codegen is coming online as we speak. We do codegen from
| TypeSpec in Azure across multiple languages, and the results
| are pretty great. We're moving that over to the TypeSpec
| project so everyone can generate code. Obviously my opinion
| is biased, but I think the results are significantly better
| than what you find elsewhere in the ecosystem.
| alecthomas wrote:
| I would be absolutely stunned if anyone, ever, has described
| OpenAPI as "beloved". It does its job, but that's about the
| best you can say about it.
| beders wrote:
| Yaml is beloved when it's maybe two pages of little indented
| config.
|
| It becomes a nightmare once people try to emulate a DSL with
| it.
| dylmye wrote:
| Writing OpenAPI by hand sucks, this looks fantastic. The only
| reason I prefer generators from code comments is because the
| structure and the code itself live in the same place, so it's
| harder to forget.
| buzziebee wrote:
| Looks interesting. Can it handle asyncapi specs too?
|
| We use boats (npm) to define our openapi and asyncapi specs which
| uses file based structure to separate definitions for models,
| paths, params etc into separate files which makes it much more
| maintainable. Native refs make it super easy to reuse
| definitions, and you can write custom helpers in js to abstract
| things like the Page definition in that example.
|
| Having a shared templating language for openapi / asyncapi specs
| which is ergonomic and can be used by more than just JS devs is a
| great idea. I'll keep an eye on this project.
| bterlson wrote:
| Hey, author here. Right now we don't have any streaming or
| eventing support, but I am working on it as we speak. My first
| goal is to support describing SSE and JSONL HTTP streaming
| endpoints, but I want to work toward an AsyncAPI emitter as a
| peer for our OpenAPI emitter.
| Alupis wrote:
| My biggest frustration when working with AsyncAPI is how
| _different_ it 's bundling logic is versus OpenAPI tooling,
| like Redocly.
|
| They're both based on JSON Schema, but AsyncAPI's bundling
| logic seems to fall down a lot when facing `$ref` in strange
| places (even though the actual spec allows), such as ref'ing an
| entire channel or operation.
|
| Looking at boats, it looks a lot like regular open/async api
| spec documents, with a few features sprinkled in. How great is
| the transpiled output? Does it pass standard validators
| provided by OpenApi and AsyncApi? Does it support AsyncApi 3.x?
| LunaSea wrote:
| I wonder why this isn't written in Typescript directly so as to
| handle documentation, validation and type definitions in one go.
|
| Especially if you want to reuse type definitions of objects
| elsewhere in your code.
|
| An API request or response usually do not contain the exact model
| attributes. Only the public facing ones which are then often
| filtered down some more, depending on the user making the request
| (and their roles, groups, permissions).
| eyelidlessness wrote:
| > I wonder why this isn't written in Typescript directly so as
| to handle documentation, validation and type definitions in one
| go.
|
| I suspect they want TypeSpec to be more runtime agnostic (as
| also tends to be the preference with TypeScript). Specifically
| to support arbitrary validation/serde.
|
| Having written a fully integrated generalized solution
| (runtime-defined schema -> parser, types, documentation,
| serializer), by far my biggest regret the first time around is
| coupling it so closely to a particular schema runtime. Partly
| this is because I have bigger dreams for designing a schema
| system purpose built for the task (I had previously piggybacked
| on io-ts, which worked but it took considerable non-essential
| effort and complexity); partly it's because there can be
| significant tradeoffs between different runtime solutions in
| terms of performance and ergonomics.
| bterlson wrote:
| Our first attempt at a TypeSpec-like thing was in fact a
| TypeScript dialect! It works great for simple cases, but at the
| limit it falls over for a number of reasons. HTTP needs a lot
| of metadata that isn't found in TypeScript, things like HTTP
| verbs, query/path params, headers, routes, etc. API patterns
| also need their own metadata, like for pagination you need to
| know various bits of metadata that describe how to paginate an
| endpoint. Also many things in the TypeScript type system and
| stdlib don't apply in the context of API descriptions, so we
| needed a subset.
|
| So after going down this road for a while, we found we were in
| a place where the syntax was actually very complex, with
| metadata spread between deeply nested generic type
| instantiations and JSDoc comments, and didn't feel at all like
| TypeScript. This didn't align with our goals of having
| something simple and easy to learn for all devs.
|
| So I wrote the first implementation of a custom language in a
| day and folks really liked the direction. 4 years later, here
| we are!
| DanHulton wrote:
| Wow, I was _just_ thinking that it would be excellent to have
| something like this the other day.
|
| I really prefer spec-first development, since it gives other
| developers an opportunity to review the API _before_ the changes
| start being implemented, and trying to do this from a codegen-
| based OpenAPI implementation frequently leads to the PR being
| "broken", since you've only changed the signature of all your
| methods and not the implementation yet.
|
| But changing a big OpenAPI yaml file is really hard to easily
| review, and breaking it up into included files only helps a
| little, honestly. We can compile the new spec and host the HTML
| in a temporary location to make it easier to view what the new
| spec will look like, but once you've done that, you're no longer
| looking at the diff of what's changed.
|
| TypeSpec looks terse enough to be easy to review (and to write!)
| which really looks like it'll help with that. I'll have to mess
| around with it in some personal projects of mine with reasonably-
| complicated specs and see if there's no obvious speedbumps first,
| but I'm hoping not, because I'd love to start using this in all
| the projects I'm contributing to!
| emi2k01 wrote:
| This is cool! I made a similar project to this one when we
| switched from GraphQl to regular REST at $work but decided to
| drop it since I didn't have the bandwidth to work on it.
|
| I see editor support for VSCode, but is it backed by an LSP or is
| it VSCode only?
| bterlson wrote:
| It's LSP, and we also have a VS extension in the marketplace.
| skrebbel wrote:
| Question for the author, why not fully adopt TypeScript? Ie have
| TypeSpec be a well-defined, limited subset of TypeScript? At
| first glance, it seems to me that a model maps to an interface,
| an op is a function, and an enum is a string literal union.
|
| I understand that you can't express everything that way, eg
| annotations, but you could get pretty far with putting those in
| JSDoc comments, no? I could see a big practical benefit to TS
| code being able to read an API spec directly without another
| conversion step, so I'm curious why you chose not to go with
| that.
| bterlson wrote:
| I answered in more detail in another comment, but we actually
| tried this first, and couldn't make it work for the kinds of
| complex APIs we deal with in Azure. But, it worked great for
| simple APIs, and I hope someone releases such a tool at some
| point.
| mdasen wrote:
| As someone who has used JAX-RS (Java) and ASP.NET, APIs are
| basically created with these kinds of annotations right in the
| language. @GET public Character
| getCharacter(@PathParam("id") int id) { return
| db.getCharacter(id); }
|
| That's very similar to TypeSpec's op
| getCharacter(@path id: safeint): Character;
|
| Java and C# classes already have the type information that you'd
| be getting from a TypeSpec: // TypeSpec
| model Character { name: string; id: safeint;
| status: "Alive" | "Dead"; class: Class; }
| enum Class { warrior; wizard; } // C# public
| class Character { public string Name { get; set; }
| public int Id { get; set; } public Status Status { get;
| set; } public CharacterClass CharacterClass { get; set;
| } } public enum Status { Alive, Dead }
| public enum CharacterClass { Warrior, Wizard }
|
| But with C#/Java you don't need to write the spec and make sure
| that it stays in sync with what your endpoints are actually
| doing. TypeSpec looks a lot better than writing YAML by hand, but
| I'd still rather something that was tied to the actual code that
| will be executing so that things are never out of sync or wrong.
| lijok wrote:
| This is the code-first vs schema-first debate. Both have their
| pros and cons. Personally a die hard advocate of schema-first.
| Petermarcu wrote:
| Agreed. If the schema can also generate the service routes,
| models, serialization, etc, and you just have to maintain the
| business logic in the service, you get the spec matches the
| service benefits that way as well. Best of both worlds?
| People using gRpc seem to be 100% fine with schema first and
| generating the service stubs.
|
| One of the key things with having the spec is that you can
| actually describe a lot more than you can with the various
| attributes and comments in the code. Especially things that
| are not as much service concerns but potentially client
| concerns or documentation concerns. You can also encapsulate
| reusable API patterns so you know different operations are
| following the same pattern.
|
| The author didn't go into all the details but there are lots
| of ways in TypeSpec to separate the concerns of different
| consumers of the spec. There is a lot of opportunity and
| creative thinking that can be done here.
| mnahkies wrote:
| Yeah I'm also on the schema first side of the debate.
|
| I think for me it comes down to a few key points:
|
| - APIs are forever, the choice of language/framework is an
| implementation detail
|
| - Constraining yourself to what can be represented in the
| specification is better than generating a specification from
| implementation that may not be capable of expressing the full
| details
|
| - When working with diverse languages it provides a common
| ground/language for discussing API changes. Eg: if you have
| java backend, kotlin android, swift iOS, react/whatever web
| you can bring everyone together with the spec
|
| - Subjective, but a good spec will include a bunch of
| documentation and examples that tend to create a lot of noise
| in the code. I personally prefer to keep this in the spec and
| the implementation smaller
|
| I think the main counterpoint to this is that you can
| generate the spec and then take that and change your mind if
| you later change language/framework etc - it's not a one-way
| door.
|
| My biggest bug bear is that regardless of spec first or
| implementation first, you should have something you write
| once and generate the rest of the glue from (eg: docs, client
| sdks). Writing each piece manually/independently always leads
| to drift and bugs.
|
| (I'm working on my own little openapi -> typescript code
| generator over here https://github.com/mnahkies/openapi-code-
| generator - eventually plan to support more than typescript,
| and adding typespec support is something I'm currently
| considering)
| bterlson wrote:
| Hey, nice work on openapi-code-generator, its output is
| very nice. I think you could probably package it up as a
| TypeSpec emitter without too much difficulty, by emitting
| OpenAPI and feeding it to your generator. I am doing a
| similar thing for a Kiota emitter I'm working on. We
| recently added an API to get the OpenAPI as a JS object
| which may be of help in this quest:
|
| https://typespec.io/docs/libraries/openapi3/reference/js-
| api...
| mnahkies wrote:
| Thanks for the reference! This is essentially what I had
| in mind - allow typespec as input and transparently
| convert to openapi then feed to the existing process.
|
| I suspect there's potential for some loss of information
| with this approach (I'm relatively new to typespec so not
| sure) but I plan to do this as a first pass and then
| consider a more direct approach later if I see gaps worth
| fixing.
| k8svet wrote:
| Cool, it certainly feels like there's some unrealized better way
| to describe APIs -- but the part that's more interesting for me
| is the codegen story? Or some way to validate the service I
| implemented is actually conformant?
|
| I guess for now it translates to OpenAPI, but the footnote
| implies that TypeSpec-driven codegen could be better. Maybe more
| is coming soon?
| Petermarcu wrote:
| The author has an example repo demonstrating some of the early
| support for codegen. https://github.com/bterlson/typespec-todo
|
| Server side codegen is also one the horizon and is a key way to
| keep the spec and service in sync. So many possibilities here!
| lijok wrote:
| What are people's thoughts on this vs AWS's Smithy?
|
| We've been looking to buy into Smithy heavily as we no longer
| have the appetite to deal with OpenAPI and it's horrendous
| ecosystem.
| hocuspocus wrote:
| I would pick Smithy if I had to go spec-first.
|
| I currently tolerate OpenAPI because I'm using a robust code-
| first approach (Tapir in Scala) on the server side, but I
| otherwise agree, the ecosystem seems brittle.
| Stoids wrote:
| I like the idea, especially the TS-like syntax around enums and
| union types. I've always preferred the SDL for GraphQL vs writing
| OpenAPI for similar reasons. Most APIs I've run into in my career
| would benefit from modeling their API responses as ADTs versus
| the usual approach of overloading 4 different union members into
| a giant spare object.
|
| I echo the sentiment others have brought up, which is the trade-
| offs of a code-driven schema vs schema-driven code.
|
| At work we use Pydantic and FastAPI to generate the OpenAPI
| contract, but there's some cruft and care needed around exposing
| those underlying Pydantic models through the API documentation.
| It's been easy to create schemas that have compatibility problems
| when run through other code generators. I know there are projects
| such as connexction[1] which attempt to inverse this, but I don't
| have much experience with it. In the GraphQL space it seems that
| code-first approaches are becoming more favored, though there's a
| different level of complexity needed to create a "typesafe"
| GraphQL server (eg. model mismatches between root query resolvers
| and field resolvers).
|
| [1] https://github.com/spec-first/connexion
| henry_pulver wrote:
| Great idea - spend far too long reading & writing OpenAPI!
|
| Particularly anyOf, allOf and oneOf (especially when nested) lead
| to really confusing nested specifications in OpenAPI. Really like
| how TypeSpec handles unions & intersections.
|
| Playground is great for getting a feel for it fast too
| mooreds wrote:
| As someone who wrote a custom dsl to Open API spec script (in
| Ruby) I'm really interested to see if type spec would be a better
| output. Our spec is > 12k lines and we haven't even included
| everything that needs to be defined.
|
| Does anyone have experience layering this on to an existing API
| (as opposed to defining the API and then building the code)?
| bterlson wrote:
| Extensive experience inside Azure. In general, supporting
| existing APIs is harder, predominantly because it might not
| sufficient to produce a semantically identical OpenAPI when
| downstream tools are sensitive to e.g. whether something is a
| ref or not, the order of properties in the document, whether
| something uses `const` or an `enum` with a single member, etc.
|
| If you have more specific questions I'm happy to consult!
| ramilefu wrote:
| I've been toying with the design of something similar here and
| there over the past couple of months. It never made it past the
| pen-and-paper stage though. Pleasantly surprised to find this
| today and will have to try it out for a work project.
|
| My original motivation was the lack of OpenAPI definitions across
| projects, despite everyone agreeing their existence is a "best
| practice". As you suggest, developers just truly hate writing
| them. Unfortunately, even the "generate it from code" approach
| tends to be complicated and often feels duct-taped together.
| zexodus wrote:
| TypeSpec is great, but if you're working with Rust and you're
| about to write a new project that will require an OpenApi spec
| sooner or later, I'd like to recommend a web framework that has
| spec generation baked in:
|
| https://github.com/poem-web/poem (see poem_openapi)
|
| All you need to do is derive a trait on your response structs and
| in return you get an almost perfectly generated spec. Unions,
| objects, enums are first class citizens.
|
| Also, if you're from coming from PHP, the controllers feel very
| much like symfony controllers.
|
| P.s. Please do recommend an ORM that would feel closer to
| doctrine. I miss doctrine.
| mmcclure wrote:
| We're also spec-first at my co (pretty mature, large API surface
| area), but maintaining the OpenAPI spec in YAML is extremely
| annoying and tedious. This checks a lot of boxes in terms of
| being interesting, but what does the path from an existing YAML
| spec to TypeSpec look like? In an ideal world we could just
| ingest our OpenAPI spec, get the TypeSpec version out, then be
| able to clean up/iterate from there.
|
| I didn't see anything in the docs, but I would love for someone
| to tell me this exists.
|
| Re: the post itself, this was much more compelling than
| TypeSpec's website imo. At the very least it feels like some of
| these examples, such as the enum bit, should be ported there.
| Also interesting that there isn't a single mention of languages
| like Cue[1].
|
| [1] https://cuelang.org/
| bterlson wrote:
| It does not exist, but it will be worked on fairly soon. You
| can track progress on GitHub [1]. It will be as you suggest - a
| one time conversion, after which you can iterate. We have this
| workflow for inside Azure, but the converter is Azure-specific
| (e.g. converts to TypeSpec that uses our extensive Azure-
| specific component library). It will take a bit to generalize.
|
| [1]: https://github.com/microsoft/typespec/issues/3038
| dajonker wrote:
| We have been using this for a couple of months now and it works
| really well. It's so much better than writing OpenAPI by hand,
| and I never ever will do that again after working with TypeSpec.
|
| Thank you!
| patrick-fitz wrote:
| This looks great, is there a converter to convert an OpenAPI yml
| file to TypeSpec?
| bterlson wrote:
| Not yet, but we're working on it. Should be out in a few weeks.
| tholm wrote:
| As someone who prefers schema-first design this looks awesome.
| Having to hand cut OpenAPI specs is gross and quickly becomes
| unwieldy.
| btown wrote:
| Ooh, magick.css ( https://css.winterveil.net/ ) spotted in the
| wild!
|
| Discussion on magick.css from 5 days ago:
| https://news.ycombinator.com/item?id=39793513
| bterlson wrote:
| It's a beauty! I had to get on the train early before it's
| everywhere.
| dadadad100 wrote:
| I don't see a way to specify the security scheme. Am I missing
| something?
| bterlson wrote:
| This docs page will hopefully help:
|
| https://typespec.io/docs/libraries/http/authentication
| jaxelr wrote:
| You can, see here:
| https://typespec.io/docs/libraries/http/authentication#avail...
___________________________________________________________________
(page generated 2024-03-27 23:01 UTC)