[HN Gopher] The perfect configuration format? Try TypeScript
___________________________________________________________________
The perfect configuration format? Try TypeScript
Author : tmcneal
Score : 120 points
Date : 2021-11-17 15:54 UTC (7 hours ago)
(HTM) web link (reflect.run)
(TXT) w3m dump (reflect.run)
| s_gourichon wrote:
| Dhall seems to gather some praises, and not fall into the pits
| mentioned here. https://news.ycombinator.com/item?id=17523623
| wyager wrote:
| Dhall is awesome. Only downside is that the included formatter
| doesn't preserve comments. Super well-designed and with
| interesting features like semantic hashing.
| harunurhan wrote:
| Not typed support but I find jsonnet [0] pretty useful when you
| have tons of configuration files.
|
| [0] - https://jsonnet.org/
| ThePhysicist wrote:
| I think parsing YAML or JSON into typed structures is the easier
| way to go. I e.g. do that in Golang using a little form
| validation and coercion library I've written. The end result is a
| nested, strongly typed data structure. Here's an example:
| https://github.com/iris-connect/eps/blob/master/settings.go (the
| accompanying form validation configuration:
| https://github.com/iris-connect/eps/blob/master/forms/settin...).
| Hashicorp has some libraries that do similar stuff, I wrote my
| own though since I need it quite often and wanted to have full
| control over it. The library is just a few hundred lines of
| Golang code (https://github.com/kiprotect/go-
| helpers/tree/master/forms). Undocumented as of now as it's
| nothing I want to explicitly share with the world.
|
| In my experience, a lot of the validation needs to be done at
| runtime anyway as type checking alone won't allow you to e.g.
| validate if a string is a valid regular expression. Also, I think
| using TypeScript for configuration requires you to compile &
| interpret the configuration file in order to check it and obtain
| the actual configuration values. Not sure if I like that as it
| requires bundling the entire TypeScript compiler with your
| program.
| nsm wrote:
| I'm not on board with using a language that only has a single
| implementation and is complex enough that implementing others is
| difficult.
|
| Please use a language designed for configuration, but still sane.
| There are several if we leave the JSON-inspired family. Starlark,
| pystachio, Dhall. All are restricted, but allow functions and
| code reuse. All are simple enough to write a new interpreter for.
| tn890 wrote:
| Honorable mention for Cue[0].
|
| [0] https://cuelang.org/
| Kinrany wrote:
| CUE does look simple enough to be implemented in every
| language. I wonder if there's anything preventing that.
| Perhaps there's space for a CUE-like language with an even
| simpler spec.
| throwaway894345 wrote:
| TypeScript has at least 2 implementations: tsc and Deno, but I
| take your point and agree. That said, most languages probably
| have a JS runtime, so a basic implementation would be
| ts->js->interpreter. The ts->js is presumably pretty easy.
| junon wrote:
| I don't think Deno actually re-implements typescript, does
| it? Seems like a lot of wasted effort. They don't even use
| Typescript internally.
| dangoor wrote:
| I'm pretty sure Deno just uses the open source TypeScript
| compiler: https://github.com/denoland/deno/tree/main/cli/tsc
| remorses wrote:
| They use swc for transpilation and bundling but, yes, for
| checking the only solution available now is tsc
| pbalau wrote:
| If you can afford to wear only one hat, then sure, use one
| special language for configuration. But I can't, I am the
| entire infra department and I still need to do product and
| framework work. Not having to switch context, language wise, is
| a very nice thing to have.
| lpapez wrote:
| Geniuenly curious, what downsides do you see with a single,
| reference implementation? I come from C++ world where we have
| several competing compilers, each slightly different from the
| other, and all this incurs a non-negligible maintenance cost
| while providing very little benefit in my eyes.
| Zababa wrote:
| The problem is not a single, reference implementation, the
| problem is how hard it is to implement another one.
| "Implementing" JSON is relatively easy, most languages have a
| JSON library. Most people could probably write a JSON
| library, not a perfect one, but that would mostly work. And
| probably do it in a reasonable amount of time. All of that
| isn't true for Typescript.
| kortex wrote:
| Ignoring S-expr based configs for a moment, of the
| configuration languages, JSON parsing is probably the most
| straightforward. YAML is a superset of JSON, TOML is
| probably on par, TS and JS are obviously more complex than
| JSON, and XML is...XML. All of that is to say, I'd rather
| write a JSON parser than any of the above.
|
| And JSON parsing is a minefield.
| http://seriot.ch/projects/parsing_json.html
|
| I don't know of any notable S-expr based config languages,
| probably because they are _so_ easy to parse, people just
| roll their own, leading to balkanization.
| deckard1 wrote:
| Windows INI format is going to be easier than any of
| those. You can parse it in a couple lines in a bash
| script.
|
| I personally find it silly to need more than that. YAML,
| JSON, etc. What are we even doing here, besides trying to
| make life more difficult? Are we doing configuration or
| are we doing plugins and calling it configuration?
|
| We'd all be better off if we just admitted we really want
| an extension language and went with embedded Lua or
| Scheme. With Dockerfile and others you have to run a
| separate lint program, assuming anyone even bothers doing
| that. At least with a proper language you would have
| built-in robustness. Most of these configuration
| formats/DSLs are half-assed and unnecessary anyway.
| strifey wrote:
| I'm curious what folks who agree with this think about setup.py
| then? There's been a push to move to setup.cfg to specifically
| avoid the fact that python libraries run arbitrary code on
| install.
| nonameiguess wrote:
| The problem with setup.py is different. Because it can run
| arbitrary code, package writers go crazy and import extra
| dependencies, but you specify your package dependencies in
| setup.py, so without some out-of-band way to tell your users
| that the setup.py file in itself also has dependencies, package
| installation on machines other than your own might fail unless
| by sheer luck your users already had the packages being used in
| setup.py.
|
| That problem doesn't apply to arbitrary configuration, but only
| to configuration files that represent package build and
| installation metadata. You do see this issue with things like
| Gradle, though, where your build.gradle file can represent the
| dependencies of the package being built, but the build file
| itself may have dependencies since you can run arbitrary Groovy
| code and import any package that can be contained in a jar
| file. It gets around this because Gradle itself can just
| download those dependencies before trying to execute the build
| script, which would be perfectly possible for a real Python
| build system, but not when setup.py is just being executed by
| the Python interpreter, which doesn't do dependency management
| and expects imports to already be available in the system
| import path.
| nonameiguess wrote:
| Heck, for what it's worth, even Make has this issue, as
| people quite often write Makefiles assuming you have a bunch
| of tools installed that may or may not be part of the POSIX
| specification. I wish I could find it now, but somewhere in
| the GNU Make guide book is a list of more or less "approved"
| utilities you're supposed to be able to safely assume are
| installed on any GNU based system that you can use in a
| Makefile, and you're not supposed to go beyond that.
| jayd16 wrote:
| Kind of interesting but now I need to run arbitrary code to read
| a config?
| taeric wrote:
| To be fair, you already have to execute arbitrary code to
| understand what config is doing.
|
| Where execute is to sequence a series of effects that are
| described in a file.
| jayd16 wrote:
| >To be fair, you already have to execute arbitrary code to
| understand what config is doing.
|
| That's not arbitrary code. That's my compiled, tested and
| verified code.
| taeric wrote:
| All code is arbitrary code if you are paranoid enough. :)
| intellix wrote:
| YAML is beautiful when it's simple and horrible when you need to
| dynamically generate it. The indentation...
| beebeepka wrote:
| Typescript is perfect for stuff like this because it's so much
| less verbose than most things out there
|
| As a bonus, you might not need validation and schemas, too.
| enriquto wrote:
| Typescript has a lot of punctuation and thus it's not really
| human-editable. I'd prefer a ini-style format or dockerfile style
| (with one directive per line of text, clearly identified by a
| single uppercase word). Moreover, these formats are so easy to
| parse that you do not even need a library to do so.
| tomtheelder wrote:
| I'm puzzled what you mean by this. Do you mean it's not easily
| editable by non-engineers?
| enriquto wrote:
| It's annoying to edit by everybody. I guess engineers will
| especially abhor the unnecessary complexity.
|
| This is the proposal of TFA: export const
| config: MyAppConfig = { locale: 'en-US',
| timezone: 'America/New_York', logLevel: 'info',
| environment: 'local', }
|
| And this is how simple it could be: LOCALE
| en-US TIMEZONE America/New_York LOGLEVEL info
| ENVIRONMENT local
|
| The first version has 19 useless punctuation symbols and a
| few spurious words that are not related to the configuration
| itself, but are constraints of the typescript implementation.
| The second version is much simpler, easier to write and to
| parse using trivial string processing, without depending on
| any third-party library.
|
| It is beyond me how could any serious engineer honestly find
| any advantage in the first version.
| ComputerGuru wrote:
| Except there's the ini where strings need to be quoted and the
| one where they don't. The one where white space is allowed
| after the equal sign and the one where it's not. The one where
| keys are case sensitive and the one where they're not.
|
| TOML cleans up most of that, though.
| emodendroket wrote:
| OK, so we're coming back around to the "configuration as code"
| trend again then.
| zffr wrote:
| Apple takes a similar approach for swift. Package.swift files run
| real swift code and export an object that specifies the package's
| configuration.
|
| In practice I think it works fairly well.
| eyelidlessness wrote:
| I have used TypeScript for config in a few projects, with node-
| config[1], and it's substantially better IME than JSON/YAML/etc.
| Yeah it's unconventional to write configs in a general purpose
| language, and I understand why that would be undesirable. But
| having configs validated at compile time is great at preventing
| bugs.
|
| I'm not sure I'd recommend node-config (it was chosen by a past
| team). Its magic undermines the confidence in type safety
| provided by choosing TypeScript in the first place. But it does
| give some good baseline conventions one can apply without all the
| magic.
|
| 1: https://github.com/lorenwest/node-config
| melolife wrote:
| I've done a similar thing with Scala in the past - ship the
| compiler jar and then compile your configuration (which is just
| an abstract class implementation) as part of the startup process.
| ravenstine wrote:
| My only real criticism is that I believe it's generally an anti-
| pattern to have configuration that necessitates DRY. It has
| essentially the same problems as OOP inheritance. Because
| configuration isn't and _shouldn 't_ be an application in and of
| itself, avoiding potential issues and misdirection from avoiding
| DRY in this case seems like the right tradeoff to having to use
| multi-select to change the config format in your IDE.
| Slasher1337 wrote:
| I've really come to prefer json5 as a config format:
| https://json5.org/
|
| It's basically Json with comments, trailing commas, and unquoted
| object keys (and some other minor things).
|
| Much nicer than real json as a config format, but still simple
| and declarative, not requiring an interpreter or being turing-
| complete.
| escherize wrote:
| Good to see people coming to the same conclusions we have had for
| a while using Clojure. There's an up and coming runtime data
| validation framework called Malli that I use for these sorts of
| files.
|
| You can get many of the benefits but also run custom validations
| (dictate how two keys in a map relate to eachother, or force a be
| of a certain shape instead of just a string).
|
| Here's the example from the script: [0]
|
| [0] - https://malli.io/?value=%7B%20%3Alocale%20%22en-
| US%22%0A%20%...
| SavantIdiot wrote:
| Except ....
|
| (...No one asked, but here are my $0.02 ;-)
|
| You need typescript in your environment to use typescript!
|
| The four formats the OP listed can be read by anything from Ada
| to EXCEL. But typescript can only be ready by a TS build flow.
|
| In fact, I think JSON was a bridge too far. A config file
| shouldn't need a _compiler_.[1]
|
| FOR GENERAL PURPOSE: big no.
|
| BUT: If this is purely for TS projects where the tooling already
| exists, then OK. I'd use it.
|
| EDIT [1] I'm referring to the article's early suggestion to run
| JSON through JSMin to support comments. So transpiling is more
| apt, not compiling.
| [deleted]
| Zababa wrote:
| > In fact, I think JSON was a bridge too far. A config file
| shouldn't need a compiler.
|
| JSON just needs a parser, like most other configuration
| formats. It's purely declarative.
| __ryan__ wrote:
| > In fact, I think JSON was a bridge too far. A config file
| shouldn't need a compiler.
|
| Does JSON need a compiler?
| a_lost_needle wrote:
| It does not. Easily implemented with combinators the whole
| way down.
| SavantIdiot wrote:
| Correct, but the very beginning of the original article
| proposes running JSON through JSMin to support comments. So
| in this context, a "compiler" is being proposed (or
| transpiler to more modern). My comment isn't clear.
| a_lost_needle wrote:
| Still easily implemented with combinators the whole way
| down. It's building an ast, that's it. Once you have a
| tree, you can do whatever you want with the result before
| passing it to a compiler for a language. Here though,
| just serialization stuff.
| SavantIdiot wrote:
| Ah, I see. I can't reply to that thread anymore, but it
| sounds like a combinator is like a first-class YACC
| grammar in the functional language that obviates the need
| to write the parser. That's really cool, thanks. I'll
| read up on it more.
| a_lost_needle wrote:
| Yea, that's pretty accurate. So now I'm curious what you
| thought the site was named after?
|
| Edit: It's not that accurate. It's just math used in a
| way to compose parsers using parsers.
| SavantIdiot wrote:
| Never gave it a moment's thought. Now after Googling it,
| I still wouldn't know because I don't program LISP (well
| I haven't since 1986, but that was only an intro in
| college). Interesting to read about it.
| a_lost_needle wrote:
| You can use most of the concepts in any language that
| allows first class functions, closures, pattern matching
| and types. Combinators are just ways to compose a
| function to create a new function that executes some
| aspect upon a set (usually).
| SavantIdiot wrote:
| Like grandma used to say: "There's nothing combinators
| can't solve. Except consumption. Rest in peace Grandpa
| Neddie."
|
| I'm dense: what's a combinator?
| a_lost_needle wrote:
| A functional programming technique that allow you to
| declaratively define your language when using parsers as
| values. If you have a good functional language
| implementation, the code look very similar to someone
| able to read BNF. And then you run it, and get out an AST
| the back end.
|
| edit: Forgot partial application. Not strictly required,
| but super handy.
| Zababa wrote:
| That's one proposed solution but not the only one. You
| could just ignore comments, like most programming
| languages do.
| nly wrote:
| Protocol Buffers text format ?
| joshuamorton wrote:
| Composability in configuration is really important beyond a
| small scale. If you're running a replicated server in n
| locations, it's far better to keep one config file that has
| some form of conditional logic based on location than to
| maintain n textprotos that need to be kept synced.
| jonny_eh wrote:
| Protobuffs remain very underrated outside of Google.
| whirlwin wrote:
| The idea of allowing logic from a programming language in a
| config format is.... at best very horrifying...
| tantalor wrote:
| Using programming languages for configuration is a bad idea.
| bryanrasmussen wrote:
| why? It can't be because then if something goes wrong you need
| to debug your configuration, because you need to do that often
| enough with a non-programming language format for even less
| power offered.
| taeric wrote:
| Emacs would like a word. :)
| threatofrain wrote:
| Then should we should do what the industry does instead --
| start with a flat configuration language like YAML and then add
| modules, variables and scopes, iteration...
|
| And what do we do when we want the Cartesian product of two
| sets? Ahem Terraform.
| IshKebab wrote:
| Why? The only reasons I can think of are:
|
| * They can be non-deterministic (do a different thing each time
| you run them).
|
| * They can be non-hermetic (access stuff in the environment you
| don't know about).
|
| * They can do naughty security things.
|
| * You can't present GUIs of them because they aren't
| declarative.
|
| All but the last one don't exclude programming languages.
| Here's an interesting project to make hermetic deterministic
| Javascript (Typescript support is planned):
|
| https://github.com/jkcfg/jk
|
| For the sorts of places where you don't have a GUI for the
| settings anyway (which is the common case) I think it makes
| loads of sense. It beats making the kind of declarative
| programming languages you see in YAML files.
| latchkey wrote:
| Static declarations without a reason are equally bad.
| inoffensivename wrote:
| gradle has entered the chat
| kaba0 wrote:
| You are not supposed to write actual build logic inside
| gradle though, as many people do. It is supposed to be a DSL
| for build scripts the same way maven is, with slight ease of
| use scripting.
| deschutes wrote:
| Gradle does suck though.
| l_r wrote:
| Why do you think so? I couldn't disagree more. Task
| caching/avoidance, rich build domain model (source sets,
| configurations, tasks, extensions), huge plugin ecosystem.
| deschutes wrote:
| My opinion is based on my experience integrating artifact
| generators (both code and config) into a build. I recall
| being funneled into building a plugin by documentation to
| deal with some idiosyncrasies of the generator. Whereas I
| just wanted to write a small script. In general I felt
| like there were a lot of extra hoops and concepts to
| learn to do basic easy shit. And there really isn't any
| payoff. For example I recall scripts and plugins have
| different capabilities. And that some things can't be
| expressed in untyped Gradle.
|
| I also found that the user documentation was just a skosh
| above useless. It felt like it was more concerned with
| not being wrong than being helpful. Reminiscent of older
| javadocs in the java collections library.
|
| The same could be said of many build systems, I suppose.
| rectang wrote:
| Couldn't agree more -- a hard requirement for a configuration
| language is that it must be _inert_.
|
| Under some circumstances, configuration needs to be generated
| dynamically so you _need_ a programming language -- and
| TypeScript would be fine for that. But a "perfect"
| configuration format isn't perfect if it can run arbitrary
| code.
| int_19h wrote:
| Using pure programming languages for configuration is fine. The
| trick is that the config "program" should basically just output
| the final - static - configuration, and set of inputs should be
| very strictly controlled (i.e. no opening random files etc).
| eb0la wrote:
| Really interesting but, sincerely, I would rather use sqlite.
|
| I know: It's a binary format. And you need to add a library to
| read it.
|
| _BUT_
|
| - Your configuration _has_ data types.
|
| - You can have a copy with an old config within the file easily
| (create table backup_config as select * from actual_config; ).
|
| - The file is small.
|
| - Encryption is built-in if you need it.
|
| - And it's programable
| shadowgovt wrote:
| I like the idea of the IDE assisting with the schema-correctness
| of the config (which you get for free with TypeScript coupled
| with any competent IDE configured to use the typechecker to
| validate the code).
|
| ... but as a rule of thumb, if your config language is Turing
| complete, you're setting yourself up for future pain as
| complexity of config grows with time and someone makes the bad
| decision to take advantage of that Turing completeness...
| the_arun wrote:
| How to add comments in json or TypeScript config files?
| Commenting is very much needed for any manually written code or
| config we keep in repositories.
| vander_elst wrote:
| What would be the problem with using something like protobufs? To
| write the configuration, you can use whatever language you want,
| what matters are the outputs anyway. The outputs are stored by
| automation in text format so that they can be better inspected.
| The programs that use such configuration have the advantage of
| knowing the schema of the configuration and directly parse it
| avoiding possible typing problems. The nice thing is that you can
| use the same language to write the programs and the
| configuration.
| mikojan wrote:
| protobuf may very well be total overkill. What you wanted to do
| was strongly type your configuration. What you get is a
| complete RPC layer.
|
| This might not even be warranted when inter-process
| communication is a primary concern. After all, there's nothing
| inherently wrong with using whatever data transfer mechanism
| your standard lib provides.
|
| It's not just protobuf though. Almost every data format for
| serializing structured data suffers from pushing a jungle on
| you when all you asked for was a banana.
|
| The least invasive format I know of is FlatBuffers[0]. Doesn't
| push RPC on you. Allows you to parse a message even if you do
| not know this specific message's data structure in advance
| (like JSON).
|
| [0]: https://google.github.io/flatbuffers/
| vander_elst wrote:
| 1. As said in the original comment, it doesn't necessarily
| have to be protobuf, but just a data serialization system. 2.
| Protobuf per se is just a serialization system, it does not
| push RPC "Protocol buffers are Google's language-neutral,
| platform-neutral, extensible mechanism for serializing
| structured data - think XML, but smaller, faster, and
| simpler." [0]. It can be used in combination with RPC
| systems, but does not have to.
|
| [0] https://developers.google.com/protocol-buffers
| pphysch wrote:
| My money is on https://cuelang.org in this space.
|
| As I understand it it's essentially the Google Configuration
| Language (GCL) author responding to Hashicorp Configuration
| Language (HCL), with language development modeled after Golang.
| Still in early stages but very promising.
| progre wrote:
| TypeScript needs to be compiled, right? So what use is a
| configuration file if I need to run it through the build
| pipeline?
|
| The point of having _configuration_ is to gain the ability to
| _configure_ software without rebuilding in my mind. Maybe I 'm
| missing something.
| dragonwriter wrote:
| > TypeScript needs to be compiled, right?
|
| Yes.
|
| > So what use is a configuration file if I need to run it
| through the build pipeline?
|
| Compilation doesn't necessarily happen in a "build pipeline";
| typescript is available as a library.
|
| > The point of having configuration is to gain the ability to
| configure software without rebuilding in my mind.
|
| There are many different purposes of configuration, and they
| are sometimes in tension, which is why there are so many
| configuration formats (spanning from limited languages like
| TOML through general purpose programming languages, whether
| interpreted or compiled) with an ebb and flow as to which
| approaches are more and less popular over time.
| klodolph wrote:
| The fact that it is compiled does not mean you need to run it
| through an additional step in your build process. In practice,
| it would just mean that you include TypeScript in your package,
| which can either be used as an executable or as a library. It
| can be completely transparent to the user.
|
| You can even get by without including TypeScript, if you are
| willing to just strip out the type information (there are
| various libraries that do this).
| handrous wrote:
| I think we got here because everyone's mixed up _describing a
| state_ with _describing the process to describe a state_. Or,
| configuration versus scripts that _apply_ configuration. Most
| of the popularly config-as-code systems mix the two up,
| including Ansible, my favorite one. You end up with "config
| files" that are, all at once in the same file: 1)
| configuration, 2) templates for configuration, and 3) programs
| that modify config when they're run.
|
| I think the whole approach is just fundamentally misguided.
| Each of these systems should probably use 2-3 different
| languages & formats, not just one. And especially not YAML.
| Ugh.
|
| So you're right, _config files_ should not be code.
| Unfortunately, much of "modern" config management involves
| writing _programs_ in _config file formats_ , which is also
| extremely bad in a whole different way.
| nverno wrote:
| > To use this approach today requires that your project also be
| written in Typescript. It also means you'll need to forgo a
| separate flat-file for your configuration.
|
| You're right, the summary presents this configuration as a
| stop-gap, a possible pre-cursor to a better, future JSON.
| strogonoff wrote:
| TypeScript is limiting for describing schema in my experience.
| Try requiring an array of certain length, for example.
|
| Also, it's Turing-complete, so you are enabling future developers
| or users to shoot themselves in the foot and write something that
| does not halt already at configuration stage.
|
| To me the perfect language for configuration seems to be Dhall,
| but it doesn't seem to have gained any traction.
| throwaway894345 wrote:
| Dhall is too unfamiliar for most developers, IMO. Developers
| don't want to have to grok an entirely new syntax just for
| config. I'm personally surprised no one has added type
| annotations to Starlark yet.
| mattw1810 wrote:
| The Rust implementation has it as an experimental extension
| (https://github.com/facebookexperimental/starlark-
| rust/blob/m...)
| throwaway894345 wrote:
| Oooh, very cool! I'll check it out and maybe port to Go.
| strogonoff wrote:
| Everything is unfamiliar, until one adopts it.
| [deleted]
| throwaway894345 wrote:
| I didn't think it needed to be explained, but not all
| things are equally unfamiliar nor equally easily adopted.
| ahuth wrote:
| Depending on what you're looking to do, you can use a tuple
| type to get an array of a certain length.
|
| [number, number, number]
| strogonoff wrote:
| I meant length that depends on another type, not length known
| beforehand (e.g., if N widgets are defined somewhere, [up to]
| N elements can exist somewhere else, or configuration should
| not compile). Naturally, this had to involve generics
| (similar to [0], except I don't recall why but the `length`
| property workaround didn't work or didn't provide sufficient
| type safety in my use case). Eventually, after reading
| workarounds in a related issue[1], I gave up. TypeScript's
| type system is great for its intended use, but when applied
| to expressing schema constraints it quickly falls short and
| runtime code execution is required to validate a given
| structure with all the corresponding drawbacks.
|
| [0] https://stackoverflow.com/q/56034226/247441
|
| [1] https://github.com/microsoft/TypeScript/issues/26223
| jonny_eh wrote:
| > not length known beforehand
|
| Is that a feature of other languages?
| strogonoff wrote:
| Haven't had much exposure to other statically typed
| languages recently, so can't say off the top of my head.
| Intuitively it looks like something modern generics
| should allow easily though, doesn't it?
|
| As to Dhall, for example, I could validate such a
| constraint (and more) using a function I can be
| reasonably sure is safe and won't halt (i.e. without all
| the drawbacks of TS and other general-purpose languages).
| ljm wrote:
| I've only worked with cdk8s, but the experience of that alone was
| enough for me to regret it. It's not that cdk8s is bad, per-se,
| it's that it turned an otherwise declarative kubernetes config
| into something that was much harder to grok in the first
| instance.
|
| One of the arguments for TS in the post is that it can encourage
| DRY. It was DRY that made this TS-based kubernetes config so hard
| to follow, because simple repetitive statements got abstracted
| into parameterised functions, or classes to inherit from, or
| other things that are resolved dynamically at runtime. Not to
| mention that you're also going to start mixing in random
| dependencies from NPM to help abstract things further. This is
| great for application development but not so great for pure
| configuration.
|
| A language designed for config, like Dhall or CUE, can give you
| much of what a typed language offers without letting you go crazy
| with a full-blown language runtime.
| tetha wrote:
| Those were my concerns as well. Your config file can stay
| simple, sure. But I'm pretty sure it won't.
|
| And from some years of experience with configuration management
| systems, configuration creation can accrue a surprising amount
| of complexity over time. Mostly because unlike the code they
| support, a config management system has to support many
| versions of the code and multiple versions of a configuration
| at the same time.
| dogleash wrote:
| Sounds like the Configuration Complexity Clock
|
| http://mikehadlow.blogspot.com/2012/05/configuration-complex...
| jeffreygoesto wrote:
| Did I just hear ABAP@10? No, must be my brain predicting...
| steeve wrote:
| Starlark or Cue, but not a general purpose language please. Side
| effects in a configuration language is an atrocity...
| IshKebab wrote:
| It's possible to sandbox most languages, and with some work you
| can probably make them deterministic too.
|
| Here's an example: https://github.com/jkcfg/jk
|
| That beats having to learn an entirely new language.
| fay59 wrote:
| Probably important to mention that at the moment, TypeScript as a
| configuration format means anyone who writes a configuration blob
| for you has arbitrary code execution. This is rarely an issue in
| practice, though.
| mejutoco wrote:
| This could be solved by having some kind of sandbox
| (https://github.com/patriksimek/vm2), but I agree it
| complicates it.
|
| It would be cool if tsc had a flag --sandboxed or similar that
| does not allow any sideeffects (fs access, output, forking, net
| requests, etc)
| realrocker wrote:
| One of such sandboxes is Deno.
| cormacrelf wrote:
| Not even Deno can solve the halting problem, unfortunately.
| rkeene2 wrote:
| Tcl's safe interpreters [0] have various limits like
| instruction count, command count, time, and memory [1].
|
| [0] https://www.tcl.tk/man/tcl8.6/TclCmd/safe.html
|
| [1] https://www.tcl.tk/man/tcl8.6/TclCmd/interp.html#M48
| Zababa wrote:
| No need to solve the halting problem, just put a timeout.
| cormacrelf wrote:
| If you use it by running a V8 isolate that transforms the
| exported config into JSON, then sure. But TS-based config
| is largely for people using TS elsewhere. And in that
| case: // this returns in 0 ms
| export const config: any = { get value() {
| while (true) {} } } as const;
| // accessing config.value runs forever
| console.log(config.value);
|
| For a less unrealistic perspective, if you write enough
| configuration in a Turing-complete language, you will
| eventually be tempted to use the full power of the
| language. You may find yourself having to write tests for
| the config, which might be e.g. parsing other files,
| making HTTP requests, querying the filesystem/OS and
| more. If you're doing that, you are no longer really
| writing configuration.
|
| The point of having a config file in the first place was
| so that it could be replaced, at start time or better yet
| at runtime, without touching the rest of the code, such
| that the same code can be parametrized and instantiated
| as you wish. When you use a fully-powered language, this
| becomes a matter of discipline instead of something
| guaranteed by the format. Again, this applies more
| strongly where you are importing the source file directly
| instead of translating it to JSON.
|
| ALL of the other competing config file formats are
| attempts to walk this line between too little expressive
| power and too much expressive power, such that people
| naturally separate their code and config. Typescript-as-
| config just abandons this goal; it's not a new idea, just
| one that has been rejected over and over again by each
| new format because everyone else saw the benefit of
| limiting expressive power.
| cphoover wrote:
| It also means your configuration must be transpiled into JS
| first before execution unless run through a typescript runtime
| interpreter.
| the_duke wrote:
| Deno is sandboxed by default and doesn't allow disk or network
| access, which is ideal for configuration.
|
| It also runs Typescript directly.
| crabmusket wrote:
| Be careful with terms like "directly" as people seem to get
| very confused when talking about Deno. "Transparently" may be
| more accurate.
|
| My understanding of the situation right now is that by
| default, Deno runs TS files by transpiling using SWC, which
| strips types without checking them, and then sends the result
| to V8. Optionally you can enable type checking which invokes
| TSC (with no emission) before SWC. All this is done
| "magically" from the user's point of view.
| pramodbiligiri wrote:
| Anyone have experiences to share about Dhall (https://dhall-
| lang.org)?
|
| On their homepage they call it a "programmable configuration
| language that you can think of as: JSON + functions + types +
| imports"
| Reitet00 wrote:
| Yeah, I've used that a year ago or so.
|
| Pros: not Turing complete but quite powerful, nice type system
| and functions. For example one can write a type safe config
| migration from v1 to v2 (and check statically that it works),
| exports to JSON, YAML etc.
|
| Cons: uses Haskell like syntax, no bindings to various
| languages. You basically need to ship the dhall binary to
| convert the config to something else that your target
| application will consume.
| ReleaseCandidat wrote:
| > no bindings to various languages.
|
| There are bindings to some languages: Haskell, Clojure, Go,
| Rust, Ruby and Purescript. https://docs.dhall-
| lang.org/howtos/How-to-integrate-Dhall.ht...
| junon wrote:
| Sorta. I evaluated it a long time ago and decided against ever
| touching it due to its remote import functionality. Apparently
| it's crazy to think that your configuration files shouldn't be
| issuing HTTP requests.
| strogonoff wrote:
| > Apparently it's crazy to think that your configuration
| files shouldn't be issuing HTTP requests.
|
| Depends on what language runtime can do with those responses.
| Arbitrary code execution in a config (a la Django settings
| and indeed OP's idea) can do much more damage, but even that
| doesn't scare many people as this kind of configuration is
| considered an inherently trusted source.
| sbr464 wrote:
| I use TypeScript and like the ecosystem.
|
| For broader use, I would like to see native support or an
| official extension for creating dependent types or specific
| constraints.
|
| Things like number ranges, or the basics from JSON Schema, etc.
| taylodl wrote:
| You know what we don't need? Yet Another Configuration Format!
| Also, don't tell me I can't add comments to JSON because I sure
| can - add it as a field! If a comment is that important then it's
| important enough to transfer along with the dataset and durably
| persist.
| pstch wrote:
| Is evaluation lazy in TypeScript ?
|
| For me, one of the most important features of a configuration
| language is the ability to directly refer to other values of the
| configuration, and more importantly, in the case of a
| configuration split in separate modules, to refer to values that
| could be provided by the other modules.
|
| Essentially, this means that the final configuration should be
| the fixed point of a function, that the user would provide the
| configuration as a definition of this function (or in the case of
| multiple modules, as multiple definitions that would be merged
| using function composition), and that the evaluator would be able
| to determine the fixed point of this function.
|
| I feel like this kind of configuration (used notably in NixOS)
| provides a lot of what's missing in traditional configuration
| languages, makes it easier to compose configuration modules,
| without requiring to write the configuration in a Turing-complete
| language.
| richeyryan wrote:
| Typescript is definitely eagerly evaluated unless you were to
| use a first class function to defer execution
| richeyryan wrote:
| I feel like the AWS CDK is a great example of this working well.
| Rather than puzzling over what config options are valid, I get
| autocomplete and compiler errors when I insist on using an
| incorrect option. You also get stuff like deprecation warnings
| showing up in your editor.
|
| It also becomes trivial to parameterise config so I can setup the
| same general Cloudfront distribution once and tweak it based on
| the individual application. It's open to abuse but it's better
| than interpolating values into Cloudformation templates.
| Lucasoato wrote:
| This is so cool, I feel like other similar projects, like
| k8cdk, are going in the same direction for that reason.
|
| The only problem with AWS CDK is that sometimes the type hints
| aren't clear enough in Python, that part could have been a lot
| smoother imo.
| [deleted]
| Shadonototra wrote:
| I can't believe what i am reading...
| crate_barre wrote:
| JS people gonna JS.
| npsimons wrote:
| > JS people gonna JS.
|
| Also reinvent the wheel. They can list four other
| configuration formats du jour, but conveniently leave out
| s-expressions, which have solved most problems people bring
| up over half a century ago.
| 11235813213455 wrote:
| or json-schema with jsonc or js files
| ReleaseCandidat wrote:
| Great idea. And because you need many constants that often change
| in such a configuration file you could put that all in another
| file and include that in the 'real' configuration file. Why not
| call that, hmm, `config_config.txt`. But as plain text is a bit
| hard to parse, we could give that file some structure, so that it
| is like, well, another markup language, just without markup and
| easier to read and write.
| sirtimbly wrote:
| The line between configuration and code is always a little
| blurry. There are some projects where using a .js file for the
| "configuration" of another operation is incredibly useful. TS
| makes even more sense in many of those situations. Lots of
| configuration files require pre-processing already, like merging
| shell env vars in with textutil or something like that. So, I
| wouldn't dismiss this opinion outright.
| realrocker wrote:
| Great little writeup ! After mangling YAML, HCL, JSON for years
| as an ops engineer, I have come to the same realisation. In fact,
| I have put this into practice in production pipelines by using:
| jkcfg[1] for the last couple of years. Two data points: 1. Zero
| developer support contract rate around YAML syntax and templating
| issues 2. High number of contributions in our private typescript
| configuration library from developers. Using typescript as an ops
| frontend has made operations a lot more approachable to folks.
|
| Recently I took what learnt in the last 2 years using
| jkcfg/typescript and taken it to Deno in form of an opinionated
| port of jkcfg called: dxcfg[2]. Its early days, but I would bet
| on Deno/typescript for future ops configuration.
|
| [1] https://jkcfg.github.io/#/ [2] dxcfg:
| https://github.com/dxcfg/dxcfg
| fourseventy wrote:
| Personally I wouldn't use Typescript for config. It breaks my #1
| dev rule which is KISS. Keep it simple stupid. Simplicity is
| orders of magnitude more important than other things like DRY or
| config schema.
___________________________________________________________________
(page generated 2021-11-17 23:01 UTC)