[HN Gopher] The Yaml document from hell
___________________________________________________________________
The Yaml document from hell
Author : ruuda
Score : 489 points
Date : 2023-01-12 10:15 UTC (12 hours ago)
(HTM) web link (ruudvanasseldonk.com)
(TXT) w3m dump (ruudvanasseldonk.com)
| chrizel wrote:
| I've never met anyone who say's "I like YAML, it is great"...
| most people that worked with it say something like "YAML is
| annoying, I don't like it"...
|
| While introducing Kubernetes at our company in the last two
| years, we are currently in a process going more and more away
| from YAML with internal Helm charts to a much simpler process by
| just using HCL and Terraform, and defining Kuberentes resources
| as Terraform resources.
|
| As a software developer HCL just makes so much more sense than
| this YAML + Helm + Go templates hell, which feels like C
| preprocessor hell all over again. Other solutions like kustomize
| are neat, but I don't see how all of these YAML workarounds
| should be better than something like HCL with Terraform. HCL
| feels like a real declarative programming language (with real
| conditions, variables, a module system and useful built-in
| functions). YAML feels like another more complex JSON and other
| tools like Helm or Kustomize try to work around the weaknesses of
| YAML with some kind of templating system.
|
| YAML looks nice to read in simple demos and in small files, but
| is just not adequate in the real world (in my personal opinion -
| I know that YAML is used by a lot of people in production as of
| today).
| jeroenhd wrote:
| I don't think YAML is great, but I still think it's the best
| format out there.
|
| The only confusing problem I've run into was the sexagesimal
| number notation and even that was fairly obvious. Perhaps it's
| because I tend to overquote strings?
|
| I mean sure, the on/off to boolean mappings are annoying, but
| they also become very obvious when you're parsing config
| because the type validation will fail. If `flush_cache` has an
| enum `on` but no key `True` then the type validator will
| instantly complain about both the missing key and the extra key
| in the dictionary.
|
| Same with accidental numbers, any type check will show that the
| parsing failed.
|
| I find JSON for config files to become unreadable quickly
| because of the non-obvious nesting and the lack of comments.
| You can pick a JSON extension but then you need to pick one
| that your tooling will support.
| jefftk wrote:
| _> I still think it 's the best format out there._
|
| What do you think of https://toml.io ?
| jeroenhd wrote:
| TOML solves a lot of issues but I find it hard to visualise
| when you get much deeper than two or three levels.
| jacurtis wrote:
| Exactly this. I hear so many people recommend TOML over
| YAML.
|
| I see the logic in it. For simple Key-value
| configurations, TOML is superior and more straightforward
| to YAML. You can add sub-level values and it isn't too
| bad (if there aren't too many), but beyond two levels,
| TOML becomes difficult to use.
|
| If you really work in YAML in any sort of more advanced
| capacity (kubernetes, Ansible, CI/CD Pipelines) then you
| really need the complexity that YAML provides. You also
| get used to the "gotchas" mentioned here. Navigating them
| is fairly straightfoward.
|
| I think the article was vastly overblown. Is YAML
| perfect? Certainly not. But you find a better way to
| display such complex data structures in a more human-
| readable and human-writable way. The complexity is YAML's
| strength, but it comes with caveats as all complexity
| generally does. I really think its the best we have.
| xwolfi wrote:
| Yup exactly my experience as well, again a stupid idea to try
| and make a "configuration" language out of nested key value
| pairs that end up needing fancy interpreters allowing more and
| more semantic into the keys and values to start doing what a
| simple program could have done in half the time...
|
| I ve worked in 4 companies over a period of 10 years, each had
| exactly this problem, with yml, json, xml, properties file (you
| dont want to see business logic conditionals in a properties
| text file, where the keys shapes command an interpreter to
| behave dynamically...)
|
| The only times I saw a team do it well was a php backend of all
| things where the lead said they d program all their variations
| in php rather than source it from configuration flat
| descriptors and it was amazing, clear, simple and powerful.
| They had to release the backend at each config change instead
| of releasing the config change only, but Im still unsure why
| exactly that's a problem: the configs are software too if we re
| honest with ourselves, shoe-horning them in a descriptor
| language isnt gonna make them flat.
| ptsneves wrote:
| The problem is mixing data with logic. I cannot imagine how
| maintainable a small program becomes when this concept is
| employed, much less a big one.
| ilyt wrote:
| I think problem in this particular case is using YAML as DSL.
| Every other data format would be equally bad here. Replace YAML
| with TOML and you're still in same templating hell.
|
| YAML is least worst for me, and I don't think I ever hit the
| problems article is showing because
|
| * I use editor that will highlight stuff like anchors
|
| * I often generate config from CM so it can't have those errors
|
| * Loading into defined struct in statically typed language also
| makes them impossible.
| tipiirai wrote:
| I like YAML. More specifically, the subset of YAML, like the
| author suggests. Clear, intuitive, and allows expressing of
| complex data structures like JSON does. Much better than TOML,
| which easily becomes a mess with more complex data.
| andybak wrote:
| Yeah. My headcanon YAML is amazing.
| mickeyp wrote:
| I like YAML.
|
| I like that you can use anchors and merges. It greatly
| simplifies complex, repetive structures. And most of the
| complaints about yaml can be worked around by string-quoting.
|
| The whitespace can get in the way if you're templating, but
| then you can also use [1, 2, 3] as a list notation, for
| example.
|
| In fact, most of the complaints could be resolved by running it
| through a linter.
| dijit wrote:
| > I've never met anyone who say's "I like YAML, it is great"
|
| Maybe I'm older than you, but I have definitely heard that
| line.
|
| Mostly because the alternatives were XML, INI or the myriad of
| bespoke formats, relayd/apachehttpd .conf or iptables etc;etc;
|
| INI has parsers that operate in different ways and doesn't
| support heirarchies... so that's not ideal.
|
| JSON and YAML came to the fore around the same time, and JSONs
| limitations in comments and it's picky semantics meant that
| people _did_ prefer YAML over JSON for human readable configs.
|
| YAML itself is _fine_ , it has some really awkward warts and
| the parsers are usually programatically unsafe in their
| implementation (leading to less compatible "safe_load" or other
| types of loaders)[0]; the issue we actually have with YAML is
| that we:
|
| A) Template it (jinja, mustache whatever)
|
| B) Put entirely too much stuff into it. (kubernetes manfiests
| can grow to the hundreds of lines really easily)
|
| These problems will affect _any_ configuration file format we
| choose to use, including TOML (which is comparatively new on
| the block), because reading templated /enormous files is really
| difficult.
|
| What I've taken to doing is programatically generating objects
| and then serialising them as whatever my software depends on.
| It might feel ugly to use an entire turing complete language to
| generate objects that are mostly static: but honestly... the
| ability to breakpoint, test and print the subsections of output
| is astonishingly nice.
|
| Then I don't care at all what the format is.
|
| [0]: https://www.serendipidata.com/posts/safe-api-design-and-
| pyya...
| falcolas wrote:
| I kinda miss XML. Yeah, I'm weird.
|
| The tooling is super mature, it's easy to emit, it's easy to
| parse, it's easy to validate, it can just a little hard to
| read and write by hand (and I mostly blame SOAP for that).
| Still, basic XML isn't _that_ hard to read or write, thanks
| to editor support.
| dijit wrote:
| I still wouldn't want to read this, and this is a simple
| example. :\
| <apiVersion>apps/v1</apiVersion>
| <kind>Deployment</kind> <metadata> <name>
| some-deployment </name> <namespace>
| deployment-namespace </namespace>
| </metadata> <spec> <replicas>1</replicas>
| <revisionHistoryLimit>3</revisionHistoryLimit>
| <selector> <matchLabels> <app>
| some </app> </matchLabels>
| </selector> <template> <metadata>
| <labels> <app> some
| </app> </labels> </metadata>
| <spec>
| <serviceAccountName>dbuser</serviceAccountName>
| <nodeSelector> <iam.gke.io/gke-metadata-
| server-enabled>true</iam.gke.io/gke-metadata-server-
| enabled> </nodeSelector>
| <imagePullSecrets> <name>dckr-auth</name>
| </imagePullSecrets> <containers>
| <name> some-service-container
| </name> <image>
| dckr.io/some/deploymentImage:latest
| </image>
| <imagePullPolicy>Always</imagePullPolicy>
| <ports>
| <containerPort>8000</containerPort>
| </ports> <env>
| <name>DB_URL</name> <value>postgresql://p
| gsql%40{{.Env.GCP_PROJECT_ID}}.iam@127.0.0.1:5432/{{.Env.NA
| MESPACE}}-{{.Env.SERVICE | strings.TrimSuffix
| "-service"}}</value> </env>
| <env> <name>LOGGING_LEVEL</name>
| <value>debug</value> </env>
| <env> <name>TRACING_ENABLED</name>
| <value>false</value> </env>
| <env> <name>APP_NAME</name>
| <value> some </value>
| </env> <env>
| <name>AUTH_URL</name>
| <value>http://auth/auth</value> </env>
| <env> <name>KEYCLOAK_KEYSET</name>
| <value>/protocol/openid-connect/certs</value>
| </env> <env>
| <name>MAILCHIMP_URL</name>
| <value>https://mandrillapp.com/api/1.0</value>
| </env> <env>
| <name>STREAMING_SASL_ENABLED</name>
| <value>true</value> </env>
| <env> <name>CORS_ENABLED</name>
| <value>true</value> </env>
| <env> <name>CORS_ALLOWED_ORIGINS</name>
| <value>*</value> </env>
| <env> <name>CORS_ALLOWED_METHODS</name>
| <value>*</value> </env>
| <envFrom> <secretRef>
| <name>kafka-access-secret</name>
| </secretRef> </envFrom>
| </containers> <containers>
| <name>cloud-sql-proxy</name>
| <image>gcr.io/cloud-sql-connectors/cloud-sql-
| proxy:2.0.0-preview.3</image> <args>--auto-
| iam-authn</args> <args>--structured-
| logs</args>
| <args>{{.Env.SQL_CONNECTION_STRING}}</args>
| <securityContext>
| <runAsNonRoot>true</runAsNonRoot>
| </securityContext> </containers>
| </spec> </template> </spec>
| falcolas wrote:
| Honestly, it doesn't feel too bad to me. But it's also a
| bit overly nested for many things; I wouldn't want to
| read that in JSON either.
|
| XPath (and JQ) FTW here.
| [deleted]
| lostmsu wrote:
| Reads fine to me, although would benefit from a few
| element->attribute switches.
| IanDrake wrote:
| [dead]
| SuperSandro2000 wrote:
| I saw NixOS
| singingfish wrote:
| YAML is nuts, and JSON is annoying (trailing comma limitations,
| lack of comment syntax no matter how annoying it is that the spec
| is correct about why there are no comments).
|
| Both have their place though. YAML came out of perl, and both are
| some confluence between awesome and horriffic (although yaml wins
| the horrific crown for sure).
|
| I've had a little bit to do with Ingy - the inventor of yaml, and
| I've worked closely with some of his collaborators. Ingy is nuts,
| mostly in a good way, but I wouldn't put him in charge of the
| architecture, I'd put him in charge of the abyss.
| reilly3000 wrote:
| > I wouldn't put him in charge of the architecture, I'd put him
| in charge of the abyss.
|
| :-) I enjoyed this bit a bit too much, now I want it on my
| tombstone #lifegoals
| api wrote:
| Do people dislike TOML only because it looks like a Windows INI
| file? I think it's nice. Rust chose it in keeping with their
| penchant for sanity most of the time.
| ilyt wrote:
| I like YAML for reading and TOML is entirely worse for
| reading (still million times better than JSON tho), and as
| the use cases are mostly read, rarely written, and if written
| they are code-generated (using configuration management),
| YAML fits better.
| red_admiral wrote:
| I would prefer if logically nested blocks could also be
| phyisically nested (and indented), so you can have a full
| tree structure. If you're describing something that can have
| variable levels of nesting (think folders) then it can
| sometimes make the format easier to understand.
| ilyt wrote:
| YAML used from withing statically typed language gets rid of
| most of the problem, but the main one seems to be "well, we
| figured out which stuff was just a bad idea and put it in 1.2,
| except nobody uses it"
|
| > Both have their place though. YAML came out of perl, and both
| are some confluence between awesome and horriffic (although
| yaml wins the horrific crown for sure).
|
| Weirdly enough I'm not getting most of those issues in Perl
| YAML, "norway problem" for example use
| Data::Dumper; use YAML; my $a ="---
| geoblock_regions: - dk - fi - is
| - no - se "; print Dumper(Load($a));
| $VAR1 = { 'geoblock_regions' => [
| 'dk', 'fi',
| 'is', 'no',
| 'se' ]
| };
| jimmaswell wrote:
| > lack of comment syntax no matter how annoying it is that the
| spec is correct about why there are no comments
|
| This completely arbitrary ideological purity has come at the
| expense of countless wasted hours, headaches, and suboptimal
| workarounds like using strings as comments, with zero tangible
| benefit - zero bad things would have ever happened if JSON
| allowed comments. There is nothing correct about it.
| singingfish wrote:
| I dunno about that. I remember how microsoft were back in the
| IE days. They could have used comments for really bad stuff.
| steveBK123 wrote:
| This reminds me of a certain architect at my last shop who
| invented a DSL on top of his Python superapp. He expected all
| projects to go through his superapp. The DSL was configured in
| YAML. The YAML was often so dense he recommended devs use Jinja
| to generate the YAML.
|
| This meant debug was hell, plus it wasn't always clear if what
| you were trying to do was even supported / if not why & what
| needed to be changed. This was because you were now 3 levels of
| abstraction away from the Python code that was actually
| executing.
|
| Every time a dev took on a new project they had to jump on a
| call with architect or right hand man to figure out if what
| they were trying to do was going to be possible.
|
| It escalated into the architect demanding to know a sprint in
| advance any task devs were trying to do, in a review session,
| so he could explain if it was possible or not and try to triage
| in his DSL..
| rollcat wrote:
| I've read about places that do this kind of stuff. Although
| it sounds like pure hell, I'm sure there's always a
| reasonable explanation, an intent. What kinds of problems was
| the org facing that led to the development of this?
| steveBK123 wrote:
| It wasn't really necessary. It also made the core superapp
| a blocker of essentially every user delivery. If it has to
| do everything, then you have to add a lot of features to
| it.
|
| So you have N devs doing jinja/yaml/dsl and N/4 doing core
| superapp underlying dev.
|
| For the first Z features/projects, you inevitably see new
| things that your superapp doesn't support yet, and becomes
| emergent blockers midway through implementation.
|
| Given the ratio of devs, new blockers were being generated
| faster than they could be cleared.
|
| Eventually business side pulled the fire alarm and grabbed
| most of the devs over to use a more common AWS-centric
| service directly and exit the superapp dev use completely.
|
| IE - if you are going to depend on something, would you
| rather it be an AWS service with 10s-100s of dev-years
| behind it, or some internal superapp spun up 3 months ago
| with 2 guys on it? Which is more likely to already support
| what you need?
| ljm wrote:
| This is basically how I feel about working with K8S and
| dredging through a repo full of templated YAML spaghetti.
| What am I looking at now? Helm, Keda, Flux, Argo,
| OperatorHub, GitHub Actions? oh actually this bit is in
| Terraform in another folder, whoops.
|
| You can't actually deploy something unless you can mentally
| untangle it all, it just sits in front of your infra as a
| sort of DevOps Coming of Age ritual, where you look
| whistfully over your shoulder at the old Heroku or Vercel
| account you grew up with. Simpler times.
| duxup wrote:
| That sounds like a layer of insanity that would make me
| consider jobs elsewhere. It sounds entirely unnecessary and
| burdensome , but was it unnecessary?
| steveBK123 wrote:
| It was unnecessary because he was being too clever & having
| a good time, versus ever having delivered real production
| systems in our industry.
|
| It also put devs on a dead end path which they realized
| pretty quickly. Do you want to work for years on this team
| becoming experts in jinja to yaml to in-house DSL you'll
| never use anywhere else? Or do you want to write some
| python? If you can't get "promoted" into the team writing
| the core python engine, then you are obviously a second
| rate.. why stay?
| duxup wrote:
| Even beyond that I would worry about the decision making
| at the top that... they thought that was a good idea and
| kept at it.
|
| At that point a lot of potentially absurd things could be
| in play.
| steveBK123 wrote:
| Less technical management hires a hero who tells them
| everything they want to hear!
|
| "I'm going to deliver the superapp, everything will be
| super centralized & tidy.. small dev team, then all the
| specific implementations will be grunt work by cheap
| devs!"
|
| Throw in some buzzwords and they are sold.
|
| Same audience that always signs the checks for no
| code/low code stuff no one actually wants.
| ilyt wrote:
| >The DSL was configured in YAML. The YAML was often so dense
| he recommended devs use Jinja to generate the YAML.
|
| Did he then went on to design Ansible ? It falls into same
| trap
|
| Only way you should be generating data format using
| language's templating system is <%=
| YAML.dump(@config) %>
|
| Also 9 times out of 10 I wished the app designer just used
| <app language> or <any common embeddable language> (like Lua)
| instead of making any kind of DSL (whether that's just data
| file pretending to be code or micro programming language)
| steveBK123 wrote:
| Yes. I think this is like the uncanny valley of
| development.
|
| It's not no-code UI driven stuff you can put in front of a
| business user.
|
| It's not real coding, which an engineer wants to do.
|
| It's config jockeying, which devs find boring, and is
| generally far more limited. So you end up building out more
| and more complex layers of config to work around the
| limits, including scripts to generate config.. etc.
|
| Seems like what you really want is modular apps that are
| easy to extend in the native programming language(s).
|
| Also maybe I'm just stupid, but 9 out of 10 times, a text
| file or csv/demitted file accomplishes most of what you
| need for pure config that really belongs in config.
| gpderetta wrote:
| Tell me! I'm currently attempting to resist an upcoming
| transition form .csv to yaml...
|
| ... which is used to generate .xml.
| goodoldneon wrote:
| I would quit within a month
| steveBK123 wrote:
| Some did :-D
| singingfish wrote:
| I guess YAML has a place in that it would prevent that kind
| of thing happening in the first place.
|
| YAML is easy to debug (thanks to having comment syntax)
| because it just deserialises into code. Sometimes it
| deserialises into code that compiles on the fly mind you
| which is never a good idea.
|
| On the other hand one time I debugged a really nasty memory
| leak by dumping many megabytes of YAML then running git diff
| against the dumps. That was fun. Of course the client used
| the quick and bad hack rather than the demonstrably correct
| fix (thanks to the dumps) because they were frightened of
| their own code.
| formerly_proven wrote:
| At work someone is trying to introduce a system where a bunch
| of Jinja templates in a repository are used to generate XML
| which can then be used to generate another XML document which
| can then be "executed", resulting in an annotated XML
| document :)
| steveBK123 wrote:
| Wow that's like 2000s & 2020s mutant baby
| formerly_proven wrote:
| And the context variables for the jinja is in part read
| from a YAML file iirc
| red_admiral wrote:
| JSON also borrows (earlier versions of?) JavaScript's "all
| numbers are doubles", leading to this kind of thing:
| https://rachelbythebay.com/w/2021/02/03/bits/
|
| Though, in fairness, I think old Perl did that too. It's super
| convenient until it isn't.
|
| Rachel also doesn't approve of JSON in high-reliability systems
| for other reasons:
| https://rachelbythebay.com/w/2019/07/21/reliability/ and point
| taken, if you're sending data from your service A to your
| service B and neither is a web browser, nor are they written in
| JS, then there's far better formats and you almost need a
| reason not to use protobuf.
| rwmj wrote:
| There was (I think probably still is) a qemu bug with JSON.
| It accepted requests to read guest memory in JSON format,
| with the memory addresses encoded as JSON numbers.
|
| When reading out guest kernel memory (addresses are at the
| top of 64 bit space) these would silently be rounded to the
| nearest whole double. It took me a very long time to
| understand what was going on.
| anonymoushn wrote:
| JSON doesn't specify what numbers are. Integers that take 2MB
| to represent are valid JSON numbers.
|
| Regarding protobuf, the following opinion is obviously
| insane, and if your org is already using protobuf you should
| ignore it: protobuf actually seems pretty bad? It has a bunch
| of vestigial features that people just say not to use. Its
| integer encoding bloats the encoded size and causes
| unnecessary dependency chains in the decoder. I would
| strongly prefer sending simdjson tape between processes and
| storing simdjson tape at rest, but if my coworkers insisted
| on doing something normal, maybe I would look into
| flatbuffers or capnproto.
| scrollaway wrote:
| This is a great post but my understanding is this has nothing
| to do with JSON, which is unopinionated about numbers.
| Rather, with JS's JSON parser.
|
| Python, for example, has several JSON libraries which let you
| swap out the numeric parser so it yields Decimal objects all
| the time. It's overkill for most use cases, but essential if
| you're working with REST APIs in Fintech.
| svieira wrote:
| Actually JSON doesn't specify what numbers are - it would be
| perfectly licit for a JSON parser to transparently use a real
| numerical tower, allowing perfect representations of any non-
| repeating decimal fractional number (since it has to be
| represented as a dotted-fraction and there's no support for a
| vinculum (aka U+0305 / COMBINING OVERLINE / * / 3.21) there's
| no way to represent non-repeating fractions if there is not a
| non-repeating representation in base 10). A few JSON parsers
| even do this. That said, if you don't control the both sides
| sending something that won't be handled by the lowest-common-
| denominator (browser JSON parsers / JS numbers) is asking for
| trouble.
| int_19h wrote:
| The short story is that if you want real foolproof
| interoperability, always represent numbers as strings in
| JSON.
| themulticaster wrote:
| Interestingly, your example renders incorrectly for me
| (Firefox/W10). The overline is placed between 1 and ).
| IshKebab wrote:
| JSON5 is the answer. I have yet to find a better option.
| garfieldnate wrote:
| Is this the same Ingy that made Test::Base? It's the best data-
| driven testing framework I've ever used, and I've missed it
| often while working with other languages. The follow-up
| polyglot framework just didn't cut it for me.
| singingfish wrote:
| I would be really surprised (and slightly worried) if there
| was more than one Ingy :)
| djrobstep wrote:
| Amazing that developers still spend so much time getting bogged
| down in encoding issues in 2023. Why can't we just edit some data
| structures directly and send send them around seamlessly? It's a
| deeply dysfunctional state of affairs.
| ablob wrote:
| For that, you'd need a set of commonly accepted data
| structures.
|
| Alternatively, you can describe and send arbitrary data
| structures. Although that begs the question of how to process
| arbitrary data structures. (JSON and XML aren't arbitrary, for
| that matter)
| nrvn wrote:
| it is an old song.
|
| JSON (as per spec) is a lightweight data-interchange format.
|
| YAML is a "human-friendly data serialization language for all
| programming languages" and by mere coincidence (it does not look
| to be by design until version 1.2.2 [0]) JSON is its subset and
| not vice versa!
|
| And YAML is human-friendly in two ways:
|
| - it is notoriously "human" in interpreting input, i.e. ignoring
| the context or misinterpreting others' words;
|
| - it helps read serialized data via indentation-based syntax.
|
| The biggest caveat is for whatever reason everyone ignores the
| key highlights of both JSON and YAML that are "data-interchange
| format" and "data serialization language". If I were to interpret
| these two terms I would rather say they both present great means
| for heterogenous programs to exchange data with YAML being a
| fancier brother better suited for human eyes. They were not meant
| to be used for configuration purposes or what not. But people
| started using them exactly for this purpose instead of creating a
| configuration format/language that would render into program-
| readable JSON(if needed).
|
| CUE and Dhall are perhaps the first two attempts to create
| specifically a configuration language with the ability to
| serialize in yaml/json/etc.
|
| I very much hope using YAML, JSON and even TOML for writing
| configurations will become an anti-pattern soon and will be
| replaced by something more suited for this role.
|
| [0] https://yaml.org/spec/1.2.2/#12-yaml-history
| jackcviers3 wrote:
| This is why configurations as code should be written in actual
| programming languages. Dhall, Nix, and Pulumi are all solutions
| geared towards configuration but written with fully capable,
| user-extensible, programming languages.
| amarshall wrote:
| > The key on is common in the wild because it is used in GitHub
| Actions. I would be really curious to know whether GitHub
| Actions' parser looks at "on" or true under the hood.
|
| I can say, at least, that using both on: and 'on' work in GHA. I
| usually write the latter, makes it simpler to do ad-hoc
| YAML.parse or whatever on the workflow files.
| pnt12 wrote:
| An unquoted version 3.10 in yaml becomes 3.1. Fun stuff!
| TheFlyingFish wrote:
| To me, the #1 reason to use YAML rather than anything else is if
| you need to embed complex multi-line strings in your document.
| This is a practical necessity in CI systems: you always end up
| with at least a little bit of shell scripting to glue things
| together, and embedding shell scripts into anything other than
| YAML is an absolute nightmare.
|
| Yes, there are too many options for expressing multi-line strings
| (In the vast majority of cases, I really don't care whether it
| has a trailing newline or not). Yes, there are weird rules that
| don't make sense, like how in "folding" mode a newline followed
| by two spaces more than the current indentation level _doesn 't_
| get folded. But it's still the best option for CI, which is why
| Github actually ditched HCL in favor of YAML after a while:
| https://news.ycombinator.com/item?id=20647691
| dmos62 wrote:
| I was looking for a human-friendly format to store data for a
| staticly generated website. Stuff like copy, lists, addresses,
| contact information (so that the rendering logic is kept
| separate).
|
| Landed on NestedText, and apart from it having few
| implementations, I loved it! https://nestedtext.org/en/stable/
|
| I did feel like I missed a more popular alternative, because I
| can't be the only one to want a human-friendly data format.
| loa_in_ wrote:
| There's a lot of alternatives out there. One of the most complete
| (in terms of specification) and very promising is concise
| encoding (https://concise-encoding.org/) which focuses on the
| duality of editable human compatible representation (as text) and
| efficient binary encoding/decoding. The biggest issue right now
| is lack of actual implementations. The project is open source
| (I'm not affiliated with it).
| elgertam wrote:
| Many commends are (correctly IMO) pointing to JSON as superior to
| YAML given YAML's problems, but these ignore JSON's problems due
| to its lack of semantics. When moving JSON between systems, the
| JSON parser makes certain assumptions about the JSON document,
| such as whether the numbers represent integers or single-
| precision or double-precision floats. This is certainly less
| obnoxious than "no" being treated as a bool, but it is a problem.
|
| A friend of mine created Preserves[0] in part to resolve this
| problem. I'd encourage everyone to check it out. To be fair, he's
| also quite pro-Dhall and Nix.
|
| [0] https://news.ycombinator.com/item?id=34354614
| theamk wrote:
| The simple answer to that is to store anything that may not fit
| in double without loss as a string. (I don't know of JSON
| decoders which use floats)
|
| After all, one of the biggest upsides of JSON is its wide
| language/tool compatibility. And that means intentionally
| limiting the format to the most basic subset of types that all
| languages have.
| rsynnott wrote:
| But, of course, _all_ yaml documents are from hell.
| smitty1e wrote:
| I like to write Python to manage JSON.
|
| Keeping the data and the logic orthogonal (these effin' Fn::'s
| make my eyes bleed) is a win.
| vsajip wrote:
| The CFG configuration format is a text format for configuration
| files which is similar to, and a superset of, the JSON format.
| It's not new - it dates from well before its first announcement
| in 2008 - and has the following aims:
|
| * Allow a hierarchical configuration scheme with support for key-
| value mappings, lists and basic types such as strings, ints,
| floats, Booleans and date/times.
|
| * Support cross-references between one part of the configuration
| and another.
|
| * Provide a string interpolation facility to easily build up
| configuration values from other configuration values.
|
| * Provide the ability to compose configurations (using include
| and merge facilities).
|
| * Provide the ability to access real application objects safely,
| where supported by the platform.
|
| * Be completely declarative.
|
| It's similar to newer formats such as JSON5, HJSON, HOCON and
| similar but offers a number of features [0] which they don't, as
| indicated by the above list. It's not intended to occupy the
| niche where you find things like Cue, Jsonnet, Dhall and similar.
|
| It was just never especially publicised when first implemented
| for use in Python projects, but it now also has implementations
| for the JVM, .NET, Go, Rust, D, JavaScript [1], Ruby and Elixir
| (all BSD-3-Clause licensed) and it would be great to get feedback
| on the project from the HN community.
|
| [0]: https://docs.red-dove.com/cfg/intro.html - description of
| features and comparison with other similar systems
|
| [1]: https://docs.red-dove.com/cfg/playground.html - uses the JS
| implementation to create an interactive playground
| Kaotique wrote:
| While it is indeed total insanity it could easily be avoided by
| putting quotes around all string values.
| FpUser wrote:
| This is why my own configs are either ini where I know 100% that
| it would never become complex and take more than few lines or XML
| for everything else.
| haileys wrote:
| > Templating yaml is a terrible, terrible idea
|
| I've had a good time using ytt: https://carvel.dev/ytt/. It
| implements language-aware templating, which is IMO the only
| reasonable way to do it.
| kevincox wrote:
| Language aware is definitely important. But honestly I've
| usually found it easier to use a "real programming language" to
| generate the structures. You can read and write as normal. Then
| you can just serialize them to YAML at the end.
|
| Helm is so bad. It has a `| quote` function that is almost
| always wrong. People use it to escape strings into YAML strings
| but it just adds quotes around the values. It doesn't actually
| escape to YAML. I had a surprising hard time convincing people
| that they should be using `| toJson` and ideally be doing it
| for every interpolation.
|
| But really working with text is just an inconvenience here. You
| can imagine how much better everything would work if your base
| template was `console.log(JSON.stringify({ /* your config here
| */ }))`
| marc_omorain wrote:
| At CircleCI, we have had to handle many strange YAML documents.
| Two that come to mind:
|
| A recursive document: key: &anchor -
| *anchor
|
| The `~` character represents null: ~:
| - ~
|
| This parses as a map of {null: [null]}
| Hendrikto wrote:
| What were the authors trying to accomplish with these strange
| documents?
| tiagod wrote:
| Break their parser, probably
| marc_omorain wrote:
| The recursive anchors seemed to be user-error. Either
| copy+paste bugs, or an anchors places in the wrong position.
|
| The null character would appear when people try to write a
| string path, and use the the `~` character without quotes.
| jrmg wrote:
| I do think YAML is overly complex - but there is some hyperbole
| in this document.
|
| - Many of these complaints are about YAML 1.1.
|
| - YAML 1.2 was released _14 years ago_.
|
| - The author makes some allusions to 1.2.2, and it requiring a
| team of people 14 years to make, but, from the yaml.com
| announcement they link to: "No normative changes from the 1.2.1
| revision. YAML 1.2 has not been changed"
|
| I guess my first two comments are undercut by PyYAML using YAML
| 1.1 (Really?! Python's had 24 years of the Norway problem?!)
| IshKebab wrote:
| The article mentions the fact that YAML 1.2 is really old and
| the fact that it doesn't matter because YAML 1.1 is still the
| most commonly supported version, and the fact that it's
| arguably even worse because YAML 1.2 gives different parsing
| results to YAML 1.1!
|
| I highly recommend reading the article - it's very good.
| wyuenho wrote:
| Yes, the problem is PyYAML.
| harshreality wrote:
| I don't get all the YAML hate. The article even mentions
| solutions, which are all better than adopting some nonstandard
| json variant or toml.
|
| - for whatever language you're using, be aware of which YAML
| version its YAML library supports and its defaults, and how to
| safe load yaml in that language.
|
| - defensively quote strings, particularly if you're on a language
| with an antiquated yaml library that defaults to yaml 1.1.
|
| - defensively use true/false only, and proactively convert any
| other booleans in your codebase to true/false.
|
| - Depending on your language, you can avoid any custom data types
| for any externally-distributed applications to mitigate the
| risks, even when it might make things more convenient. Use safe
| loading (most languages with yaml libs support it) to avoid
| loading any. The other major YAML alternatives that YAML haters
| recommend won't have custom data types either.
|
| Features like references and folding semantics are very
| convenient, and you don't even have to use them. Basic yaml
| enjoys better readability than json. toml is only fine if you
| don't have much nested data.
|
| The author's note that there are json _variants_ that help with
| some json failings makes no sense. If you 're going to adopt some
| non-standard json variant, why not just adopt yaml 1.2, make sure
| your language has a yaml lib that supports it, and use that? At
| least yaml 1.2 is standardized. It's not their fault if python's
| libyaml only supports yaml 1.1. It looks like pyyaml is
| essentially in maintenance mode and ruamel.yaml is what everyone
| should be using? Unfortunately nobody's gotten around to
| implementing safe loading natively, but that's a python problem,
| not a yaml 1.2 problem, and ruamel.yaml supports pure-py safe
| loading that's compliant with 1.2 (no integer-interpretation
| gotchas), which is fine in most cases where yaml is only loaded
| occasionally, i.e. at start-up, and performance isn't critical.
|
| Obviously YAML has historical problems, but what's better? Using
| another flawed or even more limited data format, inventing your
| own which will begin with zero adoption, or simply ensuring your
| environment/app uses yaml 1.2 and best practices?
| speleding wrote:
| I agree with most of your suggestions, but I wouldn't use
| "safe_load" unless the file has untrusted input (in which case,
| you could argue, you shouldn't be using YAML at all).
|
| If you use safe_load then you'll get an exception if you use
| dates or timestamps, even if it's an unquoted string that
| happens to look like a date (but you can get around that last
| one by quoting).
| chii wrote:
| Having to watch out for footguns (by being defensive as you
| described) means you're just having to learn these rules. What
| benefit is YAML giving you then?
|
| Why not use JSON? Conversion of types from YAML to the
| language's native types is a poor feature to get in return for
| footguns.
|
| If you _need_ complex types - like a in RPC serialization -
| then i would imagine using XML is better, and forget about
| making it human-readable. Create a client/testbed instead.
|
| For simple things, JSON fits really well imho.
| omnicognate wrote:
| > Obviously YAML has historical problems, but what's better?
|
| toml
| DougBTX wrote:
| One of the major use-cases for YAML is as a host language for
| DSLs, essentially basic programming languages for specific
| use-cases. TOML isn't as good there, as these languages often
| need deep nesting of data structures. GitHub Actions is the
| obvious example.
| omnicognate wrote:
| Yeah, toml isn't great for that. It's greatly preferable to
| yaml for human-readable config. Json is better as a
| serialisation format.
|
| DSLs are bit vexed, not least because they're often
| horrible in themselves - cruft-accumulating hacks with no
| tooling, blurring the distinction between code and config
| often as a workaround for unwieldy dev processes. That's
| certainly not true of all DSLs, though (can't speak for
| Github Actions) and there is a need for a simple common
| base syntax/parser for them. I don't think yaml is a good
| solution.
|
| Being a cantankerous old git, I tend to think the problem
| of a common base syntax/parser for DSLs was solved 70 years
| ago: s-expressions/lisp.
| DougBTX wrote:
| Yes, agreed. I've found TOML for config to be quite nice.
|
| This is my go-to article for discussions about generic
| languages vs DSLs:
| http://mikehadlow.blogspot.com/2012/05/configuration-
| complex...
|
| I'm very iffy on DSLs in general. One that I'm using at
| the moment has foreach loops but no if statements! And...
| all the inputs must be specified in a YAML file!
| int_19h wrote:
| Quite frankly, all YAML DSLs in the wild are terrible, so
| if that's a major use-case, I'd say it was a failure.
| Sharlin wrote:
| Why are the solutions better than adopting toml? You don't
| provide any justification for that claim.
| harshreality wrote:
| I did. Extensively nested data structures.
| [deleted]
| throw_a_grenade wrote:
| The amount of "defensively [do something]" and "best practices"
| is another way of saying "this format is so bad you need to
| thread carefully or else we'll blame it on you".
|
| Good format should not require defensive writing, and also
| "best practice" is a bit of misunderstanding: it's best
| practice to avoid such a bad format, not attempt to screw
| people into it and then blame them for not following some
| arbitrary set of rules.
| jwmoz wrote:
| This explains a lot! I just know my docker compose files were
| always a pain and unintuitive to write.
| miohtama wrote:
| What are the practical modern alternatives for YAML use cased?
| TOML, others?
| progx wrote:
| TOML is good for simple plain structures. As soon as you have
| more nested structures, you can also use json.
| ainar-g wrote:
| People often forget about the environment, and I feel like a
| lot of the simpler services and applications could be
| configured through it easily.
|
| For anything that requires more complex structures, JSON5
| (https://json5.org/) has been gaining traction. It's basically
| JSON that allows a couple of more things, including comments,
| trailing commas, and even unquoted keys, if you're into that.
| rr888 wrote:
| This for this it indeed looks very much better than json or
| yaml.
| RedNifre wrote:
| edn
|
| Extensible Data Notation, it's to Clojure what json is to
| Javascript.
| CipherThrowaway wrote:
| For simple use cases like key-value style config you can use
| JSON or TOML without too many issues.
|
| For IaC the alternative is general purpose programming
| languages using declarative infrastructure APIs. E.g. TS, Go,
| Python with CDK, CDKTF, Pulumi. For CI are stuck with YAML for
| now.
| brazzy wrote:
| Read the article?
| jgalt212 wrote:
| Someone needs to write
|
| YAML: The Good Parts
|
| If you're strict about how you use it, it's more than workable.
| And, yes, the Norway problem is insane.
| polyrand wrote:
| I wonder why generated JSON is not used more widely, something
| like jsonnet[0], although I've never used it.
|
| Most systems come with Python 3 installed, which supports working
| with JSON[1] without installing any dependencies. So, I'm
| wondering if we could use Python to generate JSON data that is
| then consumed by other tools. This would fix the lack of comments
| and trailing commas in JSON, you get functions and other
| abstractions from Python, and you don't need to install any
| third-party tools (vs. something like jsonnet).
|
| [0]: https://jsonnet.org/
|
| [1]: https://docs.python.org/3/library/json.html
| reilly3000 wrote:
| I'm at a crossroads and evaluating whether to introduce jsonnet
| to my org. We are doing a massive conversion of legacy yaml
| manifests from a proprietary vendor format, and there are not a
| lot of chances like this where you can make such a change at
| scale. Any experience reports or advice on jsonnet would be
| appreciated.
| Duologic wrote:
| At Grafana Labs we're using jsonnet at scale, while being a
| powerful functional language it is also excellent for
| rendering JSON/YAML config. We have developed Tanka[0] to
| work with Kubernetes, for other purposes I can recommend this
| course[1] (authored by me).
|
| [0] https://tanka.dev/
|
| [1] https://jsonnet-libs.github.io/jsonnet-training-course/
| anonumous wrote:
| [flagged]
| pprotas wrote:
| Did you create this account just to place an insulting comment?
| Be kind
| anonumous wrote:
| you should be kind to idiots? then how will they learn that
| they are idiots?
| unsafecast wrote:
| Indeed, we're being too kind to you.
| chrisshroba wrote:
| Call me naive or quote xkcd, but whenever I read about the
| shortcomings of YAML, I daydream about a new standard that's
| mostly a subset of YAML, that looks like this:
| rules: - "All strings must be quoted and are parsed
| identically to JSON strings" - "true and false are the
| only boolean values" - """multiline strings are
| allowed""" - """| there can even be special
| multiline string types to indicate things like "dedent
| every line and trim the beginning and end lines if
| empty """ # Comments are allowed! -
| Number syntax just like JSON better_than_yaml:
| true "a number with multiword key": 1e9 nesting:
| present: true difficult: false
|
| It has the ease of coding of yaml, maps 1:1 with json (so can
| easily be re-serialized to json or yaml if needed), is familiar
| to folks who know yaml, and supports comments. And it eliminates
| ambiguous parses and gotchas, references, tags/directives, and
| other complexities.
| PurpleRamen wrote:
| YAML seems like a good that gets harmed by feature-creep. All I
| want when using YAML is a simple human-friendly text-format for
| slightly structured data. But all those special features and edge
| cases...as if I even have the time to learn about them. Values
| should be only text. Outside of the handful clear specified cases
| of numbers and maybe true/false/none, they should not have any
| other interpretation on the parser.
|
| Isn't there something like a simple YAML-definition and some
| YAMl-schema for the powerusers?
| singularity2001 wrote:
| Wouldn't it be easy to make those features optional and
| switched off by default in "YAML--"?
| shp0ngle wrote:
| YAML has come from Perl world.
|
| Of course it has feature creep. It's a Perl thing.
| kitd wrote:
| He mentions it at the end (the 'YAML subset' alternative), but
| the edge cases only cause issues if your keys and values begin
| with special characters but are intended to be simple text. I
| tend to defensively quote stuff (probably much more than I have
| to), and I rarely find these issues. I have other issues with
| YAML but not this.
| ilyt wrote:
| > Values should be only text. Outside of the handful clear
| specified cases of numbers and maybe true/false/none, they
| should not have any other interpretation on the parser.
|
| I think simpler way would be just forcing quoting on text. That
| fixes the boolean and the different base numerics problem
| tgv wrote:
| TLDR: unexpected conversions to other types in YAML 1.1.
|
| A larger JSON document is hell to write and read, though.
| [deleted]
| ancieque wrote:
| JSON is arguably much worse for usage in Git. Have fun matching
| braces in the default diff tools out in the wild.
|
| YAML is the much better alternative with similar semantics to
| JSON. The numeric issues still hold true though.
| mixmastamyk wrote:
| Strict yaml fixes this, I recommend it. Before that I disabled as
| many of yaml's features in the library I was using. Look for
| "safe parsing."
| rochak wrote:
| YAML is the worst format I've had the displeasure of working
| with. Given how ubiquitous it is, I don't see it going away soon,
| but I am still hopeful.
| [deleted]
| vbezhenar wrote:
| The way to solve YAML issues is to invent some widely used yaml
| pretty printer and shout at everyone not using it.
|
| There's `yq -P` but it does not implement proper normalization
| for plenty of mentioned issues which is a shame, really.
|
| As I see it, everyone can just start with JSON (as JSON is a
| subset of YAML) and pretty-print it and use the result whenever
| it's supposed to be used.
|
| In other words use safe subset of YAML. May be it'll be
| formalized some day. Yamlite.
| speed_spread wrote:
| That safe subset exists and is implemented in a number of
| languages. It is called strict-yaml:
| https://hitchdev.com/strictyaml/
| vbezhenar wrote:
| They don't use quotes, so not safe.
| kergonath wrote:
| In which languages is it implemented? I have seen a
| supposedly equivalent Rust library, but could not find
| anything else. Is there any C library or something like that?
| Unfortunately, these are the easiest to use from a lot of
| languages because of how easy they are to wrap.
|
| (I am genuinely interested in any useful library better than
| libyaml).
| synergy20 wrote:
| so the suggestion is: use toml for configuration, otherwise just
| stay with json?
|
| there is jsonc which is json+comments but I have not used.
| nathell wrote:
| EDN's my favourite config format.
|
| + Simple (like JSON)
|
| + Has comments
|
| + Extensible
|
| + Built-in sets, UUIDs & timestamps
|
| + Well-suited to configuration (cf. Aero)
|
| + Lispy (well, Clojurey)
|
| - Lispy (well, Clojurey)
|
| - Handwavey specs
|
| - Not widely supported
| npteljes wrote:
| Would you please link an example, a specification, or
| something? I think I found something on GitHub, but it was not
| straightforward.
| nathell wrote:
| Sure!
|
| Spec: https://github.com/edn-format/edn
|
| Example (linter config): https://github.com/clj-kondo/clj-
| kondo/blob/634294183a0aa2ca...
| sigmonsays wrote:
| i want *any* configuration format that can rewrite itself(loaded
| and written to disk again) with two properties we've seem to have
| lost when we switched away from xml.
|
| 1. keep ordering of document 2. keep comments of document
|
| Does anything like that exist?
| wg0 wrote:
| YAML with templating becomes troublesome even more so when these
| are Django tmeplates. That's why I do not prefer ansible for
| automation. It becomes so much cumbersome at certain point that
| it is more valuable to jump into a real programming language
| instead of wresteling with both YAML, Django templating and
| python.
|
| That's where I am enthusiastic proponent of pyinfra.
| rollcat wrote:
| Ansible and YAML were my primary (de)motivators to create Judo
| (https://github.com/rollcat/judo). This combo is extremely
| frustrating: for every line in a (hypothetical) shell script
| that would do one thing, I needed 3-5 (sometimes many more)
| lines of YAML. Most people on the team who were just getting
| started with Ansible, would often do half of their work just
| shelling out. I would usually push to do things "the Ansible
| way", but even I had to acknowledge the mental overhead of
| translating back & forth. I think what finally pushed me over
| the edge was when we started venturing into compose & k8s, and
| had to mix & juggle YAML+Jinja in two entirely different
| contexts, each with its own quirks, bugs, gotchas and brain
| damage.
|
| I figured I just need a layer of glue to run shell scripts
| across a bunch of remote hosts (hence Judo), and otherwise
| resort to other tooling (like Terraform, AWS CLI, k8s CLI, etc)
| for problems that don't map to SSH.
| sigmonsays wrote:
| why would you use this over pssh or dsh
| (https://www.garbled.net/clusterit.html ) ?
|
| It looks to do exactly the same thing, only slightly less
| efficiently.
| rollcat wrote:
| I didn't know these existed, until I read your reply.
| Thanks - looks interesting, I will have to investigate
| further.
|
| First, I've done some research into alternatives, but my
| primary motivation was to prove you can replace Ansible
| with a weekend hack that closely follows the UNIX
| philosophy. Here I am 6 years later still using it everyday
| in production, so I guess I was right.
|
| Second, dsh looks a bit antiquated - defaulting to rsh?
| Granted I have very little experience with commercial
| Unices (which seem to be the primary use case there),
| but... rsh?
| sandwell wrote:
| Agree with the final part of this article that "programmable"
| configuration languages like Nix and Dhall are the way forward.
|
| I've spent a lot of time writing YAML for Ansible,
| Cloudformation, k8s, Helm, etc. Some of the issues this article
| mentions are pitfalls but once you get a bit of experience with
| it, you know what to look out for.
|
| I've also spent and a lot of time writing Nix expressions, which
| is much more "joyful" IMO. Seemingly simple features like being
| able to create a function to reuse the same parameterized
| configuration makes life much easier.
|
| Add in a layer of type safety and some integration with the
| 'parent' app (think replacements for CloudFormation's !GetAtt or
| Ansible's handlers), the ability to perform basic unit tests,
| then configuration becomes more like writing code which I
| consider a good thing.
| dgan wrote:
| If you only need something for internal use, nothing prevents you
| from using s-expressions, and interpret atoms (number ->
| uint16_t, dates in "yyyy-mm-dd") whatever your application feels
| like... With this lil' trick all parsing problems are gone.
| Inferior Format Users will be jealous, and Inferior-Format-
| Parser-Writers will hate you
| werdnapk wrote:
| Almost all of the issues mentioned would be solved by using
| quoted strings.
| PaulHoule wrote:
| About Sometimes an application will start out
| with a need for just a configuration format, but over time
| you end up with many many similar stanzas, and you would
| like to share parts between them, and abstract some repetition
| away. This tends to happen in for example Kubernetes and
| GitHub Actions. When the configuration language does not
| support abstraction, people often reach for templating,
| which is a bad idea for the reasons explained earlier.
| Proper programming languages, possibly domain-specific ones, are
| a better fit. Some of my favorites are Nix and Python
|
| Templating is fine with YAML if you don't do it with strings but
| with a system that turns one YAML document into another YAML
| document, that is, do your substitutions in the YAML universe.
|
| The "code up your configuration in your favorite programming
| language" can really work but it frequently becomes a point of
| frustration itself. As much as I want to like YAML it has a
| number of intrinsic problems like the Norway problem... But I do
| think people are confusing the accidental and essential
| complexity and because of that they flinch and reach for non-
| solutions to the problems of configuration rather than really
| thinking it through.
| politelemon wrote:
| Isn't TOML just the .INI files of yore? At least it looks very
| similar.
| CodesInChaos wrote:
| The feature set of toml is similar to json, but with ini like
| syntax.
|
| Most significantly ini-files have a fixed two level structure
| (section -> key -> value) while toml supports arbitrary nested
| arrays and dictionaries.
| Freak_NL wrote:
| And don't forget one of the more understated benefits of TOML
| over INI:
|
| > A TOML file must be a valid UTF-8 encoded Unicode document.
| the-alchemist wrote:
| Since everyone seems to throwing their favorite format into the
| ring (), I will too: EDN [0]
|
| * no enclosing element (i.e., can be streamed)
|
| * maps, lists, even sets
|
| * tags (like "Person". UUIDs and timestamps are built-in tags)
|
| * floating point numbers
|
| * integers
|
| * comments
|
| * UTF-8
|
| * true booleans
|
| * no need to worry about too many or too few commas in the right
| or wrong place
|
| Implementations in almost every language under the sun [1].
|
| The format is simple enough that it's easy to implement, verify,
| and test. No strange string interpretation craziness (see YAML
| and "Norway problem"), no ambiguity between FP and integers (see
| JSON), comments. And if your editor has rainbow parenthesis
| support, reading is actually a pleasant experience.
|
| [0]: https://github.com/edn-format/edn
|
| [1]: https://github.com/edn-format/edn/wiki/Implementations
| [deleted]
| jwilk wrote:
| > In Python, you can avoid this pitfall by using yaml.safe_load
| instead of yaml.load.
|
| It's harder to make this mistake in Python these days.
| yaml.load(input) has been depreacted since PyYAML 5.1
| (https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-...)
| and finally stopped working in 6.0.
| groby_b wrote:
| Welcome to dynamic typing hell.
|
| As long as you ask your configuration language to infer the data
| type from a piece of text, these are the kinds of edge cases you
| get to experience. (They vary by language. Less types means less
| mistakes, and so does less expressive power - but without some
| form of type hinting, that's where you are)
|
| Personally, I like YAML for expressive simplicity at the start of
| a project, and then if it becomes an actual load bearing piece of
| software (vs a haphazard one off, or an experiment), let's add
| some schema validation & typing.
| d4rkp4ttern wrote:
| For managing configs of ML experiments (where each experiment can
| override a base config, and "variant" configs can further
| override the experiment config, etc), Hydra + Yaml + OmegaConf is
| really nice.
|
| https://hydra.cc/
|
| I admit I don't fully understand all the advanced options in
| Hydra, but the basic usage is already very useful. A nice guide
| is here:
|
| https://florianwilhelm.info/2022/01/configuration_via_yaml_a...
| rqtwteye wrote:
| At least YAML has comments, which JSON doesn't. I know people
| will say "JSON is not meant as config format" but in reality it
| is used as such.
|
| In general I don't understand why there is so much churn and
| focus on config file formats. They can be annoying but in the end
| the real problem is that we have complex configurations so a
| different format is not going to help much.
|
| Personally I prefer YAML over XML or JSON for config because I
| don't have to worry about closing brackets or braces. Maybe we
| need a good editor to make editing them easier.
| lucasfcosta wrote:
| I honestly don't understand why people still write YAML
| considering there are alternatives like TOML available.
|
| I think attributing semantic value to indentation is a terrible
| idea. It gets even worse when there is so much visual ambiguity
| when it comes to defining values and escaping certain character
| sequences.
| notinfuriated wrote:
| To me, the yaml document from hell is a series of yaml documents
| I helped create that replaced a bunch of database config and
| hard-coded constants. It helped 'unlock' the business to create
| newer forms of products much more quickly and correctly, but it
| never got moved to an actual database. So at this point, they
| have some yaml documents that, if expanded to include all
| references, end up becoming this massive fucking tangled mess.
|
| IIRC we never moved it to a database partly because we did not
| have time to solve for how we actually wanted references to work.
|
| The yaml document and code that reads it still work as a way of
| speeding up the business compared to how things used to be. But
| that code takes a long time to parse, has probably caused a
| decent level of confusion for developers who have to mess with it
| who aren't on that team, and has absolutely taken a lot of clock
| time being parsed for validation checks in unit tests.
|
| Still not sure I would have solved that problem differently,
| aside from perhaps moving the thing to a database immediately
| after the first phase of working on it.
| d12bb wrote:
| When I first discovered YAML, I loved it's apparent simplicity.
| Nowadays, I sigh every time I have to use it and am very happy
| with the Rust world having mostly adopted TOML.
| kabdib wrote:
| I will never willingly use YAML on a project ever again. It's a
| bloody disaster. I would actually prefer XML over YAML.
| donatj wrote:
| I remember the first time I played with YAML ten-ish years ago. I
| experimented with converting our configuration file from INI to
| YAML. It came with an actual noticeable increase to our app
| startup time in the tens of milliseconds.
|
| The YAML evangelist in my company when asked told me I should
| cache the results... of reading our config file... from the local
| filesystem. What do I store that cache as? JSON? Just cut out the
| middle man. Left a really bad taste in my mouth at the time I
| have yet to get out.
| ainar-g wrote:
| "YAML evangelist" sounds terrifying, and I have never met one
| in real life. Most people who used YAML basically resorted to
| the old argument of "Well JSON doesn't have comments, so YAML
| it is".
| sampo wrote:
| Is there a tool that parses yaml and outputs json, so you could
| check if the yaml you wrote parses as you intent?
| chriswarbo wrote:
| I use https://kislyuk.github.io/yq (it also includes and 'xq'
| command, which converts XML to JSON)
| ancieque wrote:
| I guess this can easily be scripted with
| https://pypi.org/project/yq/ Internally this seems to be a
| wrapper around jq :)
| shp0ngle wrote:
| What would be good is "strict yaml", that is, yaml with all those
| weird cases strictly disallowed?
|
| Which would make it backwards compatible, as the cases would just
| make it invalid "strict yaml"?
|
| And I'm thinking it might already exist? I'm surely not the first
| one to think of that
| dbfa wrote:
| It exists, and it is called "JSON" :)
| bambax wrote:
| I never could understand the hate XML got, but I'm having a bit
| of schadenfreude seeing what people suffer through with its
| replacements.
|
| An XML document with a well-thought-out domain-specific DTD would
| solve all these problems; instead, we have something where _no_
| sometimes means _false_ (but not always) and _22:22_ sometimes
| means _1342_ (but not always!)... because... why not?!?
|
| All that horrible mess, it seems, because people didn't like to
| have to close tags.
| ppseafield wrote:
| Just like bad/fragmented YAML libraries now, there were plenty
| of bad XML libraries and implementations. I didn't think XML
| was too bad until I had to write a SOAP request where the
| endpoint would throw an error if the arguments (in their own
| tags, mind you) weren't in a specific order. The endpoint gave
| me no hints.
|
| Additionally we had to update WSDLs for these services, but the
| service generating the WSDL used features our server's SOAP
| library didn't support, so someone had to manually transform
| the XML and via lengthy trial and error to get it to work.
| Jenk wrote:
| Hard disagree.
|
| XML is great for systems to read and write, but utterly
| abhorrent for humans. It's not just having to close tags. It's
| content vs attribute confusion, namespacing noise (which are
| also muddled with attributes), and there's a squint factor
| incurred by the density of information.
| Ygg2 wrote:
| On the other hand XML document with good schema is pleasure
| to write in any decent IDE.
|
| It just autocompletes itself. I know JSON schema exist, but
| it never managed to just work that well.
| vincnetas wrote:
| i think that propper schema and autocompleate would save the
| day. even i would say make things easier. talking about XML
| piskov wrote:
| I've been using this for (schema + autocomplete) for at
| least 18(!) years.
|
| Moreover I even got comments on xml attributes and
| elements, warnings if required element is missing or
| required count is not met.
|
| You even got visual hierarchical collapsable UI for
| viewing/editing that XSD schema.
|
| I repeat: 18 years of autocomplete, comments on hover, and
| validation for typos/count/order for XML in Visual Studio.
| Jenk wrote:
| 20-something years of web/app.config is the primary
| reason why I hate XML. Very, very closely followed by
| Java ("a DSL for turning XML into stacktraces") and its
| over complicated ecosystem of xml configurations.
| thrwawayjan1223 wrote:
| I have worked with Java for over two decades, and your
| experience does not match mine.
|
| At $work there are only two XML files in each project:
|
| - logback.xml: configuring logging which is usually only
| done once or twice and is the same for each project.
|
| - pom.xml: Maven config which is mostly autogenerated by
| IntelliJ when creating a new project. I only add a few
| lines for new dependencies.
|
| In rare cases I may use some plugins to do specific build
| operations, but the actual XML is simple. Most of the
| time is spent on understanding the plugin itself, and not
| "fighting" XML. Moving this to YAML has no effect on my
| productivity
|
| All other configuration is either done using YAML,
| properties or configuration in code (Spring Boot).
|
| I see Spring still has options to use XML configurations,
| but I don't see any reasons why anyone would do that in
| 2023. The "new" standard in Spring is configuration by
| code, which I have done for the last 8 years.
|
| The ecosystem for Java today can not be compared to what
| it was 10 years ago. If editing XML configurations is
| still a problem at your workplace, find a new job :-)
| Either you work with a legacy system that nobody wants to
| update to the new standards, or you have architects who
| are stuck.
| bambax wrote:
| Namespacing is the opposite of noise: it lets different
| domains coexist with no possible confusion about which is
| which.
|
| But experience has taught me that trying to explain why XML
| is great and simple to read and write for humans, to someone
| who thinks differently, is useless. And so, I won't.
|
| Yet, please accept that some people, like me, really liked it
| and didn't mind the little quirks in light of all it offered.
| wartijn_ wrote:
| It makes sense that explaining why xml is great doesn't
| work. Maybe you can convince someone who's never worked
| with xml. But if someone has used xml and thinks it's a bad
| format based on those experiences, you'll need to bring
| _really_ good arguments to let them think it's a good
| format anyway.
|
| Just ask yourself: could a comment here convince you that
| xml is a bad format? If not, why would the opposite work?
| lostmsu wrote:
| Well yes, if it described a format as pleasant for humans
| as YAML, while not being ambigious and error-prone.
| [deleted]
| deafpolygon wrote:
| I disliked it intensely at first, but over time .. I now
| see the merit in it. I like XML, especially compared to
| everything else (that came after). It was possible to
| clearly define things.
| Jenk wrote:
| On the same system? I can see that. One thing I've
| lamented about XML for years is that it's a pain when
| moving to different systems that have their own
| namespaces/DTDs.
| scott_w wrote:
| > All that horrible mess, it seems, because people didn't like
| to have to close tags.
|
| The issue isn't that it's bad per-se. The issue is that it's
| cumbersome to write. When you want to setup and tweak a tool,
| it gets annoying very quickly having to deal with little errors
| because you misspelled a closing tag. Maybe you want to test
| enabling a setting? Typing out 4 characters is so much less
| friction than the 20 or more (including the < > : / characters)
| you'd need for an XML config.
| acdha wrote:
| It's more typing but it's simple typing most people don't
| even think about because it's predictable (not to mention
| automatic in many editors). I'd take that over the frictional
| cost of thinking your YAML is done and then having to debug
| magic data conversion or realize that you left out one
| character causing something to be parsed completely
| differently.
|
| Where XML falls down hard is tool usability. There's still no
| standard formatter or good validation tool, and things like
| namespace support is a constant source of friction.
| scott_w wrote:
| > I'd take that over the frictional cost of
|
| You might but the rest of us didn't. At a visual level,
| YAML is really nice. The problem is when you actually want
| to use it for... well... anything.
|
| On the flip side, XML is downright ugly to read and write,
| which makes it a no-go for configuration files that are
| written and modified by hand.
| acdha wrote:
| Look, I'm not saying XML is perfect but the worthy
| criticism is more substantial than closing tags. It's a
| very easy convention to learn requiring little thought
| and I haven't used an editor which didn't automate that
| process since the turn of the century. That's less
| cognitive load than having to remember a bunch of
| context-sensitive rules about YAML's magic behavior - I'm
| thinking in particular of people I know who've burned
| hours only realize that they'd missed a character
| somewhere, forgot to escape one value, or had an
| indentation issue causing something to be ignored.
|
| The criticisms I would make are more fundamental: the XML
| data model is different than the most popular data
| structures and the APIs in most languages are quite
| cumbersome and can lead to silent data loss (name spacing
| and selectors). Someone probably could have done an XML5
| effort 15 years ago but by now I don't see that
| happening.
|
| If I had to rank the options for a configuration
| language, I'd probably go HCL, TOML, JSONC, JSON, YAML +
| Prettier + a YAML lint schema, YAML, XML. XML could rise
| up with better tools but so could a low-magic YAML
| variant.
| wirrbel wrote:
| I mean it wasn't just that.
|
| * `<?xml ...` something line lost a percentage of people
| already, etc.
|
| * Essentially XML was a markup language (SGML stripped down),
| while you can use markup to mark up key-value pairs, it just
| wasn't designed or optimized for that.
|
| * To the contrary, it also did away with the nice SGML
| features for that instead of `<tag>value</tag>` one could
| write `<tag/value/` and the self-closing tags (like `<ol><li>
| ... <li></ol>` in HTML. For most particularly deep config
| files self closing tags would have been perfect.
|
| * XML attributes vs substructure was just an awkward choice
| to give out. I.e. `<myConfig userId="asdf" host="asdfasdf"
| ...>` vs `<myConfig><userId>asdf</userId><host>asdfasdf</host
| ></myConfig>`.
|
| More so I get the feeling when looking at the XSL, XSLT, etc.
| mania that a similar pattern was at work as with UML: special
| interest groups and tooling providers driving the evolution
| of a standard with their interests in mind and not the
| interest of users or developers.
|
| Overall XML could have been something like this `<myConfig /
| <userId/asdf/ <host/asdfasdf/` if it was SGML.
|
| YAML and JSON succeeded because they had a clean and
| predictable, no-nonsense mapping between encoding and object-
| model after decoding. Probably we should all switch to an
| almost-yaml format that does away with the peculiarities, and
| the FANG companies would have the momentum to make that
| happen.
|
| I personally would like for HJSON (https://hjson.github.io)
| to see more adoption, but that train has passed...
| donatj wrote:
| It's the learning curve really.
|
| Our industry has the remarkable properties of being almost
| entirely newbies, a constant churn of green developers,
| combined with being very bad about passing down generational
| knowledge. This is why things that are easier to explain win
| out over things that are technically superior but take longer
| to get your head around almost every time.
|
| The thing JSON had going for it over XML is that it maps
| cleanly to most languages object models so you can read the
| results directly. No writing XQueries or DOMs to read values.
|
| It's only major downfall is lack of comments, which has lead
| people to YAML. (There's plenty of other things it lacks in
| comparison to XML, like native schemas, but most of that falls
| into things newbies don't know they want)
|
| YAML is in this weird middle place where it's easy to explain
| but impossible for a human to master. It appears as simple as
| JSON to newcomers who adopt it, but the long time users of it
| find it full of foot guns. People wanted JSON with comments but
| instead they got the complexity of XML minus the clarity.
| makeitdouble wrote:
| XML is/was hated by newbies and old farts alike.
|
| Of course some of the hate come from the application where
| XML was used, more than XML itself, but it also is a deeply
| flawed language.
|
| Nowadays the main issue would be that it requires a complex
| generator and parser libraries to be any useful (you'll never
| want to deal with XML parsing/escaping by yourself) yet it's
| not as efficient as binary formats like protobuff for
| instance.
|
| That means that anything you'll want to edit by hand or be
| purely textual and readble will be better done in yaml or
| json, and anything beyond that can be done in other ways. The
| need for a single language trying to awkwardly span all the
| spectrum isn't big.
| gpderetta wrote:
| We all hate XML.
|
| We equally hate all replacements (as a config language)
| after it. At least XML stayed around long enough to have
| decent tooling.
| singingfish wrote:
| XML has the hate because back in the day everyone was like "oh
| XML is a human and computer readable data format, that will
| solve all of our problems".
|
| Sure it solved all of the problems by making a data format that
| both humans and computers found difficult to read.
|
| "Oh I've got a problem. I'll solve it with XML. Now you have
| two problems" ;)
| red_admiral wrote:
| XML is ok when both the reader and writer are machines, but you
| still want a text-based format (otherwise I'd just use
| protobuf) or when maybe you occasinally want to edit by hand
| but not often.
|
| I think it's the same reason why, although HTML is a great
| standard, lots of us like writing in Markdown and having
| something convert that to HTML, despite all the problems when
| you try and push Markdown further than it was intended.
|
| YAML, as I see it, is trying to be to XML configurations what
| Markdown is to HTML, with the added bonus of an attempt at a
| tag/reference system to store object graphs that are not trees.
| Back in the day of XML-based Spring config files, we had <bean>
| and <ref bean=...> but as far as I know that's implemented on
| the Spring layer, it's not a generic property of XML, whereas
| YAML tries to abstract that into the format itself.
| thrwawayjan1223 wrote:
| I worked with hand editing XML files in the past, and I
| didn't really have an issue with simple XML files. I actually
| prefer XML in some cases.
|
| I see your point if you need to edit a complex XML with
| multiple namespaces mixed together, but a plain XML file can
| be just as readable as JSON.
|
| Some JSON files can be really hard to edit by hand too. At my
| current workplace I often have to deal with nested JSON
| files, where a JSON contains values that are also JSON, but
| encoded and escaped so that it is difficult to edit.
| bambax wrote:
| > _YAML, as I see it, is trying to be to XML configurations
| what Markdown is to HTML_
|
| Yes, that's probably a good analogy. The big difference
| though is that if some Markdown fails to parse, nothing
| really bad happens, while a YAML file that fails to parse can
| bring a whole system down.
|
| Also, Markdown is a famously ambiguous format; it trades
| precision for ease of write, and that's fine, mostly.
|
| But in a configuration file, ambiguity is really the opposite
| of what you want.
| YeGoblynQueenne wrote:
| >> All that horrible mess, it seems, because people didn't like
| to have to close tags.
|
| If people keep burning their faces off it's not the fault of
| the flamethrower! If they keep hacking each other's limbs off
| it's not the fault of the chainsaw! If they keep blowing their
| houses up it's not the fault of the gas bottles! It's not like
| we need to accommodate every possible failure mode of the human
| brain, and make things safe just because the next idiot that
| comes along is going to cause a catastrophe. It's the idiot's
| fault that they forget to apply the breaks and take advantage
| of the failsafes, it's not the responsibility of the system to
| have the breaks and failsafes where any idiot can find them!
|
| I've head that kind of excuse 0 times, outside of software
| circles. But of course I'm exaggerating because nobody really
| suffers because of XML other than the people who use it
| everyday. Like this once junior dev, for example, who was made
| to create XML by hand by eyballing an Excel document, just
| because paying a junior dev salary is easier than getting
| middle management (and clients) to learn to structure an Excel
| spreadsheet properly.
|
| Sometimes, just sometimes, you have to design a tool with the
| way it will actually be used in the real world in mind. And all
| the rest of the time? Well, all the rest of the time you have
| to do it that way, too.
| melx wrote:
| I miss the old days of working with XSLT and XPath. A nice way
| of giving some design touches and filtering/sorting to XML
| documents. It wasn't perfect but I would take any time over the
| yaml/json/toml/ini.
| thraxil wrote:
| I don't know if I'd say I "miss" XSLT and XPath, but yeah, it
| was fun and powerful at the time. I made some crazy stuff
| with Cocoon and AxKit that, if you didn't mind the syntax,
| were actually pretty elegant.
| manbart wrote:
| I'm surprised XQuery never became more popular, it's
| essentially XSLT with a 'normal' syntax
| rwmj wrote:
| XPath is cool (and I still use it for work regularly). jq
| JSON expressions seem to be inspired by it as well.
|
| XSLT was horrible though. A programming language with XML
| syntax, no thanks!
| int_19h wrote:
| XQuery gives you 90% of XSLT with a syntax that is a
| superset of XPath. Verbose queries
| (https://en.wikipedia.org/wiki/FLWOR) are especially nice
| and readable.
| bambax wrote:
| Me too!
| MilStdJunkie wrote:
| Eh, definitely there's some generational round robin going on.
| And the "amsterdam problem" in YML is just insanity. But you
| got to admit, XML has some very low level intrinsic problems
|
| * Schemas violate underlying XML rules all the time, and due to
| variance in XML parsers, it gets a pass but only on specific
| configurations. There's no one Holy XML Parser. Leading
| whitespace in attributes? Sure, why not?! Whitespace sensitive
| element order? Ooh yeah, lots of it! But try and feed it to
| libxml and the whole thing collapses - without error handling,
| mind you, but that's not XML's fault.
|
| * Whitespace agnostic means diffs and merges are element aware,
| and it's nigh-impossible to estimate the upper compute limit of
| an element aware diff
|
| * There is NO OFFICIAL WHITESPACE SPEC - so if you try and fix
| it with normalization and lines, it's going to be different
| everywhere you go. So you're forced to switch on element aware
| diff / merge in your VCS, which is a pretty big change, and
| it's one you have to sell other departments on.
|
| * XML breaking 1NF, which, aka means that XML can have XML
| inside of XML infinite recursion when playing data format =
| which breaks so very very many things . . it's actually going
| against the whole concept of a data hierarchy, which is built
| in to XML at a low level.
|
| * Sort of riding on that, in order to parse XML you have to eat
| it element by element, and there's no way to tell when an
| element is going to end or of it recurses N levels. This makes
| it computationally expensive. The CAD STEP format has some of
| this disease - it has to be loaded in its entirety to parse,
| which can be holy hell with a TB file.
|
| * Yeah sure, namespace hell. No one really figured out a way to
| fix it. S1000D (XML spec) to this day just denies that XML
| namespaces ever existed, and I can't really fault their WG for
| doing that. I can fault them for so many other things, but not
| that.
| xienze wrote:
| > I never could understand the hate XML got
|
| It's because so many people would see examples of complex _XML-
| based formats (WSDL, SOAP, etc.)_ and infer that _XML itself_
| is complicated. XML is really not that difficult to understand,
| and I find it quite amusing that the same people who don't bat
| an eye at writing HTML complain about how baffling XML is.
|
| Is writing an XML parser difficult? Yes, very much so. But
| again, that doesn't make XML itself complicated. And before
| anyone tries to call out this particular comment, keep in mind
| that writing a fast, correct, and safe JSON parser is no walk
| in the park either.
|
| Are there many examples of complicated XML-based formats? Yes,
| but that's just a reflection of the complexity of some
| particular configuration model, not XML itself.
| silvestrov wrote:
| The really great thing about xml with XSD is the ability to
| validate the document really well.
|
| I.e. having content matching regexps, so you can be sure
| version numbers are always \d+(\\.\d+)* and does not have
| unexpected letters or spaces in it. That date values are in ISO
| format.
|
| Very useful for APIs to 3rd parties as it now is very easy to
| catch most errors and provide an useful error message for them
| without having to code every check.
| philjohn wrote:
| True - but then again, writing an XML config file can be a bit
| of a pain as it's more verbose.
| api wrote:
| It's the verbosity for me. Why does everything need a fully
| spelled out closing tag?
| FpUser wrote:
| I see nothing wrong about closing tag. Helps to navigate and
| understand in my opinion. I have way more problem with
| developers trying to be super concise and writing constructs
| that are very hard to parse mentally when looking at.
| singularity2001 wrote:
| So that in long nested data you know which tag closes what.
| {{{{{}}}}}
| mmis1000 wrote:
| These days, JSON also has JSON schema that allows you to
| enforce the structure of a JSON file.
|
| If you need a enum that only allow specific value, you just
| list it.
| talideon wrote:
| The problem with XML is mainly in the 'M': it's a _markup_
| language. Using it for configuration and arbitrary data
| serialisation isn't where its strengths lie, but it got shoved
| into those niches because if you look at it _just right_, you
| can make it work in them.
|
| I don't hate XML myself, but I do hate how it's been abused
| over the years.
| PurpleRamen wrote:
| XML is cool and useful, but for humans you need good tooling to
| handle it well. And it still is very noisy with the all it's
| boilerplate and what advanced features can bring in. And
| overall it has a culture of making things complex, and
| complicated.
| singularity2001 wrote:
| XML is preferable for long or heavily nested data structures.
|
| For small flat trees the alternatives are preferable
| acdha wrote:
| I remember being excited when XML hit 1.0 (as a new programmer
| this seemed like a huge advance over things like classic Unix
| configuration files), and progressively disappointed over the
| next decade as the promise not only wasn't delivered on.
|
| The things which killed XML seem to me to be related to the old
| standards culture: the people involved assumed adoption was
| inevitable and distracted themselves with increasingly arcane
| thickets of new standards, with the assumption that someone
| else would spend time on the "boring" work of building
| professional-quality tools and documentation or cleaning up
| usability warts. That other 80% of the work never happened and
| most people who had a choice moved on.
|
| As a thought experience, imagine if libxml2 had had even a
| single dedicated developer focused on tracking standards or
| making usability improvements, instead of training multiple
| generations of users that XML was slow and hostile to users.
| Various XML committees' travel expenses building standards
| which were never used likely cost more than that. Not leaving
| XPath frozen around the turn of the century would have helped
| in so many places.
|
| The other wart I think would have made a surprising difference
| is the usability disaster around namespaces. So many tool
| developers forced users to switch between the short
| namespace:attribute form they used everywhere in the document
| and the {namespace url}attribute form that resolves to, or
| forced you to respecify the namespaces on every operation
| rather than reusing the values the parser had already loaded.
| Users begrudged that verbosity but they hated it when it meant
| something silently returned incorrect results because a
| selector using the document's own syntax didn't find the
| element they could see using those exact values. Absolutely
| nothing anyone did in the XML world was a better use of time
| than fixing that would have been since it trained people to
| think of XML-based tools as a painful, error-prone experience
| to be avoided -- and they did as soon as they could.
| makeitdouble wrote:
| > An XML document with a well-thought-out domain-specific DTD
| would solve all these problems;
|
| But then a YAML document written by disciplined and well-
| thinking people will also not have these problems.
|
| Any of the complex formats work well when used in the most
| fitting way and go horribly wrong when people try to benefit
| too much from the complexity. XML was also a nightmare when put
| in the wrong hand, just as yaml is, and the next hyped and
| turing complete document syntax will be.
| enw wrote:
| If something's too easy to misuse, it's bad design.
| avereveard wrote:
| "Parsing json is easy"
|
| https://seriot.ch/projects/parsing_json.html
|
| https://raw.githubusercontent.com/nst/JSONTestSuite/master/r...
| netgusto wrote:
| This is not claimed by the article afaict. It's stated though
| that json spec/syntax is simple, and that it's simple as a
| language; my interpretation is that this means
| "simple/understandable/predictable for humans".
| shp0ngle wrote:
| JSON is a subset of YAML. (you can just dump JSON in your YAML
| document and it's a valid YAML.)
|
| So parsing YAML will be strictly harder, as it needs to include
| a JSON parser.
| jefftk wrote:
| If language A is a subset of language B it doesn't follow
| that parsing B is strictly harder.
|
| This is not guaranteed. Consider two languages:
|
| 1. Some number of "a" followed by the same number of "b". For
| example, aaabbb but not aabbb.
|
| 2. Any number of "a" followed by any number of "b". For
| example, aabbb but not aba.
|
| In this case (1) is a subset of (2), but parsing (2) is
| easier than parsing (1) since in (1) you have to verify that
| the numbers match (which, in technical terms, means its not
| regular).
| CipherThrowaway wrote:
| I was at a place 7-8 years ago with a hand-rolled JSON parser
| that had been written across a few afternoons by one guy
| without much ceremony. It was untouched after that aside from
| some renaming and other minor refactors. As usual with these
| things, the decision to hand-roll was made by one very
| opinionated dev who got his way because the business depended
| on him.
|
| When that article was published it started an argument among
| the team. The guy ended up implementing the test suite to
| defend his decision. To my surprise the parser passed all tests
| apart from 3 or 4 unicodey-ones where it was more permissive
| than it should have been. There were plenty of production bugs
| at this place but none of them were due to the parser.
|
| IMO parsing JSON really is simple. If you read through the
| article you'll see that most of the test cases are dead simple
| and hardly landmines.
| jasode wrote:
| _> The literals off, no, and n, in various capitalizations (but
| not any capitalization!), are all false in yaml 1.1, while on,
| yes, and y are true. _
|
| Fyi... I looked at PyYAML source code and the parser code omitted
| single-character 'y' in the regex. Previous comment:
| https://news.ycombinator.com/item?id=26681295
| edent wrote:
| This is why I invented KYLI -
| https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior...
|
| People shouldn't be hand-editing these sorts of configuration
| documents. 21st century tooling would prevent so many of these
| "foot-guns".
| progx wrote:
| This one liners lead to unreadable configuration. With tools,
| the format is not important. But we did not talk about tools,
| we talk about that humans can easy read and write a
| configuration.
| Firadeoclus wrote:
| A text editor is a tool. Text files are only "human
| readable"/"human editable" in the sense that such tools are
| ubiquitous whereas structured data editors are not.
|
| Which is a real shame. Entering, manipulating and validating
| small amounts of structured data is such a fundamental task
| when using computers that it deserves dedicated tools, and
| the sheer amount of issues that only exist because such tools
| are not ubiquitous is absolutely mindboggling.
| bjarneh wrote:
| > There were two changes to it in 2005 (the removal of comments,
| and the addition of scientific notation for numbers)
|
| They actually had comments in JSON but removed them in 2005. That
| is odd and a bit annoying :-)
| djcris wrote:
| [flagged]
| danjc wrote:
| In the linked YouTube video talking about JSON: "People were
| putting instructions to the parser in comments... so I [removed]
| the comments." - Douglas Crockford.
|
| Let the impact of that sink in - that decision has almost
| certainly literally cost millions of developer hours! In fact,
| the article actually claims that the lack of comments in part led
| to YAML itself so by implication...
|
| I jest (sorta) but what a great example of an initially helpful
| decision having last consequences.
| chii wrote:
| Crockford probably didn't imagine the idiocy of creating YAML
| to replace the lack of comments in JSON.
| wodenokoto wrote:
| Every time I've reached for yaml after being bitten by the
| Norway problem has been because I wrote comments in my json
| file and it failed be read.
| krapp wrote:
| Wow, you must really be forgetful. Maybe your productivity
| would improve with some post-it notes stuck next to your
| monitor, reminding you of the basics about how these formats
| work.
| revskill wrote:
| Secret fact: Hackernews data is stored as a huge YAML file on
| disk.
| jjgreen wrote:
| What we need is some kind of extensible markup language ...
| andybak wrote:
| Funny but not quite.
|
| Nobody wants extensibility and "document markup" isn't the same
| as "configuration specification"
| dig1 wrote:
| > is that for a long time it was the only viable configuration
| format
|
| Actually, this is not the case. We had INI format for simple
| stuff and XML (protected with entire schema) for complex things
| many years ago, which worked. Yet, we wanted something readable
| (like INI), but able to express complex types (XML).
|
| I don't think Toml is a viable replacement - for me, it has an
| INI-level of simplicity with even worst hacks to have nested
| structures. But, give it time and you'll have another YAML.
|
| But yes, YAML is confusing for edge cases (true/false, quoting),
| but I'm going to find a powerful replacement that is not XML.
| Maybe EDN can jump in, but for anything more complex, I'd rather
| have a Lisp interpreter and structures around than any of
| mentioned and upcoming formats.
|
| Thinking that JSON is a suitable replacement, imagine writing
| Ansible or k8s stuff in it; that would be fun ;)
| CipherThrowaway wrote:
| > Thinking that JSON is a suitable replacement, imagine writing
| Ansible or k8s stuff in it; that would be fun ;)
|
| Writing in YAML doesn't feel much better. YMMV but I've been on
| teams using Pulumi for k8s and the developer experience has
| been significantly better. I can automate, type check, lint,
| click through to definitions the same way I do with other
| typescript.
|
| Pulumi is a young product with many rough edges but it's
| already been a game changer for me.
| cocochanel wrote:
| I checked the sample Kubernetes project in Java [0] and it
| looks hideous. YAML looks much better in comparison.
|
| [0] https://www.pulumi.com/docs/get-
| started/kubernetes/review-pr...
| CipherThrowaway wrote:
| Don't disagree there. Java support is less than a year old
| and the Java API is.. not great. TypeScript is the more
| popular and mature choice.
| danudey wrote:
| What, you mean you don't want to write code like
| Deployment("nginx",DeploymentArgs.builder().spec(Deployment
| SpecArgs.builder().selector(LabelSelectorArgs.builder().mat
| chLabels(labels).build()).replicas(1).template(PodTemplateS
| pecArgs.builder().metadata(ObjectMetaArgs.builder().labels(
| labels).build()).spec(PodSpecArgs.builder().containers(Cont
| ainerArgs.builder().name("nginx").image("nginx").ports(Cont
| ainerPortArgs.builder().containerPort(80).build()).build())
| .build()).build()).build()).build());
| PaulHoule wrote:
| JSON has no comments. That disqualifies it immediately.
| bogwog wrote:
| { "/*These are the ports we use": "*/",
| "ports": [22, 80, 443] }
| ok123456 wrote:
| Now, write a json-schema for that.
| dtech wrote:
| Can be done by either additionalProperties: true and not
| specifying the property, or "patternProperties": {
| "^/\\*": { "type": "string" }}.
| ok123456 wrote:
| I'd rather have real comments than this epicycle.
| nawgz wrote:
| "additionalProperties": true
| int_19h wrote:
| Many JSON dialects do in practice.
| ilyt wrote:
| It's very simple actually.
|
| ... just fucking don't, generate config in your configuration
| management tool of choice and then serialize it to YAML. You
| get all of the advantages (nice to read)and none of the
| disadvantages (need editor or it is PITA)
| brnewd wrote:
| I would love to swap Pulumi for Ansible, but it's cloud-only
| for the most part.
| jimmaswell wrote:
| XML did everything and it's perfectly readable if it's
| formatted and structured well. This zoo of different
| markup/object/whatever languages we have to deal with now is
| largely a mistake.
| wyuenho wrote:
| XML is a mistake. 80% of every XML document is just redundant
| noise because the IBM lawyer who invented the SGML syntax had
| never heard of S-expression.
| jimmaswell wrote:
| The "redundant" noise makes it more human-readable in my
| opinion. If you're in the weeds of a big XML document
| that's larger than your viewport, being able to see that
| you're at the end of a </Address> helps you keep your
| place, as opposed to something like JSON which I find much
| easier to get lost in. Even if it's not that big I
| appreciate the extra rails on the XML making it easier to
| keep my place.
| int_19h wrote:
| Writing documents with S-exprs will get very tedious once
| you have to deal with stuff like whitespace and such that
| will inevitably show up when you're dealing with _text
| documents_. SGML is specifically optimized for those use
| cases, which is why it is so much more awkward to use when
| all you want is serializing some random structured data.
| sam_lowry_ wrote:
| SGML is way shorter than XML.
|
| But as complex as YAML to understand
| bogwog wrote:
| XML is too much of a pain to write by hand.
| tetraca wrote:
| XML is pretty tolerable with a slightly more sophisticated
| editor that supports completion. The things that make YAML
| awful never get any better.
| jimmaswell wrote:
| Throw in copilot to anticipate all the boilerplate and
| repetition for you and it's downright pleasant.
| wombatpm wrote:
| XML is like violence. If it's not solving your problem, you
| need to use more
| berniedurfee wrote:
| It's not that bad really. About the same as HTML, yet more
| difficult to mess up.
|
| Has comments, unlike JSON.
|
| More difficult to mess up than YAML.
| gnulinux wrote:
| This is confusing because XML is the archetype of what I
| would consider unreadable. If I got a prompt in a programming
| language design workshop to intentionally design something
| unreadable for humans, I would start thinking about XML and
| see where it leads me. Can you name any language any more
| unreadable than XML to help me?
| jimmaswell wrote:
| https://www.mssqltips.com/tipimages2/2899_img5.jpg
|
| I look at this and I see a visually-well-organized
| hierarchy of objects with properties and children. It's a
| sideways tree diagram. I honestly don't get what's supposed
| to be unappealing about it.
|
| > Can you name any language any more unreadable than XML to
| help me?
|
| Most of the other markup languages depending on the
| usecase. The "redundant" tags everywhere like </Address> in
| XML are very helpful if you're in the weeds of a big
| document that's taller than your viewport - you still know
| exactly what you're looking at and where in the hierarchy
| you are, as opposed to JSON or YAML which I find much
| easier to get lost in.
|
| Anyone can pick up a (properly-named) XML document and tell
| what it means at a glance, but JSON is easily a mystery
| unless you know the implicit schema beforehand.
|
| Maybe there's some minor friction to get past looking at
| XML because it's a little visually busy before you learn
| how to parse it out? Is that all there is to it?
| wyuenho wrote:
| INI has no spec, and there are many variations all slightly
| incompatible with each other, kinda like Markdown. YAML is
| really the only sane configuration language that lets you
| denote nested structures while keeping them look nested.
|
| Basically, all of the problem identified in the article can be
| dealt with 1 rule - always quote your strings. I agree with the
| author we should have reduced, safe and minimalist subset of
| yaml, which is basically YAML 1.2, released in 2009.
|
| P.S. Please just stop using PyYAML.
| mike_hearn wrote:
| There's HOCON which is pretty good if you can run on a JVM.
| It's a superset of JSON designed for readability and human-
| friendliness when writing config files. It doesn't change the
| type system and doesn't have yamls weird edge cases, but is
| still a lot easier to write than JSON. There's also a
| relatively tight spec.
|
| https://github.com/lightbend/config/blob/main/HOCON.md
|
| We use an extended version of it for our app and the
| resulting config is pretty clean. You can see an example
| here:
|
| https://hydraulic.software/blog/8-packaging-electron-
| apps.ht...
|
| The only downside is that the reference implementation is
| hardly maintained anymore.
| jwilk wrote:
| > Please just stop using PyYAML.
|
| Why?
| wyuenho wrote:
| From the article: Although yaml 1.2 is more
| than 10 years old by now, you would be mistaken to think
| that it is widely supported: the latest version libyaml at
| the time of writing (which is used among others by PyYAML)
| implements yaml 1.1 and parses 22:22 as 1342.
| talideon wrote:
| I second that. And if people need to deal with YAML in
| Python, they should be using ruamel.yaml, which is a far
| superior library on just about every level:
| https://pypi.org/project/ruamel.yaml/
| lylecheatham wrote:
| Except that the author of ruamel.yaml:
|
| refuses to use git [0]
|
| refuses to take community submissions (except through Stack
| Overflow? Seems like a misuse of SO) [1]
|
| and refuses to implement .dumps() [2].
|
| He is difficult to work with, and any time I need to debug
| code that intimately deals with ruamel.yaml types, I wince.
|
| [0] https://github.com/pycontribs/ruyaml/issues/1
|
| [1] https://yaml.readthedocs.io/en/latest/contributing.html
|
| [2] https://stackoverflow.com/a/63179923/15170511
| wyuenho wrote:
| Or if you want to go really minimalist there's also
| https://github.com/crdoconnor/strictyaml
| berniedurfee wrote:
| If JSON and YAML were first on the scene; XML would be invented
| as an answer.
|
| Hard to mess up. Robust schema language. Very flexible. Easy to
| process.
|
| Yeah, it's verbose, but that's the trade-off between usability
| and robustness.
|
| Now get off my lawn!
| revskill wrote:
| This is exactly what i'm doing with this YAML to SQL converter:
| https://gist.github.com/revskill10/57ecd8efb72f361b93e6d9d9f...
| xmcqdpt2 wrote:
| Why???
| revskill wrote:
| - No parentheses is needed
|
| - No forced order of select, order, having,... Want from
| before select ? Fine.
|
| - Composability
|
| - Extensibility.
|
| - Security: Prevent SQL injection when sent from browser.
| throwawaymaths wrote:
| No. XML is broken for structured data that isn't HTML because
| it's not really clear (there aren't even "best practices"
| afaict) what should be a text node, what should be an attribute
| node, and what should be a subtag node.
| int_19h wrote:
| It's not like other formats don't face similar questions.
| E.g. if you have a list of key-value pairs to serialize to
| JSON, do you translate those keys to JSON properties in an
| object, or do you translate each pair as an object in an
| array?
| throwawaymaths wrote:
| The motivation for doing that in JSON comes with best
| practices: you use the key-value objects thing if and
| _only_ if you have an ad-hoc list of items that you 'd
| really really like to shove into a well-typed schema.
| eru wrote:
| Use S-Expressions?
| usrnm wrote:
| With each tool using its own macro-based DSL for managing
| them. That would be fun
| mixedCase wrote:
| S-expressions do not necessarily imply Lisp nor an
| executable environment, it's a (family of) syntax(es) to
| build a document format. Like EDN.
| ilyt wrote:
| So even worse as you can't use any prior knowledge of
| lisp
| luisapic wrote:
| [flagged]
| tgbugs wrote:
| My distaste for yaml led me to attempt just that. The first
| piece that was missing for me was a python parser that could
| produce reasonable error messages and could be transformed
| into the desired internal representation in python. So I
| wrote one [0]. It was supposed to be a single file that was
| less than 100 lines and could be copied and pasted into any
| project that I needed. Turns out that the issue was a bit
| more complex.
|
| The issue is that there is sufficient complexity in finding a
| portable representation for configuration formats that it
| just kicks the can down the road. On the other hand it means
| that as soon as you decide what format you are going to
| support you can quickly implement it. There is more or less a
| intersectional grammar that works across most if not all
| lisps, and that is the plist `(:k v :k2 (:k3 v2))`. So I
| settled on that for my own use.
|
| After all that work I have not dealt with the fact that
| numbers and chars do not have a portable representation
| across lisp dialects, which is a key complaint in other
| threads here. Limited support for let binding constants also
| seems like a feature that would allow for just enough
| expressivity to make the format useful without opening up the
| terror that is `&` and `*` in yaml (cool and useful as it may
| be).
|
| In summary s-expressions are: 1. missing good parsers in a
| number of language ecosystems 2. not standard across lisp
| dialects 3. need additional semantics for binding, multiple
| expressions, etc. 4. still better than yaml and json
|
| 0. https://github.com/tgbugs/sxpyr
| jerf wrote:
| I know what s-expressions are, vaguely. Vaguely in terms of
| "I couldn't write a grammar for them off the top of my
| head.", that is, not "what are they".
|
| Is there a single agreed-upon defined grammar that everyone
| can use? Preferably one simple enough that like JSON's it is
| at least _capable_ of being used as a graphic on the home
| page for the format? https://www.json.org/json-en.html
|
| This is an honest question, because there may well be and I
| don't know it.
|
| However, I will put this marker down in advance: If multiple
| people jump up to say "oh, yes, of course, it's _right here_
| ", and their answers are not _100% compatible_ with each
| other, then the answer is _no_.
|
| The other marker I'll put down is "just use common lisp", I
| want verification that it _really is_ 100% standardized, no
| question what any construct means, ever, and I still bet we
| get people who would rather see Scheme or Clojure, and I bet
| there 's some sort of difference.
|
| Neither of these objections is fatal to the idea. JSON is
| technically not just "javascript objects", so if someone
| carved out a defined format from s-expressions, then held it
| up as a standard, that would be as valid as what Crockford
| did. But at least as of right now, I'm not aware of anyone
| having done that standardization work. Replies welcome.
| mirekrusin wrote:
| Dune [0] is using sexp [1].
|
| [0] https://dune.build
|
| [1] https://dev.realworldocaml.org/data-serialization.html
| jerf wrote:
| I don't see a grammar I could use for an external
| standard there. I could be missing it. And I assume there
| must be one implied, since it appears to be a read-write
| format. However such things have a nasty habit of turning
| out upon further inspection to not be as generic as
| supposed and have local idioms buried in them, often in
| surprising places. Even JSON, simple as it is, had that
| problem, and it is after all a subset in an attempt to
| squeeze out those localisms.
|
| Is it compatible with Common Lisp, such that all such
| s-expressions will be compatible with it in some natural
| way?
| lapinot wrote:
| SEXPR := YOUR-CUSTOM-LITERALS | "(" SEXPR* ")"
| jerf wrote:
| Along with the many other issues that would have for just
| straight-up not being a usable grammar, the word "CUSTOM"
| appearing in a nominal _cross language standard_ is a
| non-starter.
| lapinot wrote:
| Oh come on. Did you think anybody would write an RFC in
| the comments? Add arbitrary list of whitespace tokens
| everywhere and pick whatever favorite grammar you have
| for literals. Json strings and numbers since you seem to
| like them.
| jerf wrote:
| Of course not. But you are at least semi-seriously trying
| to propose something. I promise you it won't be that easy
| to get any sort of agreement.
|
| This is a classic "oh come on how hard can it be" that,
| once you get into it, it turns out the answer is _very_.
| Arbitrarily so. Agreement is hard. Details are hard. A
| standard with no details is not terribly different from
| "just give us some text".
| urbandw311er wrote:
| > YAML is confusing for edge cases (true/false, quoting)
|
| I'm not sure I would consider booleans an 'edge case'
| singingfish wrote:
| I'm nose deep in an Oracle to Postgres conversion at the
| moment and from that experience, among others, I can
| absolutely assure you that although booleans are definitely
| simple they are also most definitely a very sharp edge case.
| int_19h wrote:
| Booleans, or NULL specifically?
| falcolas wrote:
| But who doesn't love the OracleDB tri-state boolean?
|
| /s
| esrauch wrote:
| Booleans aren't an edge case, on is an edge case way to spell
| false.
| 411111111111111 wrote:
| specifically according to https://yaml.org/type/bool.html:
| y|Y|yes|Yes|YES|n|N|no|No|NO
| |true|True|TRUE|false|False|FALSE
| |on|On|ON|off|Off|OFF
|
| Not all parsers support all of these options, but
| true|false is always supported as YAML is a superset of
| JSON which makes these mandatory.
| xmonkee wrote:
| I can't imagine thinking this is a good idea if you've
| done any programming whatsoever. What happened here?
| orra wrote:
| Even leaving aside the word <on>, it's problematic that,
| syntactically, <true> and <foobar> look the same but are
| treated differently.
| ilyt wrote:
| The "edge case" here is "I'm using dynamically typed language
| and some parsers implemented alternative wording for
| true/false"
| int_19h wrote:
| It was a part of the YAML spec for a long time, so it has
| nothing to do with dynamic typing or parser extensions.
| ilyt wrote:
| It does. If I define struct to contain boolean, Go yaml
| parser will always parse it to boolean
|
| If I define it as any other type, it will parse it as
| that type. So any conflicts like in the article just
| don't happen, or at worst, produce error in parsing.
|
| Static typing in this case acts essentially as schema
| Normal_gaussian wrote:
| > Thinking that JSON is a suitable replacement, imagine writing
| Ansible or k8s stuff in it; that would be fun ;)
|
| k8s via helm is often templated via go template strings; which
| works by creating an unreadable and unhighightable mess,
| introducing lots of its own bugs.
| mirekrusin wrote:
| JSON is good because it doesn't let you do stupid things like
| writing programming languages in it.
| GauntletWizard wrote:
| Jsonnet and Cue are the places to be looking.
| chazu wrote:
| Cue is one of the most exciting developments that's impacted
| me professionally in recent years. I can't advocate for it
| enough.
| tremon wrote:
| Cue is interesting, but why is it only available as a
| command-line tool rather than a library? I'd want to
| integrate such a configuration language in my programs, so
| I could use its evaluation and validation capabilities
| rather than writing a custom parser/validator.
| IshKebab wrote:
| Also their website is terrible. One of those projects
| that assumes you've already decided (or been forced) to
| use it and have cleared out a week of your schedule to
| learn how to use it.
| tremon wrote:
| I agree. The website is heavy on theory and very light on
| practical usage.
| mirekrusin wrote:
| I learned cue from it during one weekend with plenty of
| time to play with kids, using it in production since
| 2020, it's been absolutely great, zero problems, very
| terse configs, intuitive formalism.
| IshKebab wrote:
| I took another look and eventually found the bit of the
| website that they _should_ put front and centre in the
| Tutorial page. Still difficult to navigate (why doesn 't
| the page tree show up on the left?) but it is at least
| well written and to the point.
|
| The "learn more" button on the front page should link to
| that, perhaps with a single paragraph giving motivation.
|
| And the main page breaks the fundamental rule of
| programming languages/formats. Put examples on the front
| page!
|
| I assumed they hadn't done that because the examples
| would be too complex or maybe the concepts were too
| difficult to demonstrate with small examples but having
| gone through the tutorial that isn't there case at all.
| GauntletWizard wrote:
| Are you talking about
| https://cuelang.org/docs/tutorials/tour/intro/ ? Even
| that is a bit light on detail for real usage, while
| https://github.com/cue-
| lang/cue/blob/v0.4.3/doc/tutorial/kub... is kinda
| rambling. I don't think that there's "One true way" to
| introduce these concepts; How you teach cue to a config-
| generation novice is very different from someone who's
| used to using an IDE to generate kubernetes YAML.
| dangoor wrote:
| If you're writing in Go, you can use it as a library.
| This is poorly documented, unfortunately.
| anonymoushn wrote:
| Is INI a format in the sense that it has a formal grammar and a
| bunch of compatible parsers and serializers, or is it more of a
| convention?
| contravariant wrote:
| If Python is valid option then lisp syntax should also be
| considered as a viable alternative.
|
| Heck the number of times I've seen attempts at creating a lisp in
| yaml, it is easier to just get it over with and make it a lisp in
| the first place.
| thorum wrote:
| > The yaml package supports most of YAML 1.2, but preserves some
| behavior from 1.1 for backwards compatibility. YAML 1.1 bools
| (yes/no, on/off) are supported as long as they are being decoded
| into a typed bool value. Otherwise they behave as a string.
|
| Good solution for using YAML with strongly typed languages. No
| reason to leave type decisions up to a parser. I assume serde
| does something similar in Rust?
| bbojan wrote:
| I've always hated YAML, and now I'm glad there's a very nicely
| written article to back me up.
|
| I also notice that I have a preference to procedural as opposed
| to declarative specifications. E.g. I find conditionals and loops
| in declarative specifications quite bad.
| milliams wrote:
| Almost all those problems are solved by marking up the strings as
| strings, using " and ".
| progx wrote:
| Then they should make quotes necessary in the YAML-
| specification. Little more to write, but better to validate.
| IshKebab wrote:
| Great idea. I would also suggest adding some syntax to make
| lists and maps more obvious (it can be a bit unclear in
| YAML). Maybe {} for maps and [] for lists.
| milliams wrote:
| I agree. They make a rod for their own back by having
| implicit conversion rules which, while well defined, are not
| well understood. "Explicit is better than implicit" and all
| that.
| progx wrote:
| Advantage: things like "literal_block: |" or ">" are not
| necessary with a simple quote.
|
| And i really don't know, why in hell they add binary-Data
| to a configuration language for humans.
| anonumous wrote:
| [flagged]
| yccs27 wrote:
| My wish for a dream config language is this: Allow a choice
| between unambiguously-typed expressions (quoted strings, only
| true/false, decimal numbers) and explicit type annotations. So
| this: regions: $[string] - no
| - se options: ${bool} a: yes b: no
|
| (with possibly a different syntax) would be equivalent to
| regions: - "no" - "se" options:
| a: true b: false
| deafpolygon wrote:
| I will stick to JSON or TOML if the choice was mine. Hell, I'd
| much rather deal with XML than deal with YAML.
| messe wrote:
| Just say Norway to YAML!
| r3trohack3r wrote:
| > loading an untrusted yaml document is generally unsafe,
|
| TIL and this is... Not great?
|
| I often parse JSON as YAML. Since YAML is a superset of JSON, all
| valid JSON files parse as a valid YAML file too, but now you can
| add comments to your JSON since YAML has comments. Didn't know I
| was opening my process up to code execution doing this...
|
| Knowing this, YAML is dead to me as a format. This really seems
| unacceptable for a markup parser.
| kroltan wrote:
| To be fair, YAML itself does not specify how tags should be
| interpreted, technically the idea of grabbing constructors from
| those is the library's issue. But then again, you wouldn't want
| to accidentally introduce code execution when you adopt a new
| library/language.
| Beltalowda wrote:
| Pretty much all parsers have some way to "safely" parse a YAML
| document which disables the custom-object unmarshal stuff;
| sometimes this is the default, sometimes not, some
| libraries/languages don't support this at all in the first
| place.
|
| If it's enabled, then it's trivial to exploit; e.g. in Python:
| !!python/object/apply:os.system args: ['ls /']
|
| If you parse this with "yaml.load()" in PyYAML (which, I
| believe, is the most commonly used YAML library for Python)
| then it will execute that code. You will need to use
| "yaml.safe_load()".
|
| I believe the situation in Ruby is similar, and in PHP the only
| way to get any safe sane behaviour is to modify a php.ini
| setting (which at least defaults to "off" now, but there is no
| way to know if a line is safe just by looking at the code).
|
| In Go this entire feature isn't supported at all AFAIK, so it's
| always safe.
|
| Not sure about other languages from the top of my head.
| twic wrote:
| I'm not sure it's true that parsing an untrusted YAML document
| is generally unsafe. That has to be a property of the parser,
| not the format.
|
| As TFA says:
|
| > In Python, you can avoid this pitfall by using yaml.safe_load
| instead of yaml.load
|
| In Ruby, there are now analogous options (including - of course
| lol - a gem which monkeypatches the YAML module to make it safe
| by default) [1]:
|
| > Use the safe-yaml gem which overrides YAML.load
|
| > Require a newer version of Psych that provides a safe-load
| option
|
| In Java, SnakeYaml has a document about security which i think
| is a long way of admitting that it's not secure [2]. It looks
| like eo-yaml just doesn't support tags at all.
|
| [1] https://trailofbits.github.io/rubysec/yaml/index.html
|
| [2]
| https://bitbucket.org/snakeyaml/snakeyaml/wiki/CVE%20&%20NIS...
| xg15 wrote:
| > _Json is so obvious that Douglas Crockford claims to have
| discovered it -- not invented._
|
| I mean, it's called "JavaScript Object Notation" for a reason.
| From what I know, the beginnings of it really were in the early
| days of AJAX: People were looking for a good way to encode
| complex data structures in dynamically generated script tags.
|
| Then someone stumbled over essentially a random syntax element in
| JavaScript that turned out to be extremely well-suited for that
| task.
|
| So I think talking about "discovering" JSON as a JS syntax
| feature which later became its own thing is quite correct here.
| adenozine wrote:
| Every day we stray farther from the light of XML.
|
| At this point, it's just plain ridiculous.
| jwilk wrote:
| > yaml is a superset of json
|
| This is oft-repeated, but incorrect claim:
|
| https://news.ycombinator.com/item?id=30052633
| InsomniacL wrote:
| "YAML is Human-readable"
|
| "You must count whitespace for nesting"
|
| Pick one or the other, but not both.
| jeroenhd wrote:
| You don't need to count whitespace for nesting:
| test: something: complicated:
| {even: { more: { levels: {can: { be: {achieved: {like: {this:
| true } } } } } } } }
|
| Whitespace nesting also doesn't seem to be a problem for Python
| programmers. I don't prefer it myself (especially because of
| the tabs vs spaces problem) but it's not good or bad.
| [deleted]
| [deleted]
| privacywiki wrote:
| Ansible collection writer here, pray for me...
| carrja99 wrote:
| We need to go back to xml. Admit it.
___________________________________________________________________
(page generated 2023-01-12 23:01 UTC)