[HN Gopher] Preserves: An Expressive Data Language
___________________________________________________________________
Preserves: An Expressive Data Language
Author : mpweiher
Score : 66 points
Date : 2025-01-29 12:30 UTC (10 hours ago)
(HTM) web link (preserves.dev)
(TXT) w3m dump (preserves.dev)
| djoldman wrote:
| Here is the ABNF:
|
| https://preserves.dev/preserves-text.html
| tonyg wrote:
| Or, in "quick reference card" form:
| https://preserves.dev/cheatsheet.html
|
| The syntax isn't the most interesting part though; the thing
| that distinguishes it from most other data languages out there
| is that it has _semantics_ (= a rigorous definition of when
| values are equal and when they aren 't). So you can use
| Preserves semantics with JSON syntax (a subset of Preserves'
| text syntax) as one way of getting actually-meaningful JSON.
|
| Plus, comments (and other annotations) ;-)
| lionkor wrote:
| Why/when/where would I need this?
| tonyg wrote:
| Useful if you have a JSON-keyed table, for example: JSON lacks
| a useful (standardised) equivalence relation, meaning you get
| weak and/or implementation-specific guarantees about how key
| lookup works. Equivalences were the motivation for developing
| Preserves: I was (and am) working on a generalized approach to
| messaging middleware, you might say, meaning that things like
| "patterns over values" and "filters" and "value-keyed tables"
| are all things I need to talk about. (This all comes out of
| RabbitMQ/AMQP thinking back in the day and my PhD-and-after
| work subsequently.)
| paddy_m wrote:
| I was going to ask the exact same question. The title makes it
| sound like something I might be interested in, then I visited
| the page and I have no idea what it does.
|
| After some brief reading of docs, I'm trying to write one
| sentence explanations. Maybe this will be helpful to you
|
| What
|
| Preserves is a specification and set of libraries in popular
| languages that lets you reliably exchange data between XML,
| JSON and EDN.
|
| Who
|
| Preserves is built for (data engineers|data framework writers)
| to reliably interchange data.
|
| Why
|
| Formats like JSON in particular are imprecise. Preserves forces
| you to deal with these vagaries up front
|
| What else?
|
| With P-Expressions you can search a preserve compliant
| datasource much like you would query JSON with JQ
|
| Who Not? Who shouldn't use this
|
| This will not help a data analyst exchange data between CSV and
| Excel
| lionkor wrote:
| Thank you! That makes more sense now
| paddy_m wrote:
| I have know idea if the project author would agree with
| those sentences, I was just proposing them.
| boxed wrote:
| > This is a good time to mention that even though from a semantic
| perspective sets and dictionaries do not carry information about
| the ordering of their elements.
|
| Except they do in Python. It is extremely useful, surprisingly
| often.
| tonyg wrote:
| Python remembers order, and exposes it in its iterations, but
| doesn't use it in its equivalence over dictionaries (==
| semantics).
|
| (ETA: What are you quoting there? I don't think that text
| appears on the Preserves site) (ETA2: Ah, it's the tutorial.
| Cool)
| sixdimensional wrote:
| Not sure about the previous post, but also python's
| OrderedDict collection guarantees order-sensitive equality
| checks. [1]
|
| Plain dict maintains insertion order but equality checks only
| check that the key/value pairs are the same. [2] [3]
|
| [1] https://docs.python.org/3/library/collections.html#:~:tex
| t=e...
|
| [2] https://docs.python.org/3/library/stdtypes.html#:~:text=d
| ict...
|
| [3] https://docs.python.org/3/library/stdtypes.html#:~:text=d
| ict...
| dangoodmanUT wrote:
| gotta have some examples on the landing before the fold,
| otherwise i have no reason to explore
| mhitza wrote:
| https://preserves.dev/TUTORIAL.html
|
| Not a fan of annotation (that can be used as comment syntax)
| having # followed by a space character have a different
| behaviour feels strange.
| porcoda wrote:
| Tutorial link too hard to click? Seriously: these comments
| inevitably come up for most language related pages, and I don't
| see how they don't fall under the site guidelines of "Please
| don't post shallow dismissals".
| yencabulator wrote:
| The comment specifically said "on the landing before the
| fold", so yours is the shallow dismissal. I agree with the
| comment; lead with the things we shall judge ye by.
| paddy_m wrote:
| I made a similar comment. I think they come up because people
| are genuinely interested in a project and trying to offer the
| creator a fresh perspective. When creating a project where
| your the domain expert it's so easy to get stuck in your own
| head, and then you start explaining the project to a newcomer
| diving into deep details when they don't understand the
| starting point.
|
| Until a project has a lot of traction (think docker, react,
| django not uv or jq) it's very safe to assume that every
| visitor to your page doesn't understand the background.
| conartist6 wrote:
| CSTML is targeting many of the same weaknesses in JSON. It's fun
| to see a whole different, competing set of design choices at
| work. I had a very different take on schema validation and how to
| use the < syntax.
| tonyg wrote:
| Do you have a link for CSTML, please? Googling is showing a
| bunch of possibilities none of which look quite relevant enough
| to be right...
| conartist6 wrote:
| https://github.com/bablr-lang/
|
| https://bablr.org/playground
| tonyg wrote:
| Thank you!
| fjfaase wrote:
| Is it true that Records are the same as Dictionaries, because the
| labels in the records can have any value?
|
| Interesting on how on one hand the size of SignedInteger is
| unlimited, but on the other hand there is a ByteString. A
| ByteString could also have been represented by as a sequence of
| SignedInteger. I also wonder if it would not better to have a
| Unicode character as an atomic unit and represent a string as a
| sequence of Unicode characters.
|
| This makes me wonder whether this is a high-level data model or
| yet another data representation.
| tonyg wrote:
| No, a record is a tagged (sequence of) value(s).
| <tag v1 v2 v3>
|
| If you put a single dictionary-valued "field" in a record, you
| get a variation with named fields <tag {
| field1: value1 field2: value2 field3: value3
| }>
|
| Records have positional "fields" because of the Scheme heritage
| of the design.
|
| --
|
| Re bytestring -- yes there are some concessions to real
| machines/languages in there that aren't absolutely required.
| Other examples include booleans and strings, which could have
| been <true> and <false> and <string [65 66 67]> etc
| respectively.
|
| There's a little more on this topic in footnote 2 on the
| "conventions" page:
| https://preserves.dev/conventions.html#fn:why-dictionaries
| carterschonwald wrote:
| Looks like a lot of excellent work!
|
| Are there any good examples of nontrivial schemas etc?
| ceving wrote:
| I can not find an Emacs mode. Does anybody know one?
| tonyg wrote:
| https://gitlab.com/preserves/preserves/-/blob/main/preserves...
| -- crude but effective! (I use it all the time)
| twism wrote:
| So EDN?
| tonyg wrote:
| Yeah EDN is quite similar. Preserves has no nil, allows any
| value as a tag, gets into the weeds more on when strings are
| equal or not, doesn't distinguish lists and vectors, and
| doesn't require each kind of tagged element to define an
| equivalence. And it has annotations (vs EDN's comments) and
| embedded values.
| mcphage wrote:
| It seems like it doesn't natively support decimals (needed for
| financial data) or any sort of date or datetime?
| layer8 wrote:
| Looking at the headings in that TOC, "Preserves" is a bit of an
| unfortunate naming choice grammatically.
| account-5 wrote:
| Might be an ignorant question but why not just use XML? It seems
| like XML could do all this, from my limited reading?
| yegle wrote:
| Does Preserves have a page with a comparison to other common
| serialization languages?
|
| As someone familiar with Protobuf, comparing Preserves vs
| Protobuf text format, here's my quick comparison between the two
| after reading through the tutorial:
|
| - Preserves' Symbol is very close to Protobuf enums. But Symbol
| can contain characters like dash
|
| - There doesn't seem to be an equivalent of Preserves' Record in
| Protobuf, but the tutorial's example of using <Unknown ...> To
| denote a missing <Date ...> can be simulated using the `oneof`
| field in Protobuf.
|
| - Having to write #t/#f in Preserves is unfortunate. I guess this
| is the result of schemaless serialization language and potential
| parsing ambiguity with a Symbol?
|
| - Protobuf have a way to annotate the schema and reuse at
| runtime, very similar to Preserves' annotations.
| yencabulator wrote:
| So `#` starts a line comment but `#t` is a boolean. Yeah, that's
| never gonna hurt anyone.
___________________________________________________________________
(page generated 2025-01-29 23:01 UTC)