[HN Gopher] A New ASN.1 API for Python
___________________________________________________________________
A New ASN.1 API for Python
Author : woodruffw
Score : 132 points
Date : 2025-04-18 14:11 UTC (8 hours ago)
(HTM) web link (blog.trailofbits.com)
(TXT) w3m dump (blog.trailofbits.com)
| woodrowbarlow wrote:
| neat!
|
| related: you can also create wireshark dissectors from ASN.1
| files
|
| https://www.wireshark.org/docs/wsdg_html_chunked/ASN1StepByS...
| venamresm__ wrote:
| In the ASN.1 space everyone hopes that someone can dethrone OSS
| Nokalva's proprietary solutions
| woodruffw wrote:
| I think it's context-dependent: I don't have insight into OSS
| Nokalva's use inside big companies, but in the Open Source
| world it certainly isn't dominant.
|
| In Open Source, I think the biggest ASN.1 implementations I
| come across are OpenSSL's, libtasn1, asn1c, and then per-
| language implementations like pyasn1.
| nicce wrote:
| Basically any commercial ASN.1 compiler prevents usage of the
| output in any open-source project. There is that.
| sobkas wrote:
| Licence also prevent you from modifying generated code.
| memling wrote:
| > In the ASN.1 space everyone hopes that someone can dethrone
| OSS Nokalva's proprietary solutions
|
| You're buying more than a compiler and runtime, though: you're
| also getting an SLA and a stricter guarantee about
| interoperability and bugs and so forth. I have no idea how good
| their support is (maybe it's atrocious?), but these are
| important. I had a client who relied on the open-sourced asn1c
| once who complained about some of the bugs they found in it;
| they got pushed into buying commercial when the cost-benefit
| outweighed the software licensing issues.
| johnisgood wrote:
| Erlang also has great ASN.1 support. For the rest, I hope OSS
| Nokalva'a proprietary solutions will go away, eventually.
|
| For Java I used yafred's asn1-tool, which is apparently not
| available anymore. Other than that, it worked well.
|
| Originally it was available here:
| https://github.com/yafred/asn1-tool (archived:
| https://web.archive.org/web/20240416031004/https://github.co...)
|
| Any recommendations?
| elFarto wrote:
| I was using the asn1bean Java library yesterday funnily enough.
| I'm sure it's fine for X.509 stuff, however lucky me got to use
| it with the more obscure parts of X.400. It's lacking support
| for COMPONENTS OF, and a bunch of other things that were likely
| deprecated from the ASN.1 spec a few decades ago.
| johnisgood wrote:
| Any luck with it?
|
| Check the README of: https://web.archive.org/web/202404160310
| 04/https://github.co...
|
| I need something like this.
| nightpool wrote:
| https://mvnrepository.com/artifact/de.rub.nds/asn1-tool links
| to https://github.com/tls-attacker/ASN.1-Tool which redirects
| to https://github.com/tls-attacker/ASN.1-Attacker, so that
| seems like it's the new name for the tool you're looking for.
| johnisgood wrote:
| No, it is not the same. Sadly the author probably made it
| private, might reach out to him, just don't know how. :( He
| did not put his e-mail anywhere from what I have found so
| far.
|
| https://github.com/zhonghuihuo/asn1-tool is still available,
| but it is very old, it is probably a VERY old fork.
|
| I need something like this :(. I need it for Java / Kotlin. I
| do not have the repository cloned, so I am kind of in the
| dark.[1]
|
| Found the archived page of the previously mentioned project
| that is probably private now: https://web.archive.org/web/202
| 40416031004/https://github.co...
|
| [1] Never mind, I found the newest (probably)
| asn1-compiler.jar! I still need an actively maintained
| alternative, however, for Java / Kotlin. For ASN.1 between
| Erlang / Elixir <> Java / Kotlin.
| nailer wrote:
| I used Peculiar Ventures ASN1.js to build an in-browser PKI
| platform years ago. It can sit on top of webcrypto and do
| everything you need in terms of managing TLS certs.
|
| https://asn1js.org/
|
| https://www.npmjs.com/package/asn1js
| nicce wrote:
| There is also rasn library for Rust that now supports most of the
| codecs (BER/CER/DER/PER/APER/OER/COER/JER/XER).
|
| Disclaimer: I have contributed a lot recently. OER codec (modern
| flair of ASN.1) is very optimized (almost as much as it can be
| with safe Rust and without CPU specific stuff). I am still
| working with benchmarking results, which I plan to share in close
| future. But it starts to be the fastest there is in open-source
| world. It is also faster than Google's Protobuf libraries or any
| Protobuf library in Rust. (naive comparison, no reflection
| support). Hopefully other codecs could be optimized too.
|
| [1] https://github.com/librasn/rasn
| dikei wrote:
| DER is still easy, UPER (unaligned packed encoding rules) is so
| much harder, yet it's prevalent in Telecom industry. Last I
| checked, there was no freely available tool than can handle UPER
| l00%
| bryancoxwell wrote:
| Not only is UPER hard to parse, but (I believe) 3GPP ASN1
| definitions are provided only in .docx files which aren't
| exactly the easiest to work with. It's just really not a fun
| domain.
| jeroenhd wrote:
| The ASN.1 format itself isn't too bad. It shows its age and
| has some very weird decisions behind it as places, but it's
| not that difficult to encode and is quite efficient.
|
| Unfortunately, the protocols themselves can be confusing,
| badly (or worse: inconsistently) documented, and the binary
| formats often lack sensible backwards compatibility (or, even
| worse: optional backwards compatibility). Definitions are
| spread across different protocols (and versions thereof) and
| vendors within the space like to make their own custom
| protocols that are like the official standardised protocols,
| but slightly different in infuriating ways.
|
| If you parser works (something open source rarely cares about
| so good luck finding one for your platform), the definitions
| extracted from those DOCX files are probably the least of
| your challenges.
| sobkas wrote:
| First you can download specifications in either PDF or
| doc(x). Second doc(x) are simple enough that simple doc(x) to
| ASCII/text is good enough to produce working ASN.1
| definition. Copy&paste is also an option.
| masklinn wrote:
| FWIW rasn linked above claims to support UPER, but I couldn't
| tell you how completely.
| nicce wrote:
| There are many tools that can handle UPER up to certain level
| (some rare ASN.1 types might not be supported). I think the
| main issue is not in the codec, rather the lack of compilers
| that can create correct language-level representation of the
| ASN.1 definitions. 3GPP specifications are enormous and you
| don't want to create them by hand. ASN.1 has some very
| difficult notations, e.g. inner subtype constraints and
| information object classes. Subtype constraints may affect
| for the encoding output in UPER and if you are not
| representing them correctly overall, then you are not
| compatible.
| flowerthoughts wrote:
| How come they don't (just) apply zlib on DER? Is telco
| equipment able to stream process UPER without buffering more
| than non-constructed values?
| eqvinox wrote:
| PER were defined in 1994; back then applying zlib wasn't
| something you "just" do. Modern use is backwards
| compatibility (or cargo cult.)
| nicce wrote:
| UPER is extremely compact encoding format. It still makes
| sense to use UPER, because after all, it is an
| international standard and telecommunication protocols
| itself are supposed to add as little overhead on top of
| actual payload as possible.
|
| For example, if you have ASN.1 UTF-8 string that is
| constrained to 52 specific characters - UPER encoding can
| present every character with 6 bits (not bytes).
|
| In modern world you can apply zlib on top of UPER encoding
| or internal payload, however, depending on the use case.
| userbinator wrote:
| When every bit passing through the network gets charged (if
| not to the customer, then it's taking up capacity that could
| otherwise be charged to the customer), and the software in
| the endpoints needs to be as low-power as possible, zlib is
| additional overhead you definitely don't want.
| nly wrote:
| asn1c claims to support "unaligned basic PER"
| ChuckMcM wrote:
| A small bit of historical context. When I was participating in
| the PKP meetings at RSADSI, I believe it was Ron who insisted
| that DER was the only reasonable choice if we were going to
| encode things with ASN.1 (which we were because both DEC and RSA
| had already insisted that it had to be OSI compatible or they
| wouldn't support it, my suggestion that we use Sun's XDR was
| soundly rebuked, but hey I had to offer)
|
| Generally it was presumed that because these were 'handshake'
| type steps (which is to say the prelude to establishing a
| cryptographic context for what would happen next) performance
| wasn't as important as determinism.
| OhMeadhbh wrote:
| oh. did i meet you there? i was contracting at RSADSI at the
| time and argued w/ Burt K. about how easy it was to mess up a
| general DER parser, much less an ASN.1 compiler. I remember we
| found about two bugs per week in ICL's compiler. Burt and Ron
| were BIG ASN.1 fans at the time and I could never figure out
| why. Ron kept pushing Burt and Bob Baldwin to include more
| generic ASN.1 features in BSAFE. Part of my misery during SET
| development can be directly traced to ICL's crappy ASN.1
| compiler, yet it was probably the best one on the market at the
| time.
|
| Anywho... XDR isn't my favourite, but I would have definitely
| preferred it to DER/BER/ASN.1.
|
| Stop me before I make a CORBA reference.
| tptacek wrote:
| One of the few concessions I'll make to Sun: XDR was under-
| appreciated.
| flowerthoughts wrote:
| Related: if you ever want to create your own serialization
| format, please at least have a cursory look at the basics of
| ASN.1. It's very complete both in terms of textual descriptions
| (how it started) and breadth of encoding rules (because it's
| practical.)
|
| (You can skip the classes and macros, though they are indeed
| cool...)
| tptacek wrote:
| This sounds dangerously like a suggestion that more people use
| ASN.1.
| dec0dedab0de wrote:
| Does anyone miss when "pure python" was a selling point of your
| library? I understand the need for speed, but I wish it were more
| common to ship a compiled binary that does the thing fast, as
| well as the same algorithm in python to fall back on if it's not
| available.
| pphysch wrote:
| I certainly don't miss needing to install additional system
| libraries in addition to my pip install.
| pjmlp wrote:
| I rather find tragic that contrary to other dynamic languages,
| Python seems to fall under the curse of rewriting bindings into
| C and C++, or nowadays more fashionable Rust.
|
| And yes, Smalltalk, Self and various Lisp variants are just as
| dynamic.
| woodruffw wrote:
| Why is it tragic? It's more or less idiomatic in Python to
| put the hot or performance-sensitive paths of a package in
| native code; Rust has arguably made that into a much safer
| practice.
| foolswisdom wrote:
| It's part of the original selling points of python, so it's
| not surprising that we've never stopped doing it.
| sharperguy wrote:
| They should just distribute it in some kind of bytecode
| compiled language with JIT VM like java. Then at least it will
| be cross platform.
| smnrchrds wrote:
| Pure Python was a huge selling point back when using a compiled
| library involved downloading and running random .exe files from
| someone's personal page on a university website. It is much
| less of a selling point now that we have binary wheels and
| uv/Poetry/etc. that create cross-platform lock files.
|
| I feel nostalgic seeing (a mirror of) that download page again,
| but that era was such a pain.
|
| Mirror: http://qiniucdn.star-
| lai.cn/portal/2016/09/05/tu3p2vd4znn
| dec0dedab0de wrote:
| I always thought the selling point of Pure Python was that
| you might be running on some esoteric implementation of
| python, or hardware that the library maintainer didn't
| include binaries for.
|
| I mean, I am glad wheels exist, they make things a lot easier
| for a lot of people. I just believed in the dream of
| targeting the language for maximum compatibility and hoping a
| better implementation would eventually run it faster.
| orthecreedence wrote:
| I was writing a cryptographically-inclined system with
| serialization in msgpack. At one point, I upgraded the libraries
| I was using and all my signatures started breaking because the
| msgpack library started using a different representation under
| the hood for some of my data structures. That's when I did some
| research and found ASN.1 DER and haven't really looked back since
| switching over to it. If you plan on signing your data structures
| and don't want to implement your own serialization format, give
| ASN.1 DER a look.
___________________________________________________________________
(page generated 2025-04-18 23:00 UTC)