[HN Gopher] Pure: A static analysis file format checker
       ___________________________________________________________________
        
       Pure: A static analysis file format checker
        
       Author : slimsag
       Score  : 93 points
       Date   : 2022-06-25 06:15 UTC (2 days ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | robertlagrant wrote:
       | This looks really interesting; can an end user make use of it
       | directly? Or is it more for providers of things?
        
         | jorangreef wrote:
         | You can run it as a CLI directly on any ZIP file that you pass
         | by path. I do want to port it to Zig though, to get rid of the
         | NPM install bootstrapping, have a single binary, and also gain
         | Zig's checked arithmetic and spatial safety.
        
       | tyingq wrote:
       | Seems interesting to build something looking for things like zip
       | file bombs, unexpected sparseness, overflow/underflow, and so on
       | with C.
        
       | ducktective wrote:
       | Is there something like this for formats other than ZIP like jpeg
       | or pdf?
        
         | jwilk wrote:
         | For tar: https://github.com/ferivoz/alquitran
        
         | jorangreef wrote:
         | I wanted to do this for those formats in particular, as well as
         | Office files. Fastmail were going to sponsor some of this work
         | to extend Pure, and I was chatting with the folks at Hey, but I
         | couldn't get enough other sponsors to fund the work.
         | 
         | The idea is to use Pure to have a "Zero-Day Defense Mode"
         | button for your email provider that you can push, if you want
         | static analysis on 90% of the file formats coming and going, so
         | that you can have stronger assurance on email attachments that
         | you open. For example, I believe that Pure would have prevented
         | last year's zero-click traversal against Apple Mail.
         | 
         | There are also so many ways to use file format anomalies to
         | take out AV engines, so Pure can act as a first line of
         | defense, before hostile data gets to them. For example,
         | protecting them from the David Fifield zip bomb.
         | 
         | What I find most interesting with these checks, is that they're
         | looking for explicit bleeds or explicit overflow bounds. So the
         | signal to noise ratio is high.
         | 
         | Mostly, if people are interested, I would really love to move
         | this all to Zig, to benefit from the checked arithmetic and
         | spatial safety, since otherwise everything is all single-
         | threaded run-to-completion.
        
           | [deleted]
        
         | jorangreef wrote:
         | PDF is also a minefield, for how many ways there are to mess
         | with the format: https://blog.didierstevens.com/2008/04/29/pdf-
         | let-me-count-t...
        
         | tgv wrote:
         | Idk, but there's an interesting project, wuffs: a programming
         | language specifically aimed at efficiently and safely parsing
         | document structure. There is a PNG decoder [2].
         | 
         | 1. https://github.com/google/wuffs
         | 
         | 2. https://news.ycombinator.com/item?id=26714831
        
       | jwilk wrote:
       | Backstory: https://news.ycombinator.com/item?id=23760809
        
         | jorangreef wrote:
         | Thanks for the link to that comment of mine. Wow, that's a
         | blast from the past!
         | 
         | Here's also a recent HN discussion about Pure, in the context
         | of buffer bleeds and memory safe languages, with tptacek:
         | https://news.ycombinator.com/item?id=31852820
        
       | fbanon wrote:
       | What does "static analysis" mean in this context?
       | 
       | In program analysis it means that you're deducing properties of
       | the program without executing it.
        
         | frabert wrote:
         | I think it's meant in the sense of "can detect the exploits
         | without having to actually attempt decompressing anything"
        
           | jorangreef wrote:
           | Exactly, although, at present, Pure will also do dynamic
           | analysis on the compressed stream once the metadata checks
           | out, to verify that the compressed stream checks out with the
           | metadata, i.e. that there's no semantic gap between the
           | designated size and the actual stream size. This guards
           | against one category of ZIP bomb.
        
       | jorangreef wrote:
       | What a surprise to open up HN today and see Pure staring back at
       | me!
       | 
       | Here's the HN thread from 2019 where rattray encouraged me just
       | to go for it and open-source the original JavaScript version:
       | https://news.ycombinator.com/item?id=20352439
       | 
       | That led to a contract with Microsoft, which then led to my work
       | at Coil on TigerBeetle, a distributed database written in Zig:
       | https://www.youtube.com/watch?v=rNmZZLant9o
        
       ___________________________________________________________________
       (page generated 2022-06-27 23:02 UTC)