[HN Gopher] ABI compatibility in Python: How hard could it be?
       ___________________________________________________________________
        
       ABI compatibility in Python: How hard could it be?
        
       Author : lukastyrychtr
       Score  : 41 points
       Date   : 2022-11-18 15:01 UTC (1 days ago)
        
 (HTM) web link (blog.trailofbits.com)
 (TXT) w3m dump (blog.trailofbits.com)
        
       | frozenport wrote:
       | One reading is that despite declared mismatches between versions.
       | There were no actual instances of incompatibility found?
        
         | SAI_Peregrinus wrote:
         | Using symbols from a newer Python version with an older ohe
         | declared _is_ an incompatibility, if run on an older Python
         | install.
        
       | selimnairb wrote:
       | May take on this, as a Python specialist who maintains a C++
       | library that exposes Python bindings (but not an expert in such),
       | is that the Python packaging ecosystem (as it has organically
       | grown) is a mess. I would like an authoritative (i.e., in the
       | standard library) way to accomplish the major packaging tasks. I
       | shouldn't have to track down the current canonical way to do
       | things from two or three sources.
        
         | [deleted]
        
         | noisenotsignal wrote:
         | Agree. I had a build pipeline pulling down an internal package
         | to build into a `.pex` [0] file for running a Python program in
         | a release pipeline that did not have access to the internal
         | package repository. I came up with this solution because the
         | typical virtualenv that you might build locally does not
         | appreciate being copied wholesale to another system (iirc it
         | had some absolute paths).
         | 
         | One day the release pipeline broke because a dependency's
         | authors published a new version containing a wheel for a newer
         | ABI, though the older ABI was still supported. My build
         | pipeline pulled the one for the later ABI because it was
         | compatible, but the release pipeline environment, which could
         | not be upgraded, did not have the newer version of gcc
         | required. It was a nasty introduction to the fact that one
         | Python package version can have multiple published wheels
         | targeting different C++ compilers!
         | 
         | But the worst part was not that I learned something new! It was
         | that I wrangled with both pex and pip for a few hours trying to
         | figure out how to download a wheel for an older ABI as both
         | tools resolve the latest compatible by default. The options to
         | do so are ostensibly there but the effect was not. I definitely
         | could have made some mistakes but in my mind it shouldn't be
         | _hard_ to get it right.
         | 
         | I ended up downloading the wheel for the internal package in
         | the build and copying it directly into the artifact. The
         | release pipeline resolved public dependencies and installed the
         | internal package from the local file.
         | 
         | [0] - https://pex.readthedocs.io/en/latest/
        
           | orf wrote:
           | The release pipeline broke because you didn't lock your
           | dependencies.
        
         | inferiorhuman wrote:
         | It always struck me as a bit ironic that such an opinionated
         | language could utterly fail to have "one right way" to package
         | things. As much as Rust's aversion to binaries can be a pain,
         | building and packaging (even cross building) is generally
         | painless.
        
       ___________________________________________________________________
       (page generated 2022-11-19 23:00 UTC)