[HN Gopher] Briefcase: Convert a Python project into a standalon...
___________________________________________________________________
Briefcase: Convert a Python project into a standalone native
application
Author : generichuman
Score : 157 points
Date : 2023-08-03 11:01 UTC (12 hours ago)
(HTM) web link (briefcase.readthedocs.io)
(TXT) w3m dump (briefcase.readthedocs.io)
| yellowapple wrote:
| > If you're on Linux, you should use the system Python provided
| by your operating system. You will be able to complete most of
| this tutorial using a non-system Python, but you won't be able to
| package your application for distribution to others.
|
| Why not?
| packetlost wrote:
| Probably because it doesn't actually package the Python
| interpreter into the executable and relies on a hashbang (ie.
| `#!/bin/python`) to actually run the bundled source files. Pex
| [0] does the same thing.
|
| It's a half solution at best, but if you control the
| infrastructure it does dramatically simplify things like
| Dockerfiles and other deployment stuff. Not super useful for
| general distribution IMO.
|
| [0]: https://pex.readthedocs.io/en/v2.1.140/
| yellowapple wrote:
| That's a bummer. An app being self-contained is kinda the
| point of an AppImage IMO, and depending on an external Python
| (if one even exists) throws a wrench in that.
| hackan wrote:
| It's a cool trick, I used it to pack a tool long time ago:
| https://github.com/HacKanCuBa/passphrase-
| py/blob/219d6374338...
|
| Basically, u zip the package, then add the shebang on the zip
| binary. Quite cool and simple! No need for external tools :p
| totetsu wrote:
| I ran into ssh cypher issues when I tried to share a packed
| python project. Worked on my machine but not my colleagues
| [deleted]
| synergy20 wrote:
| is this mainly for packaging python GUI programs cross-platform?
| how is it different from pyinstaller with tkinter?
| mike_hearn wrote:
| Interesting timing! We're just in the middle of adding Python
| support to Hydraulic Conveyor, which is a similar tool [1].
| There's a github issue [2] and mailing list that'll get notified
| when it's done. Disclosure: it's commercial but free for open
| source projects.
|
| There are many of these open source packagers and they all share
| very similar problems:
|
| 1. They don't let you do software updates, even though software
| updates are practically mandatory for any real project. Electron
| is a stand-out here because it does address this, but their
| update engines are unmaintained for years and have some major
| unfixed problems (causes a lot of issues with Windows networks,
| for example).
|
| 2. Even in the very rare cases that they do, they don't let you
| force updates on launch even though many apps need something like
| this to keep up with protocol changes. It's one of the reasons
| people like web apps.
|
| 3. They don't help you with signing. For example they don't
| simplify key management, they don't support cloud signing
| (essential since May because Microsoft now insist on HSMs for all
| keys, not just EV keys), or they don't do notarization, or they
| don't generate CSRs for you.
|
| 4. They require the use of CI to cross-build even when apps are
| written in portable frameworks that don't require compilation.
| This is because they are just thin wrappers around the native
| tooling.
|
| 5. They're invariably language specific even though there's no
| good reason to be because 80% of the work is the same regardless
| of what language or framework you use.
|
| 6. They make MSI files for Windows even though MSI is deprecated.
|
| It's possible to bite the bullet, chew glass for a while and
| solve all these problems, which is what we did for
| Electron/JVM/Flutter/native apps. You can reimplement all the
| native tooling so users can cross-build (i.e. make Mac packages
| from Linux/Windows, Windows packages from Mac/Linux etc), which
| enables releasing from developer laptops or cheap Linux CI
| workers. You can support software update by integrating Sparkle
| on macOS, apt on Debian/Ubuntu and by using MSIX on Windows (and
| by then working around all the bugs in Windows to make it work
| well). You can generate download pages that work out the user's
| OS and gives them the right download, and instructions for how to
| install self-signed apps if the developer isn't code signing with
| a recognized certificate. You can abstract platform neutral
| things and expose platform specific things. Then you can write a
| parallel incremental build system so doing all the work is as
| fast as possible, and write lots of code to detect all the myriad
| mistakes people make and give good error messages or auto-fix
| them. Then you can make it support GitHub Releases. Then you can
| document it all.
|
| But that big pile of glass isn't particularly tasty, which is why
| open source projects don't do it and we ask commercial users to
| pay for it.
|
| Briefcase looks nice but it also seems to have all the problems
| listed above. I think once we add Python support Conveyor will be
| quite useful for the Python community, especially if we can find
| a workaround for pip not support cross-building of venvs. It
| would be great if you could just whip up a quick Python script,
| run one command and your installed clients start automatically
| updating, your download page updates, and the whole thing is no
| harder than releasing a static markdown-rendered website.
|
| [1] https://hydraulic.dev/
|
| [2] https://github.com/hydraulic-software/conveyor/issues/73
| zie wrote:
| How do I get notified when you have python working? I want on
| that list.
| mike_hearn wrote:
| Follow the GitHub issue or enter your email address in
| "Subscribe to the blog" on https://hydraulic.dev/
| ptx wrote:
| > _MSI is deprecated_
|
| Is it? The WiX Toolset still seems to be actively developed and
| MSIX doesn't seem to support all use cases. Has MSI been
| officially deprecated despite that?
|
| Also, MSIX's mandatory signing makes it too expensive and
| cumbersome for a hobby project, so it would be unfortunate if
| that became the only option.
| mike_hearn wrote:
| It was marked as such at one point on their website, but now
| I searched for that again I can't find it anymore. Maybe they
| undeprecated it, hard to keep track of what MS are doing.
| It's not been developed for many years though. All the new
| features seem to go into MSIX.
|
| MSIX files can be self-signed if you install the cert first.
| Conveyor has an EXE wrapper that can do that for you. So you
| can distribute apps with it without buying signing
| certificates.
| ptx wrote:
| But by installing a self-signed certificate, that
| certificate is trusted for _all_ apps, not only for mine -
| right? This seems like a broken permission model.
|
| I only want permission to modify my own app, not every app
| of the system. If users have to trust me (and all other
| developers of self-signed applications) with permission to
| modify _all_ apps, that makes the whole signing rigamarole
| kind of pointless.
|
| But if I do want to try this approach, do I have to install
| the certificate in Trusted Root Certification Authorities,
| or should it be in Trusted Publishers, or Trusted People?
| The first one would grant me even more privileges that I
| don't need, but what about the other two?
| mike_hearn wrote:
| If you publish through the MS Store, it only costs $19 to
| verify an individual (one off, not a subscription) and
| they sign it for you. So that's pretty cheap, well within
| the range of a hobby project.
|
| If you don't have some kind of signed identity then the
| concept of "your own" app doesn't exist, so indeed in
| that model you get permission to do what you want more or
| less. That's why it requires administrator elevation,
| whereas real signed packages don't. Of course this model
| is identical to the normal pre-MSIX model where any app
| installer can fiddle with any other app's files.
|
| You can put it in Trusted People, I think. I'd have to
| check the code to see where we put it.
| formerly_proven wrote:
| > essential since May because Microsoft now insist on HSMs for
| all keys
|
| (Except their own authentication keys, of course)
| mike_hearn wrote:
| I'd be amazed if Microsoft held any important keys outside of
| HSMs. I think the incident you're referring to is where they
| appear to have been hacked by China (maybe the Chinese govt)
| and they blamed a "validation error in Microsoft code". HSMs
| help a bit but if the code that accesses them is insecure
| then you can still sign bad things. The HSM just means you
| don't have to do a key rotation to recover.
|
| On that topic, one feature Conveyor has that open source
| packaging tools usually don't is that it can protect signing
| keys in your macOS keychain, in some cases. To utilize that
| the packaging tool itself must be signed, which of course it
| is here, and this is useful because the OS will protect both
| the private key and the process that's working with it from
| malware even if it's not explicitly sandboxed. The keys
| themselves are protected using the security chip. So it's a
| very high level of security that's sort of equivalent to
| having a dedicated user account for signing on Linux, but
| without needing to set anything up.
|
| Unfortunately not every credential can be stored there yet,
| only the "root key" from which other keys are derived. Login
| credentials for Apple's notarization servers and cloud
| signing services like eSigner/KeyLocker don't get protected
| in the same way yet. But that'll come.
| doctorpangloss wrote:
| How do I turn a setuptools setup.py entrypoint into a standalone
| .exe for Windows users?
| nu11ptr wrote:
| How does this compare to PyInstaller or other packagers? What
| underlying tech does it use?
| WillAdams wrote:
| The big difference is it uses "Toga"
|
| https://beeware.org/project/projects/libraries/toga/
|
| which uses native widgets and manages basic functionality such
| as where the "Quit" menu entry is automatically.
|
| Looking into this now --- it looks perfect for my needs, just
| hoping that there's a nice graphical tool for laying out a GUI
| (if it has a graphical programming mode like to Blockly or
| Ryven, then it is perfect for me).
| fock wrote:
| so essentially what Kivy did already a bit ago.
| nbadg wrote:
| I haven't used Kivy in quite a while, so maybe this has
| changed, but as far as I know -- no, Kivy hasn't done
| something like this. In Kivy, everything looks like Kivy,
| regardless of what platform you're on, and you need to
| explicitly lay everything out, including whichever changes
| you want for a particular platform. Whereas with Toga, the
| UI is changed to match the user's expectations for that
| platform, automatically.
|
| One example of this, as parent mentioned, is where the
| "quit" button is located. Different platforms put this in
| different places. Toga will automatically put it in the
| right one, depending on which platform you're on. Buttons
| will look like normal buttons on a Mac, but also like
| normal buttons on Windows -- depending on which platform
| you're on.
| dr_kiszonka wrote:
| Hi! Do you know whether Kivy or Toga/BeeWare lets you
| access Android API? For example, I want my app to listen
| to BatteryManager's broadcasts [0]. Is it possible using
| any of these frameworks? (Or any other Python-centric
| framework?) I read that Toga supports background tasks,
| but it is not clear to me if I could use them to listen
| to Android broadcasts.
|
| 0. https://developer.android.com/training/monitoring-
| device-sta...
| WillAdams wrote:
| They give an example of using Apple's Obj-C framework, so
| I believe this sort of thing is possible.
| anonymouse008 wrote:
| I'm going to ask a 'self-taught' question, so my terms may be
| incorrect.
|
| Has anyone solved the compiled libraries / dynamic libraries /
| linking problems for multiplatform apps? Everything works just
| fine with these python>native app tools until you hit some
| esoteric library that's not pure python.
| willcipriano wrote:
| Solved as in you don't have to do anything and it just works or
| solved as in there is a way to fix the problem but it requires
| some effort?
| anonymouse008 wrote:
| Ha! Yeah I want the magic wand.
|
| I have deep wounds from pre-chatGPT times of trying to
| manually figure out libraries. As in otooling each library,
| error, etc then recompiling into various places until things
| 'worked.' I'm appreciative of the experience, though now I
| think with LLMs the problem could be solved with some simple
| queries.
|
| So I guess you're right, it requires effort, but I was a 10x
| engineer in the wrong direction and never want to experience
| that again ;)
| willcipriano wrote:
| What works for me is figuring out the platforms I need to
| support and creating a build pipeline that can handle those
| requirements. If you join onto a project I don't want you
| playing around with libraries on your local machine. If you
| are I consider that a failure of the build
| script/makefile/docker compose file/etc. First it's lost
| productivity, second you might get it wrong and have
| misleading results, causing more lost productivity.
|
| A lot of places do this nonsense "spend hours configuring
| your machine" method for projects. The only reason I can
| think of doing that is "job security" and who wants to keep
| a job with horrible tooling?
|
| But yeah, it takes effort to do.
| anonymouse008 wrote:
| That's definitely working smarter. I'm coming around to
| understanding how all the tooling works - the lost weeks
| learning the basics happen less often now.
|
| In a way, that time was a chapter in 'job security.' Ran
| out of money, had to let everyone go, and picked up the
| pieces to fight another day (self-taught under threat of
| survival). By the grace of God I'm a reformed 'idea-guy'
| and am now a builder.
| pjc50 wrote:
| Python remains a mess for these use cases because it does not
| have packages.
|
| It has a very specific filesystem layout that it calls a
| package, but this is not a first class citizen; it has a
| packaging format called "wheel" that is now used for
| distribution, and that can contain native libraries. That
| currently mostly handles the distribution of compiled libraries
| for multiple platforms, but only within the traditional venv-
| style setup. Being able to ship interpreter+venv+launcher would
| go a long way towards providing a native-ish experience.
|
| A step beyond that would be being able to distribute a "virtual
| filesystem" containing a whole tree of python compiled as .pyc,
| appended to a native executable containing the interpreter and
| linked against the required native libraries already.
| simonw wrote:
| I'm a huge fan of https://github.com/indygreg/python-build-
| standalone which provides Python builds that CAN be moved
| around and work independently of any other Python
| installation.
|
| I used that for my own Python+Electron app, which I wrote
| about here: https://til.simonwillison.net/electron/python-
| inside-electro...
| piyh wrote:
| Venvs not being able to be moved to other directories is such
| a PITA.
| plonk wrote:
| They are though, at least on Windows, I've moved them
| before. The whole Lib/ tree was independent of the venv's
| path, it's only the "activate" script that hard-codes a
| full path.
|
| Since venvs on Linux symlink to the system's Python
| executable, they should also work after moving.
| ShamelessC wrote:
| > it's only the "activate" script that hard-codes a full
| path.
|
| So...they aren't then.
| dragonwriter wrote:
| > They are though, at least on Windows, I've moved them
| before. The whole Lib/ tree was independent of the venv's
| path, it's only the "activate" script that hard-codes a
| full path.
|
| So, if activate.ps1 (and the other scripts for other
| shells) were updated to dynamically identify their own
| location rather than hardcoding the path, they'd be fully
| portable? Seems to be a fairly simple update.
|
| > Since venvs on Linux symlink to the system's Python
| executable
|
| Hopefully, the executable with which they were created,
| not the system executable except where they happen to be
| the same.
| simonw wrote:
| Python wheels really can help here, but only if the downstream
| packages you depend on have published wheels for the various
| different platforms you need to support.
|
| https://cibuildwheel.readthedocs.io/ is a great tool for
| assisting with that - many projects run that in GitHub Actions
| as part of their release process.
|
| There's just one catch, in my experience: since GitHub don't
| offer Apple Silicon machines as part of GitHub Actions, it can
| be hard to build wheels there that work on those platforms.
| Maybe you can cross-compile them? I've not figured that out
| yet.
| oefrha wrote:
| PyInstaller bundles native dependencies just fine. Well, for
| Windows and macOS at least, never packaged as "native app" for
| Linux.
| Frost1x wrote:
| Eh it depends, some python packages do things like pull data
| samples from my experience and it can be quite a mess
| detangling the package to give PyInstaller everything it
| needs to bundle stuff up. It also takes a lot of additional
| configuration for a lot of packages.
|
| As much as I hate how containers are often misused, this is
| one of the reasons people obsess over them from my
| experience. The ability to bundle all your crap regardless of
| how many layers of package management idiosyncrasies your
| application ultimately woul have to deal with pretty much
| transparently.
| KeplerBoy wrote:
| esoteric as in a whole lot of essential libraries?
| [deleted]
| stuaxo wrote:
| This needs solving.
|
| It makes it really hard to distribute certain python
| libraries.
| robertlagrant wrote:
| Do wheels not solve this? I know not everyone ships wheels,
| but are they not the answer?
| BiteCode_dev wrote:
| Nuitka deals pretty well with those in general:
| https://nuitka.net/
| Jack5500 wrote:
| Until you require pytorch ;)
| schemescape wrote:
| For Node, I've seen packages just bundle a copy of the shared
| library for each supported platform. I would assume you could
| do the same for Python.
| mvdwoord wrote:
| Although I have never used any of their stuff, I had the pleasure
| of watching Russell from BeeWare at Pycon Colombia some years
| ago. It consistently presents itsself as an ecosystem I would
| like to work with, and if anything the naming of their projects
| is very nice.
|
| https://www.youtube.com/watch?v=TY4vb6V-gvc
| velosol wrote:
| Anyone happened to use Toga with video streams? I've got some
| gstreamer stuff with Qt and it doesn't need a rewrite yet but
| it's a glimmer on the horizon.
| 40four wrote:
| I've had a lot of experience with another similar library, Kivy
| (kivy.org). These types of libraries, can be quite effective for
| the right use case. I shipped a couple mobile apps on both Google
| and Apple, so there's that. I would check this out, but I do
| remember having to work through a lot of issues to get the mobile
| build just right. I wonder if a less mature project like this
| would have similar struggles?
___________________________________________________________________
(page generated 2023-08-03 23:02 UTC)