[HN Gopher] Traps for the Unwary in Python's Import System
___________________________________________________________________
Traps for the Unwary in Python's Import System
Author : mmastrac
Score : 39 points
Date : 2022-08-16 15:12 UTC (1 days ago)
(HTM) web link (python-notes.curiousefficiency.org)
(TXT) w3m dump (python-notes.curiousefficiency.org)
| loeg wrote:
| Another potentially surprising trap is the sheer number of
| filesystem syscalls per import. Python searches a _ton_ of
| potential filesystem locations for each import, and the default
| config may not search the most likely locations first. This had a
| significant impact at a previous employer.
| svnpenn wrote:
| The single worst part of Python import for me is "from". I think
| "from" was a mistake, and should never have been added. Like many
| "pythonism" its just another layer of sugar thats not needed, and
| is only marginally useful. Especially when dealing with new
| packages, its essentially trial and error for me with:
| import hello from hello import world import
| hello.world from hello import world.bye from
| hello.world import bye
|
| I didn't realize until I went to Go, where "from" doesn't exist,
| how much I hated it with Python.
| buscoquadnary wrote:
| Honestly I will always be saddened by the fact in this reality
| Python won out over Ruby in the readable scripting language wars.
| Writing Ruby feels elegant, it feels fluid, it feels like
| creating.
|
| Working with Python always leaves me with a sour taste in my
| mouth and frustration after having to try and figure out how to
| make modern concepts work well, and giving up and going back to
| old imperative styles of coding.
| shadowofneptune wrote:
| Python's mission statement has been to be the most ergonomic
| imperative language. That it doesn't support other styles of
| programming doesn't seem so bad to me, languages do not need to
| be multi-paradigm.
|
| Where does the frustration come from with Python being more
| popular than Ruby, is it that some libraries can only be found
| in Python?
| foobarbecue wrote:
| Woah, we don't need __init__.py any more. Mind blown.
| jlmark wrote:
| Python packages are a hack that is held together by duct tape and
| Stackoverflow answers that everyone copies.
|
| Even worse is packaging the packages for PyPI, especially if
| multiple packages depend on each other.
|
| Everyone complains about C++, but since C++11 many things are so
| much easier than in Python. Not to mention Lisp, OCaml, Java,
| which all have far better solutions.
| simonw wrote:
| This stuff has got SO MUCH better over the past few years.
|
| The hardest thing about it these days is mainly that there are
| so many historical artifacts, so if you're trying to figure out
| how to use packaging there are a lot of outdated resources.
|
| Modern Python packages are really pleasant to work with. Here's
| the best current tutorial that I've seen:
| https://packaging.python.org/en/latest/tutorials/packaging-p...
| sseagull wrote:
| I'm not so sure. I went down this rabbit hole. It's really a
| mess.
|
| Setuptools? Poetry? Hatch? PBR? Flit? PDM? Why are there so
| many build systems? How do I choose the proper one?
|
| The article you posted uses hatch, which I hadn't even heard
| of until last week.
|
| Should I use setup.py, setup.cfg, or pyproject.toml? Things
| seem to moving to pyproject.toml but lots of existing
| functionality seems to point to setup.py. Looking for answers
| on SO seems to result in a mix of all different combinations.
|
| And then there's virtual environments...
| theptip wrote:
| Ten years ago things were pretty bad, and I can see how it
| could be confusing trying to pick through all of the
| different options that have risen and fallen over the
| years.
|
| In 2022, If you're building an application and are looking
| for a simple answer, just use Poetry; it's similar enough
| to what you'll be familiar with if coming from other
| languages (e.g. yarn, cargo, ...). For most usecases you
| can just lean on Poetry's venv management; you don't need
| to do more than `poetry shell` or `poetry run` to get
| access to it. Poetry will create, and for the most part
| manage, your pyproject.toml file for you.
|
| If you want a bit more flexibility Virtualenv is part of
| python now (`python -m venv`). If you're on MacOS you can
| get great venv management with pyenv and pyenv-virtualenv.
| And for sure, things get more complex if you're building a
| library. But I think Poetry is a solid place to start.
| sseagull wrote:
| But this directly contradicts the link given above - the
| "official" python packaging tutorial recommends Hatch,
| and barely mentions poetry...
| donio wrote:
| But that's always the story with Python isn't it. There is
| always this new way to do things that is so much better than
| all the previous ones. If only everybody was doing it the
| right way. At least until the next one comes along.
|
| We've come a long way since PEP 20.
| zdragnar wrote:
| Every time I see a language or a framework proclaim that it
| is designed so that there is one "right" or "obvious" way
| to do something, you can inevitably see that right way has
| changed multiple times over the course of its life, and it
| becomes obvious the larger an older any given project is.
|
| I left one company 3 years ago in the middle of a major
| transition between versions. The "right way" changed fairly
| drastically, and surprise surprise, they're still stuck in
| the middle of that transition.
|
| I think this is largely why I have found myself preferring
| static typing with roll-your-own framework style / library-
| based ecosystems over batteries-included systems.
|
| When your language or framework of choice suddenly insists
| that you use AA batteries instead of button cells, and it
| is up to you to modify every electronic device in your
| house to use them, it gets real old real fast (an
| exaggerated example, but not entirely unheard of).
| [deleted]
| [deleted]
| [deleted]
| altruios wrote:
| Wait: does c++ have a package manager like pip or npm or gem
| that I didn't know about?
| molq wrote:
| There are _file system packages_ (with __init__.py) and _PyPI
| packages_.
|
| The former are an extremely poor version of Lisp packages,
| OCaml modules or C++ name spaces.
|
| The latter are _distribution packages_ (even for distribution
| packages C++ has Conan etc.).
|
| PyPI packaging gets even worse than usual when attempting to
| distribute a file system package.
| autoexec wrote:
| Getting started in python I quickly ran into the "name shadowing
| trap" and saw this was the exact same problem as this poor guy
| had way back in 2004!: https://bugs.python.org/issue946373
|
| There are hundreds of "reserved" names that must be avoided when
| naming scripts, many of them are exactly the sort of names
| someone might use playing around in a new language, and every
| update/new module installed can add any number of new ones
| breaking your existing programs.
|
| There really should be a flag or environment variable to specify
| that the current directory should be ignored (or considered to be
| at the end of sys.path) for all scripts across the whole system.
|
| For something with such an easy fix I'm surprised this foot gun
| has been left out for 20 years. I mean, the bug report even had a
| patch included to fix the problem!
|
| Wherever you are now wrobell, know that I have shared in your
| pain and that I thank you for trying to fix this mess.
| Spivak wrote:
| I don't know if there's a good solution to this because
| ignoring the current directory would only solve the first order
| problem but all those package names would still be essentially
| reserved, you'll just get bit less often, and you'll have to
| add your own code into the path explicitly. I think forcing
| relative imports and adding a new syntax for "project absolute"
| imports is the real way forward but is totally infeasible
| because it break everyone's code.
| autoexec wrote:
| It'd also be much improved if the names of scripts could
| never be confused for the names of packages... maybe a new
| file extension like packagename.pm (to steal from perl)?
|
| It'd require an update to package names, but nobody would
| have to update their code.
| blibble wrote:
| python 3 (being not completely compatible) was a great chance to
| add... namespaces everywhere
|
| "Namespaces are one honking great idea -- let's do more of
| those!" after all
|
| sadly I still have to worry about a system package one day being
| pulled in instead of my code
|
| (and pypi could do with namespaces too)
|
| one of the few things the Java world got spot on
| pid_0 wrote:
| zmgsabst wrote:
| I've just accepted it's weird...
|
| So I always invoke my Python as a module and do relative imports,
| which at least eliminates the conflicts from my code.
|
| YMMV.
___________________________________________________________________
(page generated 2022-08-17 23:01 UTC)