[HN Gopher] Using Python for Scripting
___________________________________________________________________
Using Python for Scripting
Author : birdculture
Score : 73 points
Date : 2025-12-08 08:52 UTC (5 days ago)
(HTM) web link (hypirion.com)
(TXT) w3m dump (hypirion.com)
| zahlman wrote:
| Dupe: https://news.ycombinator.com/item?id=46176113
| froh wrote:
| this discussion took off, however...
| mubou2 wrote:
| How do you handle packages? I want scripts to a be a single file
| with a shebang, not a repo with a requirements.txt that I need to
| run in a venv. To me, this is the biggest blocker to using Python
| for any non-trivial scripting (which is precisely the kind where
| I wouldn't want to use bash), but I'd like to know how others
| deal with it.
|
| C# scripts let you reference packages in a comment at the top of
| the file, for example:
|
| https://devblogs.microsoft.com/dotnet/announcing-dotnet-run-...
| eternityforest wrote:
| What can you do with bash that isn't in the stdlib of python?
|
| Generally the only nontrivial scripting I ever do is associated
| with a larger project, so I often already have a pyproject.toml
| and a UV environment, and I just add the dependencies to the
| dev group.
| mubou2 wrote:
| Well, that's kind of what I mean. For scripts in a python
| project, you can freely use whatever packages you need. But
| for one-off scripts, if you need bs4 or something, you're
| screwed. Either your script now has external dependencies or
| it requires special tooling.
|
| It just feels strange that C# of all languages is now a
| better scripting tool than Python, at least out of the box. I
| did notice uv has exactly the feature I'm looking for, though
| it's obviously third-party:
|
| https://docs.astral.sh/uv/guides/scripts/#declaring-
| script-d...
|
| Is everyone just using uv now instead of pip, perhaps? Or is
| just another alongside pipenv, conda, poetry, etc.? (Python's
| not my main these days, so I'm out of the loop.)
| eternityforest wrote:
| UV is taking over really fast, it seems to be much more
| popular any other option.
|
| I suspect conda still has some market share too but I've
| never needed it.
| psunavy03 wrote:
| It's amazing what happens when something just works.
| jollyllama wrote:
| I don't understand. To return to GP's point, what can you
| do in bash that you can't do in Python? Said in another
| way, what does bash offer that you would need to tackle
| with a dependency in Python? My understanding is that there
| is no such thing, and accordingly, you can still end up
| with something that is better than bash if you just use
| Python and call out to other tools with subprocess.
| dizhn wrote:
| There's bash. Then you need better loops/conditionals and
| more usable structures. That's when one should think of
| using a scripting language instead. I think the parent
| goes too far after that and what he's talking about is
| not something bash can do (well).
|
| That said, a lot of very complicated things are actually
| written in bash. Distrobox I think is for example.
| DonHopkins wrote:
| >That said, a lot of very complicated things are actually
| written in bash. Distrobox I think is for example.
|
| They're only complicated BECAUSE they're written in bash.
| If they were written in Python they would be much less
| complicated, more modular, able to use many existing
| industrial strength well tested python modules instead of
| rolling their own ad-hoc ones, and much easier to
| maintain.
| chillaranand wrote:
| You can specify requirements at the top of file and uv can run
| the script after automatically installing the dependencies.
|
| https://avilpage.com/2025/04/learn-python-uv-in-100-seconds....
| simonw wrote:
| This works really well in my experience, but it does mean you
| need to have a working internet connection the first time you
| run the script. # /// script #
| dependencies = [ # "cowsay", # ] # ///
| import cowsay cowsay.cow("Hello World")
|
| Then: uv run cowscript.py
|
| It manages a disposable hidden virtual environment
| automatically, via a very fast symlink-based caching
| mechanism.
|
| You can also add a shebang line so you can execute it
| directly: #!/usr/bin/env -S uv run --script
| # # /// script # dependencies = ["cowsay"]
| # /// import cowsay cowsay.cow("Hello World")
|
| Then: chmod 755 cowscript ./cowscript
| emidln wrote:
| I wish env -S was more portable. It's a newer feature of
| the coreutils env implementation and isn't supported
| elsewhere afaik.
| collinfunk wrote:
| FreeBSD 6.0 added 'env -S'. They have adopted a few
| different GNU inspired options recently, which I am happy
| about.
| networked wrote:
| You can use so-called "exec magic" instead of `env -S`.
| Here is an explanation with Python examples:
| https://dbohdan.com/scripts-with-dependencies#exec-magic
| (disclosure: my site). In short: #!
| /bin/sh "exec" "/usr/bin/env" "uv" "run" "--quiet"
| "--script" "$0" "$@" # /// script #
| dependencies = [ # "cowsay", # ] #
| /// import cowsay cowsay.cow("Hello, world!")
|
| On systems that can't run uv, like NetBSD and OpenBSD,
| switch to pipx: #! /bin/sh "exec"
| "/usr/bin/env" "pipx" "run" "$0" "$@" # ...
| presbyterian wrote:
| As someone who writes a lot of python, I love uv, but isn't
| on nearly every system like python is, which is one of the
| arguments for using python here in the first place
| ewidar wrote:
| Sure but the sub question here was about packages.
|
| If you are installing packages, then starting with
| installing uv should be fine.
| Rucadi wrote:
| Nix allows you to do this with any language and required
| dependency: https://wiki.nixos.org/wiki/Nix-shell_shebang
| albertoCaroM wrote:
| Whoa! This is a revelation. I already loved Nix and used nix-
| shell extensively, but this is the missing piece: fully
| reproducible Python scripts without compromise.
| luz666 wrote:
| Or install direnv and put your dependencies into a
| shell.nix, setup the bash hook according the manual, create
| the .envrc with the content "use nix". Then type "direnv
| allow".
|
| Then you can do e.g. use other persons python scripts
| without modifying their shebang.
| HumanOstrich wrote:
| Now you have to set up Nix first and deal with that nightmare
| of a learning curve. Just to auto-install some dependencies
| for a script.
|
| Might as well rewrite all your scripts in Rust too while
| you're layering on unholy amounts of complexity.
| Diti wrote:
| It's like Vim, you learn it once, and you keep using it
| forever once you're used to it.
|
| I'm so thankful to see a flake.nix file in every single
| cool project on code forges.
| HumanOstrich wrote:
| Yea that's a common theme of excuses for both Rust and
| Nix. Wrong though, because most anyone who can use a
| computer at all can learn the basics of Vim.
|
| Seeing that flake.nix badge of complexity lets me know a
| project will be a nightmare to set up and will break
| every other week. It's usually right next to the
| Cargo.toml badge with 400 dependencies underneath.
| kokada wrote:
| Nix with Flakes never randomly break, I still have
| projects from 3 or 4 years ago that I can still run `nix
| build` and getting it running. Yes, if you try to update
| the `flake.lock` this may introduce breakages, but this
| is expected if you're pining `nixos-unstable` instead of
| a stable branch.
| milliams wrote:
| https://peps.python.org/pep-0723/, e.g.: # ///
| script # requires-python = ">=3.11" # dependencies
| = [ # "requests<3", # "rich", # ] #
| /// import requests from rich.pretty import
| pprint resp =
| requests.get("https://peps.python.org/api/peps.json")
| data = resp.json() pprint([(k, v["title"]) for k, v in
| data.items()][:10])
| paulddraper wrote:
| > How do you handle packages?
|
| The same way you handle them with bash?
|
| Install them?
|
| What are we talking about here?
| fridder wrote:
| I think it is the fact that python packaging has been
| problematic for some time. setuptools/easy_install, pip/pipx,
| poetry, conda, uv all promise they will be the thing that
| "fixes" it
| paulddraper wrote:
| What does Bash use for its packaging?
|
| Use that.
| bccdee wrote:
| Ok tbh bash uses the system package manager to install
| command-line utilities, and using the system package
| manager for python packages HAS been tried, and the
| parallel use of system packaging and independent package
| managers is part of why python package distribution is
| such a mess.
| skydhash wrote:
| Why using independent package managers alongside the
| system one? I think the introduction of non system
| packagers is what brought us the whole mess we are in.
| Most system packagers allows for custom repositories.
| a-dub wrote:
| what can you do with bash that you cannot do with the python
| standard library and shelling out?
| kjkjadksj wrote:
| What is wrong with writing a short wrapper in bash to activate
| a conda env before running the script? Too unsexy?
| DonHopkins wrote:
| Isn't the fact that bash doesn't have a wonderful ecosystem of
| reusable modules a much more enormous insurmountable problem
| than the fact that you have to install python modules? You're
| really missing the forest for the trees.
|
| Not only does bash not have a module system like python, or a
| vast ecosystem of modules like python, but also that it's much
| too weak and brittle a language to implement most of those
| modules that Python has, and can't even call native code
| libraries directly.
|
| Even with just its standard built in "batteries included"
| libraries and no extension modules or native code modules,
| Python is still much more powerful than bash and easier to code
| and maintain.
|
| If you're complaining about having to install Python modules,
| you're usually already doing something that's impossible or
| incredibly difficult to do in bash anyway.
|
| Even something as simple and essential as fetching files via
| http or parsing json. Bash has to call out to other programs to
| do that, but you have to install those programs too, and while
| Python can certainly call out to curl or wget or jq, it doesn't
| have to, since it has all that and more built in.
|
| There really is no comparison, because bash loses along so many
| dimensions at once compared to Python.
| froh wrote:
| Isn't pipx addressing exactly that?
|
| once the script is non-trivial, 'install' it using pipx, in
| editable mode when you work on the script and as normal pipx
| installed cli utility otherwise.
|
| the venv part is then completely under the hood.
| N_Lens wrote:
| Weren't we already?
| sevensor wrote:
| The Python stdlib does not get enough credit. People complain
| about things like how its http client is dated and slow, but it's
| pretty amazing that it's just right there if you need it, no
| external dependencies needed. And it's sitting right next to
| difflib, graphlib, pathlib, struct, glob, tkinter, and dozens of
| others. Sure, every one of these is limited individually, but
| those limitations are stable and well understood!
| rjmill wrote:
| Odd, I don't see any mention of subprocess.run, the workhorse of
| python scripting.
|
| Quick rundown for the unfamiliar:
|
| Give it a command as a list of strings (e.g.,
| subprocess.run(["echo", "foo"]).)
|
| It takes a bunch of flags, but the most useful (but not
| immediately obvious) ones are: check=True: Raise
| an error if the command fails capture_output=True: Captures
| stdout/stderr on the CompletedProcess text=True:
| Automatically convert the stdout/stderr bytes to strings
|
| By default, subprocess.run will print the stdout/stderr to the
| script's output (like bash, basically), so I only bother with
| capture_output if I need information in the output for a later
| step.
| bccdee wrote:
| Also `asyncio.subprocess`, which lets you manage multiple
| concurrently running commands. Very handy if you need to
| orchestrate several commands together.
| theomega wrote:
| One thing I can recommend that makes scripting in python with
| external commands a lot easier is the `sh` module:
|
| https://pypi.org/project/sh/
|
| Basically you can just `from sh import [command]` and then have
| an installed binary command available as function
| from sh import ifconfig print(ifconfig("eth0"))
| code_biologist wrote:
| uv for using sh as a dependency in scripts, managed inline,
| has changed it from "eh, I'll just use subprocess" to "why
| not" for me.
|
| https://docs.astral.sh/uv/guides/scripts/#using-different-
| py...
| pxc wrote:
| I've used Plumbum for this for some projects at work, and
| really like it for this.
|
| https://plumbum.readthedocs.io/en/latest/local_commands.html.
| ..
|
| It also does argument parsing and validation, so it's
| generally pretty useful for writing little CLI tools that
| invoke other CLI tools.
|
| https://plumbum.readthedocs.io/en/latest/cli.html
| albertzeyer wrote:
| I think the point is that for most things, you don't need to
| call any external tools. Python's standard library comes
| already with lots of features, and there are many packages you
| can install.
| tayo42 wrote:
| Pretty much anything longer then a throwaway one liner I write in
| python.
|
| Would be cool if python had a pipe operator though.
|
| The back ticks in ruby is pretty ergonomic too. Wish python had a
| simpler way to run commands. Kind of tedious to look up
| subprocess run arguments and also break things up into arrays.
| mackeye wrote:
| nushell (https://www.nushell.sh/) is essentially perfect for
| this use case! i use it rather than posix(y) shells for most
| projects where id normally reach for python.
| kstrauser wrote:
| You can always set shell=True and pass in an entire command
| line as a string, but... don't do that. It seems really nice
| until the first time you get the shell escaping wrong, and then
| it's something you tend never to do again.
|
| For example, subprocess.run("rm -rf ~/ some
| file", shell=True)
|
| and subprocess.run(["rm", "-rf", "~/ some
| file"])
|
| have significant different behavior.
| davidkhess wrote:
| Xonsh is perfect for this: https://xon.sh
| abhiyerra wrote:
| I have been converting a lot of my makefiles to pyinvoke and
| fabric and it makes things so much easier to manage than bash or
| make. Don't know why I held on for so long.
| Rendello wrote:
| I've never liked shell scripting. Last year, I switched my build
| system of a Rust project over to Python (Cargo is actually quite
| limited as a build system). For a newer project, I'm using Rust
| itself with the XTask pattern. I'm not sure if I prefer the
| Python or Rust approach yet.
| seabrookmx wrote:
| I quite liked SCons back when I wrote C++!
| kkfx wrote:
| Python for scripting honestly is Xonsh :)
| gabrielsroka wrote:
| > Python is installed on pretty much every machine
|
| > Python 3 is installed on basically every machine out there.
|
| > Python will work the same on all the machines you run your
| script on
|
| No, no, and no.
| nilslindemann wrote:
| More like, "no, but installation is easy", "yes, if the machine
| is safe", "depends on what you do"
| GutenYe wrote:
| Same, use Javascript for Scripting.
| https://github.com/gutenye/script.js
| archargelod wrote:
| If a script is simple - I use posix sh + awk, sed, etc.
|
| But if a script I write needs to use arrays, sets, hashtable or
| processes many files - I use Nim[0]. It's a compiled systems-
| programming language that feels like a scripting language:
|
| - Nim is easy to write and reads almost like a pseudocode.
|
| - Nim is very portable language, runs almost anywhere C can run
| (both compiler and programs).
|
| - `nim r script.nim` to compile and run (cached on subsequent
| runs) or use a shebang `#!/bin/env -S nim r`
|
| - Nim programs are fast to compile (use debug mode and tcc
| compiler for almost instant compile times)
|
| - Nim scripts run very fast <10ms (something that was very
| annoying to me with bash and Python)
|
| - good chances you don't need external dependencies, because
| stdlib is batteries included and full of goodies.
|
| - if you need external deps - just statically link them and
| distribute a cross-compiled binary (use zigcc[1] for easy Nim
| cross-compilation).
|
| [0] - https://nim-lang.org
|
| [1] - https://github.com/enthus1ast/zigcc
| pxc wrote:
| > If a script is simple - I use posix sh + awk, sed, etc.
|
| > But if a script I write needs to use arrays, sets, hashtable
| or processes many files
|
| One option that I sometimes use at work (in addition to writing
| some Python CLIs) that is a pretty nice next step on this
| spectrum is Bash-with-all-the-fixins'.
|
| I use a Nix-based dependency resolver for shell scripts called
| resholve1 to parse scripts so that it can identify all of their
| dependencies (including Bash itself), then produce a "compiled"
| script as a Nix build where all of the references to external
| programs are replaced with pinned Nix store paths.
|
| Then I have a fixed (and recent) version of GNU Bash regardless
| of platform, so I'm free to use Bash features that are newer or
| nicer than POSIX sh. My favorite such features are `mapfile`
| and `lastpipe` for writing in a more functional style, plus of
| course maps ("associative arrays").
|
| I don't have to worry about portability problems with common
| utilities, because my scripts will bring along the expected
| implementations of coreutils, `find`, `grep`, etc. I'm free to
| use "modern" alternatives to classic utilities (like `rg`
| instead of GNU grep or `fd` instead of GNU findutils) if they
| offer better performance or more readable syntax. Polished
| interactivity is easy to build by just embedding a copy of
| `fzf`. And while I don't love Bash for working with structured
| data, it becomes a lot less painful when I can just pull in
| `jq`.
|
| It's obviously got some disadvantages versus an option like
| Python, but the amount-of-code to functionality ratio is also
| much more favorable than Python's.
|
| I typically use this for scripting build and development tasks
| in projects that already use Nix (via Devenv-- I have some
| unpublished changes to the scripts module that add resholve and
| ShellCheck integration) to manage their environments, so
| there's no special packaging/distribution/setup burden.
|
| IME it makes for vastly more readable and maintainable shell
| scripts than painstakingly limiting oneself to pure POSIX sh,
| so it can serve quite well as a middle ground option between
| barebones sh and a "real" programming language as your little
| script gradually grows in complexity.
|
| > if you need external deps - just statically link them and
| distribute a cross-compiled binary (use zigcc[1] or easy Nim
| cross-compilation).
|
| The deployment story sounds very slick and hard to beat! This
| is something I might want to try for scripts I want to be easy
| to distribute on macOS systems that don't have Nix installed.
|
| --
|
| 1: https://github.com/abathur/resholve
| kazinator wrote:
| > By the way, don't add a comma after the elements in the list,
| ...
|
| Python: forty_two = ( 42 )
| tuple_containing_forty_two = ( 42, )
| sevensor wrote:
| You don't even need parentheses here. 42,
|
| This expression is a tuple all by itself.
| kazinator wrote:
| cp version.template build/version.txt sed -i
| "s/@VERSION@/${COMMIT_TAG:-dev}/" build/version.txt sed -i
| "s/@BUILD_DATE@/${BUILD_DATE}/" build/version.txt
|
| Eww! Mutating a file in place, and needing a GNU extension for
| it. sed \ -e
| "s/@VERSION@/${COMMIT_TAG:-dev}/" \ -e
| "s/@BUILD_DATE@/${BUILD_DATE}/" \ <
| build/version.template > build/version.txt
| headcrash wrote:
| why not use perl?
| headcrash wrote:
| Why not use Perl?
| binary132 wrote:
| I like using Go for "scripting". To each their own I suppose.
| Alifatisk wrote:
| Is this comment some kind of subliminal message?
| yoavsha1 wrote:
| I strongly agree with this. At $WORK I usually work on projects
| comprising many small bits in various languages - some PowerShell
| here, some JS there, along with a "build process" that helps
| minify each and combine them into the final product. After
| switching from shell scripts to Just and having to deal with a
| ton of issues on the way (how does quoting work in each system?
| How does argument passing? Environment variables?) I simply wrote
| a simple script with Python, UV shebang and PEP723 dependencies.
| Typer takes care of the command line parsing, and each "build
| target" is a simple, composable, readable python function that
| takes arguments and can call other ones if it needs to. Can't be
| simpler than that and the LLMs love it too.
| Alifatisk wrote:
| I like the message the article is trying to convey, Python is
| good alternative to complicated shell scripts in my opinion.
|
| I do wonder, let's say the scripting file is using lots of
| libraries, do you have to include some kind of requirements.txt
| file with it aswell when you want to share it with other people?
|
| In Ruby, there is inline bundler which makes sharing a single
| Ruby script very portable.
|
| https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
___________________________________________________________________
(page generated 2025-12-13 23:00 UTC)