[HN Gopher] Show HN: Localscope-Limit scope of Python functions ...
___________________________________________________________________
Show HN: Localscope-Limit scope of Python functions for
reproducible execution
localscope is a small Python package that disassembles functions to
check if they access global variables they shouldn't. I wrote this
a few years ago to detect scope bugs which are common in Jupyter
notebooks. It's recently come in handy writing jax code
(https://github.com/jax-ml/jax) because it requires pure functions.
Thought I'd share.
Author : tillahoffmann
Score : 52 points
Date : 2025-03-17 17:33 UTC (5 hours ago)
(HTM) web link (localscope.readthedocs.io)
(TXT) w3m dump (localscope.readthedocs.io)
| dleeftink wrote:
| > Everything works nicely, and you package the code in a function
| for later use but forget about the scale factor introduced
| earlier in the notebook.
|
| You see a problem, you fix it with library, and I applaud that.
| You have to wonder though, how many years does it take for a
| _reproducible_ notebook environment to implement out of scope
| variable guards..
| nerdponx wrote:
| This is for people who don't want to switch notebook
| environments, because Jupyter(lab) is getting better faster
| than alternatives (which support things like reactive cell
| execution) are becoming usable for day-to-day work.
|
| It's just a safeguard for well-intentioned people to prevent
| themselves from making mistakes with their existing tools,
| instead of changing to a completely different set of tools.
| jampekka wrote:
| Jupyter-style notebooks are a good example of a deep
| architectural mistake that needs hacks on hacks on hacks to
| remain barely serviceable.
|
| Luckily there are new approaches, e.g. Marimo and Pluto, that
| don't have the same root issue.
| tillahoffmann wrote:
| Yes, Jupyter notebooks are flawed (there's a great talk at
| https://www.youtube.com/watch?v=7jiPeIFXb6U). But they are
| also very convenient for scrappy work and exploratory data
| analysis.
| jampekka wrote:
| Notebooks can be convenient and have a consistent state at
| the same time, as done in e.g. Marimo.
| nine_k wrote:
| Nice! This approach can be used to implement coeffects (as e.g.
| seen in Hack [1]), by only passing explicit effect-producing
| objects. Imagine a function that's guaranteed to not write to any
| files, because it can't access `open()`, or can only access a
| version of `open()` that only accepts read-only access.
|
| [1]: https://docs.hhvm.com/hack/contexts-and-
| capabilities/introdu...
| pkaodev wrote:
| Maybe I'm being naive, but it feels like if this is a problem for
| people they should be looking at how they write their code, not
| reaching for a library.
| disgruntledphd2 wrote:
| This bug basically hits people who run long lived code sessions
| with large objects (i.e. data people).
|
| I've hit it in both R and Python in interactive sessions.
| Otherwise it's generally a non issue.
|
| I can definitely see this being helpful, particularly for the
| intended context. It would also be useful when you need to move
| notebook code to a different environment.
| RadiozRadioz wrote:
| Completely. I interact with data people sometimes at work.
| There are bespoke one-of-a-kind venvs and giant notebooks
| everywhere. But that's okay. They don't have our job, it is
| not their job to build robust & properly deployed software.
| They know how to do their thing and we shouldn't lecture them
| on how to do it the "proper" way. Because I sure don't know
| how to do what they do - Pandas & matplotlib take me hours of
| googling to do anything.
|
| I completely see the value in this tool for a particular
| style of programming.
| dleeftink wrote:
| I don't think the issue is telling someone how to do their
| jobs, but that when it comes to it, we are not properly
| communicating our stack preferences at all. Online we do,
| sure, but let's not shy away from having these discussions
| in the open. There is much to learn from each other's best
| and worst practices.
| aiono wrote:
| Software is complex and our memory is bottleneck to create
| software. We can only remember so many things therefore
| anything that we can make computer to "think" instead of us we
| have more memory to use for other work. In this case, instead
| of worrying if you are accessing some random global variable by
| accident is unnecessary cognitive work you. Why not just let
| computer do it while you think for actual things?
| serial_dev wrote:
| But how can you make sure that people annotate their functions
| with localscope?
| igorguerrero wrote:
| Same question people have with hinting, what makes Python cool
| is how loose it is... You could also make a lint rule for that
| and make fail CI if that rule failed, or... Write your own
| compiler you know?
| escapecharacter wrote:
| Since I've started using Jupyter notebooks, I've wanted a feature
| to "undo" running a cell. This feels so important to for
| spontaneous exploration. This work feels like an important
| building block for this!
| mythrowaway49 wrote:
| agree! I feel like there must be a good workaround. Currently,
| I just need to go back and run a bunch of cells again..
| olejorgenb wrote:
| A crazy, but cool idea to implement this is using fork:
| https://github.com/thomasballinger/rlundo
| Vaslo wrote:
| Only solved by the ever efficient clear outputs, restart, then
| run all the code all over again...
| nathan_compton wrote:
| I have this idea of tools which help you do the right thing and
| tools that let you do the wrong thing longer. Jupyter is already
| the latter sort of tool, and this is the bad kind of tool on top
| of the bad kind of tool.
| matt3210 wrote:
| If you don't want pyhton flavored scope and lifetime rules, use
| another language :emothonless-face:
| simonw wrote:
| I was curious how this works - it's using the Python standard
| library disassembler: https://docs.python.org/3/library/dis.html
| for instruction in dis.get_instructions(code): ...
|
| Code here:
| https://github.com/tillahoffmann/localscope/blob/092392d5bdb...
| mpeg wrote:
| I wrote myself a similar decorator for a completely different
| purpose - ensuring a function I'm going to serialise over the
| network doesn't have any outside dependencies
|
| This is actually a cleaner API so might switch my code to it,
| amazing work
| bpshaver wrote:
| > Have you ever hunted bugs caused by accidentally using a global
| variable in a function in a Jupyter notebook?
|
| Nope!
|
| > Have you ever scratched your head because your code broke after
| restarting the Python kernel?
|
| Also nope!
|
| I'm technically a data scientist but I don't use notebooks very
| often. Even when I did, though, I can't recall having this error
| very often. Is this really necessary? Anyway, this kind of thing
| is only useful if people adopt it. The people who are likely to
| install this tool and use it probably already avoid this issue
| while the people who have this issue will never hear of this tool
| or will not consistently employ it.
|
| I guess I'm just salty from years of dealing with people who give
| their Python code half-hearted incorrect type hints but never run
| a static type checker on their code.
| codetrotter wrote:
| > Anyway, this kind of thing is only useful if people adopt it.
| The people who are likely to install this tool and use it
| probably already avoid this issue while the people who have
| this issue will never hear of this tool or will not
| consistently employ it.
|
| I could see projects adopting this, or something like it, in
| their CI pipelines.
|
| Both for open source projects, but also at companies with a lot
| of internal repositories with Python code - especially if it is
| common that other people with little experience for some repo
| contribute code to it.
|
| In the case of an open source project if it's a popular repo
| that a lot of people are users of and different people
| frequently contribute to for the first time.
|
| And in the case of company internal repositories, if it's code
| that different teams only touch now and then, with only a
| smaller group of people intimately familiar with all of the
| inner workings of the code and the other people being mainly
| focused on other repos in the company but still having to
| sometimes contribute changes because their code is relying on
| the code in that other repos.
___________________________________________________________________
(page generated 2025-03-17 23:00 UTC)