[HN Gopher] Show HN: deptry 0.14.0 - detect unused Python depend...
___________________________________________________________________
Show HN: deptry 0.14.0 - detect unused Python dependencies up to 10
times faster
deptry 0.14.0 was just released, bringing significant speed
improvements: It is now up to 10 times faster than the previous
release! For those unfamiliar with deptry; deptry is a command
line tool to check for issues with dependencies in a Python
project, such as unused or missing dependencies. GitHub:
https://github.com/fpgmaas/deptry For some benchmarks of the new
release, see the release notes:
https://github.com/fpgmaas/deptry/releases/tag/0.14.0 The
performance improvement was achieved by leveraging Rust to parse
the AST and extract the import statements from .py files, rather
than using Python's ast module. The addition of Rust to the
project also opens up doors for more optimizations in the future,
so stay tuned!
Author : flo12392
Score : 125 points
Date : 2024-03-16 08:04 UTC (14 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| VeejayRampay wrote:
| I feel like cases where the import is different from the package
| name yield false positives
|
| i.e. if I have
| beautifulsoup4[html5lib,lxml]==4.12.3
|
| in my requirements.txt and then import with from
| bs4 import BeautifulSoup
|
| it would complain
|
| is there a way to configure that somehow?
|
| aside from that, the tool is extremely fast
| flo12392 wrote:
| This is not an issue when using `deptry`; it does not only
| match the import name against the dependency name; it first
| tries to find the modules that are provided by the dependency
| `beautifulsoup4`. If you run `deptry -v .`, you can see this in
| the verbose logs:
|
| ` Dependency 'beautifulsoup4' with top-levels: {'bs4'}.`
|
| If you ever run into a situation where this does not work, you
| can use the `pacakge_module_name_map` parameter:
| https://fpgmaas.github.io/deptry/usage/#package-module-name-...
|
| Hope this helps! Florian
| verdverm wrote:
| VS Code does this for me.
|
| What use case is this after? Linting in CI or similar?
| flo12392 wrote:
| I am not aware of functionality within VS Code that does the
| same thing as `deptry` does. What most IDE's support is
| removing unused imports, which sounds similar but is different
| from what deptry aims to do: deptry tries to remove unused
| dependencies. So if you have e.g. `pandas` in your
| `requirements.txt`, but you never import (from) `pandas` in
| your project, `deptry` will tell you that you should remove it
| from your `requirements.txt`. Does that answer your question?
| parpfish wrote:
| This is particularly helpful if at one point somebody
| replaced a nice succinct requirements.txt with the output of
| 'pip freeze'
| hereonout2 wrote:
| This is a bit more useful than things like vscode or flake8
| (my preferred option) telling you that you "import xxx" but
| don't use it.
|
| I'd love a deeper way to identify and selectively reduce the
| cruft from projects though. For example currently I have
| about 180 dependencies in my project's .venv - most of them
| transitive.
|
| These include things like the google-auth library when there
| is no code in my project that would ever need to talk to
| Google.
|
| Mostly this is just wasteful and not a big issue, but then
| come the conflicting dependencies you need to resolve by
| pinning version numbers. This is doubly annoying if you have
| no idea what the dependency is needed for or even if it's
| needed at all.
|
| This is not just a python issue, it's apparent in all
| languages I've worked in.
| LordKeren wrote:
| I inherited a legacy code base that suddenly had a lot more
| attention on it and had to be rapidly overhauled. Deptry was
| invaluable for quickly removing a large amount of unused
| dependencies.
|
| The newest update was a delightful surprise since deptry is now
| fast enough to run during automated pull-request analysis via
| GitHub actions
| flo12392 wrote:
| Thanks for the positive feedback! Happy to hear that it turned
| out to be useful for you.
| passion__desire wrote:
| Does it provide a tree output of the dependencies as they do
| in Scala/Java land?
| flo12392 wrote:
| If I understand your question correctly; no, that is out of
| scope for deptry. Many of the dependency management tools
| already provide a way to do so, e.g. for PDM `pdm list
| --tree`.
| passion__desire wrote:
| You could provide diff of current tree vs tree after
| applying your suggested removals? It is more complex than
| that ?
| BerislavLopac wrote:
| Use pipdeptree for that
| infocollector wrote:
| Speed is impressive. Accuracy could use some work. Input for
| authors:
|
| 1. DEP003 'click' imported but it is a transitive dependency (In
| my code base click is used @click - which it doesn't look like it
| sees)
|
| 2. DEP003 'numcodecs' imported but it is a transitive dependency
| Usage was open(...,compressor=numcodecs.xxx,...)
|
| Perhaps these warnings/errors should be removed?
| eatox wrote:
| On the contrary, it does see those imports, and indicates that
| those dependencies are used, but are not direct dependencies in
| your dependency tree, and instead installed because other
| dependencies rely on them:
| https://fpgmaas.github.io/deptry/rules-violations/#transitiv...
| globular-toast wrote:
| That's really useful. I find it really common for people to
| import transitive dependencies in python. A common one is
| putting something like matplotlib in deps then directly
| importing numpy. Seems to happen all the time.
| infocollector wrote:
| Thank you for catching this! Indeed its not a bug.
| marcooliv wrote:
| Congrats and thank you! I have a ticket on my backlog to clear
| our main project non-used dependencies. haha
| flo12392 wrote:
| Don't tell your colleagues about deptry, take the afternoon off
| and tell your manager you manually removed each dependency one
| by one and ran the unit tests to check if the dependency was
| needed ;)
| kylegalbraith wrote:
| Does this work with uv as well?
| flo12392 wrote:
| Good question, I have not looked into this myself yet. I would
| say; definitely, since it uses the dependencies from a
| `requirements.in` or a `pyproject.toml` file to generate a set
| of locked dependencies. Since `deptry` supports both formats,
| this should be no problem.
|
| One thing to keep in mind, that would also be good for `deptry`
| to think about; if users decide to create a `requirements.txt`
| file from a `requirements.in` file, `deptry` will automatically
| use the `requirements.txt` file. However, this is wrong: It
| should use the `requirements.in` file, since the
| `requirements.txt` file also contains the transitive
| dependencies. Users can change this using the `requirements-
| txt` flag though
| (https://fpgmaas.github.io/deptry/usage/#requirements-txt).
|
| Thanks for raising this, I will probably create an issue in our
| project to see how we should feal with this :)
| flo12392 wrote:
| There we go: https://github.com/fpgmaas/deptry/issues/608
| singhrac wrote:
| I have used deptry for my most recent set of projects and it is
| so useful to make sure your project is movable to another venv
| (or PEP518 compliant).
| ditsuke wrote:
| I recently introduced tweag's fawltydeps for this use-case at
| work, looking forward to try deptry when I run into this problem
| again..
___________________________________________________________________
(page generated 2024-03-16 23:01 UTC)