https://github.com/IBM/import-tracker Skip to content Sign up * Product + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} IBM / import-tracker Public * Notifications * Fork 3 * Star 164 Python utility for tracking third party dependencies within a library License MIT license 164 stars 3 forks Star Notifications * Code * Issues 4 * Pull requests 0 * Actions * Projects 1 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 3 branches 18 tags Code Latest commit @gabe-l-hart gabe-l-hart Merge pull request #39 from IBM/SingleExtras-38 ... 7f599ba May 25, 2022 Merge pull request #39 from IBM/SingleExtras-38 Single extras 38 7f599ba Git stats * 211 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github StandardRepoReadmes: Fmt on issue templates, workflows, and CODE_OF_C... Dec 2, 2021 import_tracker SingleExtras: Fix error checking for invalid extras modules in single... May 25, 2022 scripts AutoTrackingRewrite: Set up parallel tests Feb 22, 2022 test SingleExtras: Test for single extras case May 25, 2022 .coveragerc import_tests: Add test running setup Nov 29, 2021 .gitignore AutoTrackingRewrite: Add bash history mounting Feb 24, 2022 .isort.cfg BlackFormatting: Add isort configuration Nov 30, 2021 .pre-commit-config.yaml HandleSideEffectLibraries: Bump black version for fix around click br... Apr 8, 2022 .prettierignore BlackFormatting: Use 4-space tabs for prettier and ignore generated file Nov 30, 2021 .prettierrc.yaml BlackFormatting: Use 4-space tabs for prettier and ignore generated file Nov 30, 2021 CODE_OF_CONDUCT.md StandardRepoReadmes: Fmt on issue templates, workflows, and CODE_OF_C... Dec 2, 2021 CONTRIBUTING.md StandardRepoReadmes: Fix formatting for CONTRIBUTING.md Dec 2, 2021 Dockerfile StandardRepoReadmes: Run fmt.sh as part of the test phase Dec 2, 2021 LICENSE main: Add license file Dec 1, 2021 Makefile AutoTrackingRewrite: Add bash history mounting Feb 24, 2022 README.md ReadmeUpdates: Add recently-added args to the README Apr 13, 2022 requirements_test.txt AutoTrackingRewrite: Set up parallel tests Feb 22, 2022 setup.py main: HOTFIX: Fix the homepage URL Feb 25, 2022 View code Import Tracker Running Import Tracker Integrating import_tracker into a projects Enabling lazy_import_errors Using setup_tools.parse_requirements README.md Import Tracker Import Tracker is a Python package offering a number of capabilities related to tracking and managing optional dependencies in Python projects. Specifically, this project enables developers to: * Track the dependencies of a python project to map each module within the project to the set of dependencies it relies on. This can be useful for debugging of dependencies in large projects. * Enable lazy import errors in a python projects to prevent code from crashing when uninstalled imports are imported, but not utilized. This can be helpful in large projects, especially those which incorporate lots of hierarchical wild imports, as importing the top level package of such projects can often bring a lot of heavy dependencies into sys.modules. * Programmatically determine the install_requires and extras_require arguments to setuptools.setup where the extras sets are determined by a set of modules that should be optional. Running Import Tracker To run import_tracker against a project, simply invoke the module's main: python3 -m import_tracker --name The main supports the following additional arguments: * --package: Allows --name to be a relative import (see importlib.import_module) * --recursive: Recurse into sub-modules within the given module to produce the full mapping for the given module * --num_jobs: When --recurse is given, run the recursive jobs concurrently with the given number of workers (0 implies serial execution) * --indent: Indent the output json for pretty printing * --log_level: Set the level of logging (up to debug4) to debug unexpected behavior * --submodules: List of sub-modules to recurse on (only used when --recursive set) * --side_effect_modules: Modules with known import-time side effect which should always be allowed to import * --track_import_stack: Store the stack trace of imports belonging to the tracked module + NOTE: This is very slow and should be used only when performing specific debugging tasks! Integrating import_tracker into a projects When using import_tracker to implement optional dependencies in a project, there are two steps to take: 1. Enable lazy_import_errors for the set of modules that should be managed as optional 2. Use setup_tools.parse_requirements in setup.py to determine the install_requires and extras_require arguments In the following examples, we'll use a fictitious project with the following structure: my_module/ +-- __init__.py +-- utils.py +-- widgets +-- __init__.py +-- widget1.py +-- widget2.py Enabling lazy_import_errors The import_tracker.lazy_import_errors function can be invoked directly to enable lazy import errors globally, or used as a context manager to enable them only for a selcted set of modules. To globally enable lazy import errors, my_module/__init__.py would look like the following: # Globally enable lazy import errors from import_tracker import lazy_import_errors lazy_import_errors() from . import utils, widgets Alternately, applying lazy import error semantics only to the widgets would look like the following: from import_tracker import lazy_import_errors # Require all downstream imports from utils to exist from . import utils # Enable lazy import errors for widgets with lazy_import_errors(): from . import widgets When using lazy import errors, there are two ways to customize the error message that is raised when a failed import is used: 1. 1. The get_extras_modules argument takes a function which returns a Set[str] of the module names that are tracked as extras. If the import error is triggered within a module that is managed as an extras set, the error message is updated to include instructions on which extras set needs to be installed. 2. The make_error_message argument allows the caller to specify a fully custom error message generation function. Using setup_tools.parse_requirements To take advantage of the automatic dependency parsing when building a package, the setup.py would look like the following: import import_tracker import os import setuptools # Determine the path to the requirements.txt for the project requirements_file = os.path.join(os.path.dirname(__file__), "requirements.txt") # Parse the requirement sets install_requires, extras_require = import_tracker.setup_tools.parse_requirements( requirements_file=requirements_file, library_name="my_module", extras_modules=[ "my_module.widgets.widget1", "my_module.widgets.widget2", ], ) # Perform the standard setup call setuptools.setup( name="my_module", author="me", version="1.2.3", license="MIT", install_requires=install_requires, extras_require=extras_require, packages=setuptools.find_packages(), ) About Python utility for tracking third party dependencies within a library Resources Readme License MIT license Code of conduct Code of conduct Stars 164 stars Watchers 3 watching Forks 3 forks Releases 18 2.0.2 - Single extras set Latest May 25, 2022 + 17 releases Packages 0 No packages published Contributors 3 * * * Languages * Python 93.2% * Shell 4.0% * Dockerfile 1.8% * Makefile 1.0% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.