https://github.com/Tiendil/smart-imports

Skip to content
 
Sign up

  * Why GitHub?
    Features -
      + Mobile -
      + Actions -
      + Codespaces -
      + Packages -
      + Security -
      + Code review -
      + Project management -
      + Integrations -
      + GitHub Sponsors -
      + Customer stories-
      + Security -
  * Team
  * Enterprise
  * Explore
      + Explore GitHub -

    Learn and contribute

      + Topics -
      + Collections -
      + Trending -
      + Learning Lab -
      + Open source guides -

    Connect with others

      + The ReadME Project -
      + Events -
      + Community forum -
      + GitHub Education -
      + GitHub Stars program -
  * Marketplace
  * Pricing
    Plans -
      + Compare plans -
      + Contact Sales -
      + Nonprofit -
      + Education -

[                    ] [search-key]

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this user All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in Sign up
{{ message }}

Tiendil / smart-imports

  * Watch 1
  * Star 14
  * Fork 4

smart imports for Python

BSD-3-Clause License
14 stars 4 forks
Star
Watch

  * Code
  * Issues 5
  * Pull requests 0
  * Actions
  * Projects 0
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Security
  * Insights

develop
Switch branches/tags
[                    ]
Branches Tags
Nothing to show
{{ refName }} default View all branches
Nothing to show
{{ refName }} default
View all tags
2 branches 6 tags
Go to file Code
 
Clone
HTTPS GitHub CLI
[https://github.com/T]

Use Git or checkout with SVN using the web URL.

[gh repo clone Tiendi]

Work fast with our official CLI. Learn more.

  * Open with GitHub Desktop
  * Download ZIP

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Go back

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Go back

Launching Xcode

If nothing happens, download Xcode and try again.

Go back

Launching Visual Studio

If nothing happens, download the GitHub extension for Visual Studio
and try again.

Go back

Latest commit

@Tiendil
Tiendil Finish 0.2.6
...
efadf92 Oct 28, 2020
Finish 0.2.6
efadf92

Git stats

  * 63 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
smart_imports
gh-16 Support Python 3.9
Oct 28, 2020
.gitignore
demo ast_parser
Jun 10, 2018
.travis.yml
gh-16 Support Python 3.9
Oct 28, 2020
CHANGELOG.rst
gh-16 Support Python 3.9
Oct 28, 2020
LICENSE
Initial commit
Jun 10, 2018
MANIFEST.in
code for import rules
Jun 15, 2018
README.rst
gh-16 Support Python 3.9
Oct 28, 2020
setup.py
gh-16 Support Python 3.9
Oct 28, 2020
View code

README.rst

 Smart import for Python

Latest version released on PyPi Supported Python versions Test
coverage in develop

  * Gitter chat room
  * Changelog

Automatically discovers & imports entities, used in the current
module.

No magic or monkey patching. Only standard Python functionality.

                Before                                   After
import math                             import smart_imports
from my_project import calc             smart_imports.all()
# 100500 other imports                  # no any other imports

def my_code(argument, function=calc):   def my_code(argument, function=calc):
    return math.log(function(argument))     return math.log(function(argument))

MyPy supported.

 Summary

  * Get source code of the module, from which smart_imports.all() has
    called.
  * Parse it, find all not initialized variables.
  * Search imports, suitable for found variables.
  * Import them.

Library process only modules, from which smart_imports called
explicitly.

 Main idea

With time every complex project develops own naming convention. If we
translate that convention into more formal rules, we will be able to
make automatic imports of every entity, knowing only its name.

For example, we will not need to write import math to call math.pi,
since our system will understand that math is the module of the
standard library.

 How it works

Code from the header works in such way:

  * smart_imports.all() builds AST of the module from which it has
    called.
  * Library analyses AST and searches for not initialized variables.
  * Name of every found variable processed thought chain of rules to
    determine the correct module (or its attribute) to import. If the
    rule finds the target module, chain breaks and the next rules
    will not be processed.
  * Library load found modules and add imported entities into the
    global namespace.

Smart Imports searches not initialized variables in every part of
code (including new Python syntax).

Automatic importing turns on only for modules, that do explicit call
of smart_imports.all().

Moreover, you can use normal imports with Smart Imports at the same
time. That helps to integrate Smart Imports step by step.

You can notice, that AST of module builts two times:

  * when CPython imports module;
  * when Smart Imports process call of smart_imports.all().

We can build AST once (for that we can add hook into the process of
importing modules with help of PEP-0302), but it will make import
event slower. I think that it is because at import time CPython
builds AST in terms of its internal structures (probably implemented
in C). Conversion from them to Python AST cost more than building new
AST from scratch.

Smart Imports build AST only once for every module.

 Default import rules

Smart Imports can be used without configuration. By default it uses
such rules:

 1. By exact match looks for the module with the required name in the
    folder of the current module.
 2. Checks if the standard library has a module with the required
    name.
     1. By exact match with top-level packages (for example, math ).
     2. For sub-packages and modules checks complex names with dots
        replaced by underscores (for example, os.path will be
        imported for name os_path).
 3. By exact match looks for installed packages with the required
    name (for example, requests ).

 Performance

Smart Imports does not slow down runtime but increases startup time.

Because of building AST, startup time increased in 1.5-2 times. For
small projects it is inconsequential. At the same time, the startup
time of large projects depends mostly on architecture and
dependencies between modules, than from the time of modules import.

In the future, part of Smart Imports can be rewritten in C -- it
should eliminate startup delays.

To speed up startup time, results of AST processing can be cached on
the file system. That behavior can be turned on in the config.
SmartImports invalidates cache when module source code changes.

Also, Smart Imports' work time highly depends on rules and their
sequence. You can reduce these costs by modifying configs. For
example, you can specify an explicit import path for a name with Rule
4: custom names.

 Configuration

The logic of default configuration was already described. It should
be enough to work with the standard library.

Default config:

{
    "cache_dir": null,
    "rules": [{"type": "rule_local_modules"},
              {"type": "rule_stdlib"},
              {"type": "rule_predefined_names"},
              {"type": "rule_global_modules"}]
}

If necessary, a more complex config can be put on a file system.

Example of complex config (from my pet project).

At the time of call smart_import.all() library detects a location of
config file by searching file smart_imports.json from the current
folder up to root. If a file will be found, it will become config for
the current module.

You can use multiple config files (place them in different folders).

There are few config parameters now:

{
    // folder to store cached AST
    // if not specified or null, cache will not be used
    "cache_dir": null|"string",

    // list of import rules (see further)
    "rules": []
}

 Import rules

A sequence of rules in configs determines the order of their
application. The first success rule stops processing and makes
import.

Rule 1: predefined names will be often used in the examples below. It
required for the correct processing of default names like print.

 Rule 1: predefined names

Rule silences import search for predefined names like __file__ and
builtins like print.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"}]
# }

import smart_imports

smart_imports.all()

# Smart Imports will not search for module with name __file__
# event if variable is not initialized explicity in code
print(__file__)

 Rule 2: local modules

Rule checks if a module with the required name exists in the folder
of the current module. If the module found, it will be imported.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_local_modules"}]
# }
#
# project on file sytem:
#
# my_package
# |-- __init__.py
# |-- a.py
# |-- b.py

# b.py
import smart_imports

smart_imports.all()

# module "a" will be found and imported
print(a)

 Rule 3: global modules

Rule tries to import the module by name.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_global_modules"}]
# }
#
# install external package
#
# pip install requests

import smart_imports

smart_imports.all()

# module "requests" will be found and imported
print(requests.get('http://example.com'))

 Rule 4: custom names

Rule links a name to the specified module and its attribute
(optionally).

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_custom",
#               "variables": {"my_import_module": {"module": "os.path"},
#                             "my_import_attribute": {"module": "random", "attribute": "seed"}}}]
# }

import smart_imports

smart_imports.all()

# we use modules of the standard library in that example
# but any module can be used
print(my_import_module)
print(my_import_attribute)

 Rule 5: standard library

Rule checks if the standard library has a module with the required
name. For example math or os.path (which will be imported for the
name os_path).

That rule works faster than Rule 3: global modules, since it searches
module by predefined list. Lists of modules for every Python version
was collected with help of stdlib-list.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_stdlib"}]
# }

import smart_imports

smart_imports.all()

print(math.pi)

 Rule 6: import by prefix

Rule imports module by name from the package, which associated with
name prefix. It can be helpful when you have a package used in the
whole project. For example, you can access modules from package utils
with prefix utils_.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_prefix",
#               "prefixes": [{"prefix": "utils_", "module": "my_package.utils"}]}]
# }
#
# project on filesystem
#
# my_package
# |-- __init__.py
# |-- utils
# |-- |-- __init__.py
# |-- |-- a.py
# |-- |-- b.py
# |-- subpackage
# |-- |-- __init__.py
# |-- |-- c.py

# c.py

import smart_imports

smart_imports.all()

print(utils_a)
print(utils_b)

 Rule 7: modules from parent package

If you have sub-packages with the same name in different parts of
your project (for example, tests or migrations), you can allow for
them to search modules by name in parent packages.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_local_modules_from_parent",
#               "suffixes": [".tests"]}]
# }
#
# project on file system:
#
# my_package
# |-- __init__.py
# |-- a.py
# |-- tests
# |-- |-- __init__.py
# |-- |-- b.py

# b.py

import smart_imports

smart_imports.all()

print(a)

 Rule 8: modules from namespace

The rule allows for modules from a specified package to import by
name modules from another package.

# config:
# {
#    "rules": [{"type": "rule_predefined_names"},
#              {"type": "rule_local_modules_from_namespace",
#               "map": {"my_package.subpackage_1": ["my_package.subpackage_2"]}}]
# }
#
# project on filesystem:
#
# my_package
# |-- __init__.py
# |-- subpackage_1
# |-- |-- __init__.py
# |-- |-- a.py
# |-- subpackage_2
# |-- |-- __init__.py
# |-- |-- b.py

# a.py

import smart_imports

smart_imports.all()

print(b)

 How to add custom rule?

 1. Subclass smart_imports.rules.BaseRule.
 2. Implement required logic.
 3. Register rule with method smart_imports.rules.register.
 4. Add rule to config.
 5. ???
 6. Profit.

Look into the implementation of current rules, if you need an
example.

 MyPY

Plugin for integration with MyPy implemented.

MyPy config (mypy.ini) example:

[mypy]
plugins = smart_imports.plugins.mypy

 Plans

I love the idea of determining code properties by used names. So, I
will try to develop it in the borders of Smart Imports and other
projects.

What I planning for Smart Imports:

  * Continue support and patch it for new versions of Python.
  * Research usage of type annotations to import automatization.
  * Try to implement lazy imports.
  * Implement utilities for automatic config generation and code
    refactoring.
  * Rewrite part of code in C, to speedup AST construction.
  * Implement integrations with popular IDEs.

I open to your suggestions. Feel free to contact me in any way.

About

smart imports for Python

Topics

python automation metaprogramming code-analysis imports

Resources

Readme

License

BSD-3-Clause License

Releases

6 tags

Packages 0

No packages published

Languages

  * Python 100.0%

  * (c) 2021 GitHub, Inc.
  * Terms
  * Privacy
  * Security
  * Status
  * Docs

 

  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.