https://pypi.org/project/specialist/ Skip to main content Switch to mobile version Warning Some features may not work without JavaScript. Please try enabling it if you encounter problems. PyPI Search PyPI [ ] Search * Help * Sponsors * Log in * Register Menu * Help * Sponsors * Log in * Register Search PyPI [ ] Search specialist 0.4.2 pip install specialist Copy PIP instructions Latest version Released: Oct 24, 2022 Visualize CPython 3.11's specializing, adaptive interpreter. Navigation * Project description * Release history * Download files Project links * Homepage Statistics GitHub statistics: * Stars: * Forks: * Open issues/PRs: View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery Meta License: MIT Author: Brandt Bucher Requires: Python >=3.11 Maintainers Avatar for brandtbucher from gravatar.com brandtbucher * Project description * Project details * Release history * Download files Project description Specialist latest versionlatest release datebuild statusissues Specialist uses fine-grained location information to create visual representations of exactly where and how CPython 3.11's new specializing, adaptive interpreter optimizes your code. [6874747073] Installation Specialist supports CPython 3.11+ on all platforms. To install, just run: $ pip install specialist Background While CPython 3.11 is running your code, it identifies "hot" regions that are being run often enough to spend time optimizing. It occasionally "quickens" these regions, which specialist represents using color. Dark, rich colors indicate code with many quickened instructions (and, therefore, high specialization potential), while light, pale colors indicate code with relatively few specialization opportunities. Most of the time, quickening involves three phases: * Replacing individual bytecode instructions with "adaptive" forms. These are actually a bit slower than normal instructions, because they attempt to "specialize" themselves periodically. If they are unable to specialize, they will remain in their adaptive form. specialist uses red to indicate the presence of adaptive instructions. * Occasionaly, adaptive instructions will convert themselves into much faster "specialized" instructions. Examples of specializations include attribute accesses on a single object or type, certain pure-Python calls, or integer addition. specialist uses green to indicate the presence of specialized instructions. * If a specialization becomes invalid after some time (for example, if an expression that previously added two integers starts concatenating two strings instead), the specialized instruction may be converted back into an adaptive one. At this point, the cycle repeats itself. Specialist aims to provide insight into this process for the maintainers of CPython itself, as well as for users seeking an optimization profile for their own code. Tutorial Suppose we have the following source file, conversions.py, which contains some utilities and tests for converting between Fahrenheit and Celsius: import math def f_to_c(f: float) -> float: """Convert Fahrenheit to Celsius.""" x = f - 32 return x * 5 / 9 def c_to_f(c: float) -> float: """Convert Celsius to Fahrenheit.""" x = c * 9 / 5 return x + 32 TEST_VALUES = [-459.67, -273.15, 0.0, 32.0, 42.0, 273.15, 100.0, 212.0, 373.15] def test_conversions() -> None: for t in TEST_VALUES: assert_round_trip(t) def assert_round_trip(t: float) -> None: # Round-trip Fahrenheit through Celsius: assert math.isclose(t, f_to_c(c_to_f(t))), f"{t} F -> C -> F failed!" # Round-trip Celsius through Fahrenheit: assert math.isclose(t, c_to_f(f_to_c(t))), f"{t} C -> F -> C failed!" if __name__ == "__main__": test_conversions() We can run this file with CPython 3.11 from the command-line using specialist: $ specialist conversions.py After the script has finished running, specialist will open a web browser and display the annotated program source: [6874747073] The green areas indicate regions of code that were successfully specialized, while the red areas indicate unsuccessful specializations (in the form of "adaptive" instructions). Mixed results are indicated by colors along the green-yellow-orange-red gradient, depending on the ratio of successes to failures. Regions of code that don't contain any attempted specializations are left white. Focusing on f_to_c and c_to_f for a moment, we can see that CPython is failing to specialize addition and subtraction by 32. It doesn't currently specialize binary operators between mixed float and int values, which is exactly what the code here is doing. It can, however, specialize addition and subtraction between two float values! Replacing 32 with 32.0 results in successful specializations (confirmed by re-running specialist): [6874747073] We can see that something similar is happening with float and int multiplication as well. One option could be to continue converting constant values to float: [6874747073] However, there's a better option! Notice that CPython doesn't attempt to specialize division at all (it's left white in the visualization). We can take advantage of CPython's constant folding optimizations by slightly changing the order of operations, which allows our scaling factors (5 / 9 and 9 / 5) to be computed at compile-time. When we do this, CPython is able to implement our converters entirely using native floating-point operations: [6874747073] A few notes on the remaining code: * The global lookup of TEST_VALUES is red because it hasn't had the opportunity to be specialized yet. Though CPython was able to identify test_conversions as "hot" code and quicken the body, this didn't happen until after TEST_VALUES was looked up (which happens only once). It would be wasteful to spend time optimizing code that is never run again! * Similarly, parts of the assert statements in assert_round_trip are red because they are "dead" code that never actually runs. * The calls to math.is_close are orange because it is implemented in C. C extensions can't be "inlined" the same way as pure-Python calls likec_to_f, f_to_c, and assert_round_trip, so most of the call sequence isn't able to be specialized. Modes Like python itself, specialist can run code a few different ways. It can be given a file path: $ specialist spam/eggs.py foo bar baz Or a module name: $ specialist -m spam.eggs foo bar baz Or a command: $ specialist -c 'import spam; spam.eggs()' foo bar baz It also has a -t/--targets option to support discovery of arbitrary "target" files to analyze after the script completes. This is useful if the script being run is different from the code you want to visualize: $ specialist --targets spam/eggs.py -c 'import uses_eggs; uses_eggs.run()' Multiple files can be provided using "glob" style patterns: $ specialist --targets 'spam/**/*.py' -m pytest Specialist can also write the generated HTML files to the filesystem instead of opening them in a browser. To do so, just provide an output directory path using the -o/--output option: $ specialist --output ../report --targets 'spam/**/*.py' -m pytest /home/brandtbucher/sketch/spam/__init__.py -> /home/brandtbucher/report/__init__.html /home/brandtbucher/sketch/spam/_spammy.py -> /home/brandtbucher/report/_spammy.html /home/brandtbucher/sketch/spam/eggs/__init__.py -> /home/brandtbucher/report/eggs/__init__.html /home/brandtbucher/sketch/spam/eggs/_eggy.py -> /home/brandtbucher/report/eggs/_eggy.html Options -b/--blue Use blue (rather than green) to indicate specialized code. Some users may find a blue-violet-magenta-red gradient easier to read than the default green-yellow-orange-red one. -d/--dark Use light text on a dark background. Some users may find a dark scheme makes them feel cooler than the default light one. Project details Project links * Homepage Statistics GitHub statistics: * Stars: * Forks: * Open issues/PRs: View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery Meta License: MIT Author: Brandt Bucher Requires: Python >=3.11 Maintainers Avatar for brandtbucher from gravatar.com brandtbucher Release history Release notifications | RSS feed This version [blue-cube] 0.4.2 Oct 24, 2022 [white-cube] 0.4.1 Oct 7, 2022 [white-cube] 0.4.0 Oct 4, 2022 [white-cube] 0.3.0 Sep 17, 2022 [white-cube] 0.2.2 Sep 14, 2022 [white-cube] 0.2.1 Sep 12, 2022 [white-cube] 0.2.0 Aug 19, 2022 [white-cube] 0.1.1 Jun 9, 2022 [white-cube] 0.0.0 Jun 6, 2022 Download files Download the file for your platform. If you're not sure which to choose, learn more about installing packages. Source Distribution specialist-0.4.2.tar.gz (10.4 kB view hashes) Uploaded Oct 24, 2022 source Built Distribution specialist-0.4.2-py3-none-any.whl (10.9 kB view hashes) Uploaded Oct 24, 2022 py3 Close Hashes for specialist-0.4.2.tar.gz Hashes for specialist-0.4.2.tar.gz Algorithm Hash digest SHA256 8cfca206f972dc8d98163b6ab92a757c58652128954088a1693e1ac9ec896b94 Copy MD5 a37c4e85252a7db4c133af40f93ccb95 Copy BLAKE2-256 30cf25842ecc9b995817343cfec743bddd752683ada3e443289d0fca9c03a421 Copy Close Close Hashes for specialist-0.4.2-py3-none-any.whl Hashes for specialist-0.4.2-py3-none-any.whl Algorithm Hash digest SHA256 e3307544bfc8cb11571029a920b1213253d069b1c354a3bc38a1dbab87ccc26e Copy MD5 6a68b9470fe872da3f3b46d827943f71 Copy BLAKE2-256 5528e9657a81d1c5eb3e69b84abe7caa2e746c6ed45fd43877bf2bc7110a0b11 Copy Close [white-cube] Help * Installing packages * Uploading packages * User guide * FAQs About PyPI * PyPI on Twitter * Infrastructure dashboard * Package index name retention * Our sponsors Contributing to PyPI * Bugs and feedback * Contribute on GitHub * Translate PyPI * Development credits Using PyPI * Code of conduct * Report security issue * Privacy policy * Terms of use --------------------------------------------------------------------- Status: all systems operational Developed and maintained by the Python community, for the Python community. Donate today! "PyPI", "Python Package Index", and the blocks logos are registered trademarks of the Python Software Foundation. (c) 2022 Python Software Foundation Site map Switch to desktop version * English * espanol * francais * Ri Ben Yu * portugues (Brasil) * ukrayins'ka * Ellenika * Deutsch * Zhong Wen (Jian Ti ) * Zhong Wen (Fan Ti ) * russkii * `bryt * esperanto Supported by AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page