https://github.com/bloomberg/memray Skip to content Sign up * Product + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} bloomberg / memray Public * Notifications * Fork 20 * Star 1k Memray is a memory profiler for Python bloomberg.github.io/memray/ Apache-2.0 License 1k stars 20 forks Star Notifications * Code * Issues 1 * Pull requests 2 * Actions * Security * Insights More * Code * Issues * Pull requests * Actions * Security * Insights This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 2 branches 1 tag Code Latest commit @pablogsal @lkollar pablogsal and lkollar Apply suggestions from code review ... 2d7a2b5 Apr 20, 2022 Apply suggestions from code review Co-authored-by: Laszlo Kiss Kollar 2d7a2b5 Git stats * 614 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github Improve the CI experience Apr 12, 2022 benchmarks Memray open source bootstrap Apr 9, 2022 docs Add memray logo Apr 20, 2022 news Memray open source bootstrap Apr 9, 2022 src Split test utilities into a separate .pyx file Apr 20, 2022 tests Split test utilities into a separate .pyx file Apr 20, 2022 .babelrc Migrate to @babel/preset-env Jun 28, 2021 .bumpversion.cfg Prepare for 1.0.2 release Apr 13, 2022 .clang-format Memray open source bootstrap Apr 9, 2022 .flake8 Suppress E203 raised by flake8 May 7, 2021 .gitignore Update the package name everywhere Apr 15, 2022 CONTRIBUTING.md docs: Remove reference to type comments Apr 19, 2022 Dockerfile Memray open source bootstrap Apr 9, 2022 LICENSE Memray open source bootstrap Apr 9, 2022 MANIFEST.in Memray open source bootstrap Apr 9, 2022 Makefile Don't build the generated javascript files when running the build Apr 12, 2022 NEWS.rst Prepare for 1.0.2 release Apr 13, 2022 README.md Apply suggestions from code review Apr 20, 2022 asv.conf.json Memray open source bootstrap Apr 9, 2022 package-lock.json Don't build the generated javascript files when running the build Apr 12, 2022 package.json Memray open source bootstrap Apr 9, 2022 pyproject.toml Add names to CI targets Apr 12, 2022 requirements-docs.txt Split documentation dependencies in separate file Jan 31, 2022 requirements-extra.txt Split documentation dependencies in separate file Jan 31, 2022 requirements-test.txt Fiat Lux Sep 21, 2020 setup.cfg Fiat Lux Sep 21, 2020 setup.py Update the package name everywhere Apr 15, 2022 tox.ini Substitute more references to the old package name Apr 12, 2022 valgrind.supp Memray open source bootstrap Apr 9, 2022 webpack.config.js Memray open source bootstrap Apr 9, 2022 View code [ ] Installation Building from source Documentation Usage Pytest plugin Native mode Live mode Sorting results Viewing different threads License Code of Conduct Security Policy Contributing Contribution Licensing Steps README.md [logo] --------------------------------------------------------------------- PyPI - Python Version PyPI - Implementation PyPI PyPI - Downloads Tests Code Style Memray output Memray is a memory profiler for Python. It can track memory allocations in Python code, in native extension modules, and in the Python interpreter itself. It can generate several different types of reports to help you analyze the captured memory usage data. While commonly used as a CLI tool, it can also be used as a library to perform more fine-grained profiling tasks. Notable features: * [?][?] Traces every function call so it can accurately represent the call stack, unlike sampling profilers. * Also handles native calls in C/C++ libraries so the entire call stack is present in the results. * Blazing fast! Profiling causes minimal slowdown in the application. Tracking native code is somewhat slower, but this can be enabled or disabled on demand. * It can generate various reports about the collected memory usage data, like flame graphs. * Works with Python threads. * Works with native-threads (e.g. C++ threads in C extensions). Memray can help with the following problems: * Analyze allocations in applications to help discover the cause of high memory usage. * Find memory leaks. * Find hotspots in code which cause a lot of allocations. Note that Memray only works on Linux and cannot be installed on other platforms. Installation Memray requires Python 3.7+ and can be easily installed using most common Python packaging tools. We recommend installing the latest stable release from PyPI with pip: pip install memray Notice that Memray contains a C extension so releases are distributed as binary wheels as well as the source code. If a binary wheel is not available for your system (Linux x86/x64), you'll need to ensure that all the dependencies are satisfied on the system where you are doing the installation. Building from source If you wish to build Memray from source you need the following binary dependencies in your system: * libunwind Check your package manager on how to install these dependencies (for example apt-get install libunwind-dev in Debian-based systems). Once you have the binary dependencies installed, you can clone the repository and follow with the normal building process: git clone git@github.com:bloomberg/memray.git memray cd memray python3 -m venv ../memray-env/ # just an example, put this wherever you want source ../memray-env/bin/activate pip install --upgrade pip pip install -e . -r requirements-test.txt -r requirements-extra.txt This will install Memray in the virtual environment in development mode (the -e of the last pip install command). Documentation You can find the latest documentation available here. Usage There are many ways to use Memray. The easiest way is to use it as a command line tool to run your script, application or library. usage: memray [-h] [-v] {run,flamegraph,table,live,tree,parse,summary,stats} ... Memory profiler for Python applications Run `memray run` to generate a memory profile report, then use a reporter command such as `memray flamegraph` or `memray table` to convert the results into HTML. Example: $ python3 -m memray run my_script.py -o output.bin $ python3 -m memray flamegraph output.bin positional arguments: {run,flamegraph,table,live,tree,parse,summary,stats} Mode of operation run Run the specified application and track memory usage flamegraph Generate an HTML flame graph for peak memory usage. table Generate an HTML table with all records in the peak memory usage. live Remotely monitor allocations in a text-based interface. tree Generate an tree view in the terminal for peak memory usage. parse Debug a results file by parsing and printing each record in it. summary Generate a terminal-based summary report of the functions that allocate most memory stats Generate high level stats of the memory usage in the terminal optional arguments: -h, --help show this help message and exit -v, --verbose Increase verbosity. Option is additive, can be specified up to 3 times. Please submit feedback, ideas and bugs by filing a new issue at https://github.com/bloomberg/memray/issues To use Memray over a script or a single python file you can use python3.x -m memray run my_script.py (where 3.x is the version of Python you installed Memray for). If you normally run your application with python3 -m my_module, you can use the -m flag with memray run: python3.x -m memray run -m my_module You can also invoke Memray as a command line tool without having to use -m to invoke it as a module: memray3.x run my_script.py memray3.x run -m my_module The output will be a binary file (like memray-my_script.2369.bin) that you can analyze in different ways. One way is to use the memray flamegraph command to generate a flame graph: memray3.x flamegraph my_script.2369.bin This will produce an HTML file with a flame graph of the memory usage that you can inspect with your favorite browser. There are multiple other reporters that you can use to generate other types of reports, some of them generating terminal-based output and some of them generating HTML files. Here is an example of a Memray flamegraph: [flamegraph] Pytest plugin If you want an easy and convenient way to use memray in your test suite, you can consider using pytest-memray. Once installed, this pytest plugin allows you to simply add --memray to the command line invocation: pytest --memray tests/ And will automatically get a report like this: python3 -m pytest tests --memray =============================================================================================================================== test session starts ================================================================================================================================ platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 rootdir: /mypackage, configfile: pytest.ini plugins: cov-2.12.0, memray-0.1.0 collected 21 items tests/test_package.py ..................... [100%] ================================================================================================================================= MEMRAY REPORT ================================================================================================================================== Allocations results for tests/test_package.py::some_test_that_allocates Total memory allocated: 24.4MiB Total allocations: 33929 Histogram of allocation sizes: |# # | Biggest allocating functions: - parse:/opt/bb/lib/python3.8/ast.py:47 -> 3.0MiB - parse:/opt/bb/lib/python3.8/ast.py:47 -> 2.3MiB - _visit:/opt/bb/lib/python3.8/site-packages/astroid/transforms.py:62 -> 576.0KiB - parse:/opt/bb/lib/python3.8/ast.py:47 -> 517.6KiB - __init__:/opt/bb/lib/python3.8/site-packages/astroid/node_classes.py:1353 -> 512.0KiB You can also use some of the included markers to make tests fail if the execution of said test allocates more memory than allowed: @pytest.mark.limit_memory("24 MB") def test_foobar(): # do some stuff that allocates memory To learn more on how the plugin can be used and configured check out the plugin documentation. Native mode Memray supports tracking native C/C++ functions as well as Python functions. This can be especially useful when profiling applications that have C extensions (such as numpy or pandas) as this gives holistic vision of how much memory is allocated by the extension and how much is allocated by Python itself. To activate native tracking, you need to provide the --native argument when using the run subcommand: python3.x -m memray run --native my_script.py This will automatically add native information to the result file and it will be automatically used by any reporter (such the flamegraph or table reporters). This means that instead of seeing this in the flamegraphs: [mandelbrot] You will now be able to see what's happening inside the Python calls: [mandelbrot] Reporters display native frames in a different color than Python frames. They can also be distinguished by looking at the file location in a frame (Python frames will generally be generated from files with a .py extension while native frames will be generated from files with extensions like .c, .cpp or .h). Live mode Memray's live mode runs a script or a module in a terminal-based interface that allows you to interactively inspect its memory usage while it runs. This is useful for debugging scripts or modules that take a long time to run or that exhibit multiple complex memory patterns. You can use the --live option to run the script or module in live mode: python3.x -m memray run --live my_script.py or if you want to execute a module: python3.x -m memray run --live -m my_module This will show the following TUI interface in your terminal: [live_runni] Sorting results The results are displayed in descending order of total memory allocated by a function and the subfunctions called by it. You can change the ordering with the following keyboard shortcuts: * t (default): Sort by total memory * o: Sort by own memory * a: Sort by allocation count The sorted column is highlighted with < > characters around the title. Viewing different threads By default, the live command will present the main thread of the program. You can look at different threads of the program by pressing the left and right arrow keys. [live_diffe] License Memray is Apache-2.0 licensed, as found in the LICENSE file. Code of Conduct * Code of Conduct This project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior which you have experienced in the project, please contact us at opensource@bloomberg.net. Security Policy * Security Policy If you believe you have identified a security vulnerability in this project, please send email to the project team at opensource@bloomberg.net, detailing the suspected issue and any methods you've found to reproduce it. Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports private until we've had an opportunity to review and address them. Contributing We welcome your contributions to help us improve and extend this project! Below you will find some basic steps required to be able to contribute to the project. If you have any questions about this process or any other aspect of contributing to a Bloomberg open source project, feel free to send an email to opensource@bloomberg.net and we'll get your questions answered as quickly as we can. Contribution Licensing Since this project is distributed under the terms of an open source license, contributions that you make are licensed under the same terms. In order for us to be able to accept your contributions, we will need explicit confirmation from you that you are able and willing to provide them under these terms, and the mechanism we use to do this is called a Developer's Certificate of Origin (DCO). This is very similar to the process used by the Linux(R) kernel, Samba, and many other major open source projects. To participate under these terms, all that you must do is include a line like the following as the last line of the commit message for each commit in your contribution: Signed-Off-By: Random J. Developer The simplest way to accomplish this is to add -s or --signoff to your git commit command. You must use your real name (sorry, no pseudonyms, and no anonymous contributions). Steps * Create an Issue, selecting 'Feature Request', and explain the proposed change. * Follow the guidelines in the issue template presented to you. * Submit the Issue. * Submit a Pull Request and link it to the Issue by including "#" in the Pull Request summary. About Memray is a memory profiler for Python bloomberg.github.io/memray/ Topics python profiler memory memory-profiler memory-leak memory-leak-detection Resources Readme License Apache-2.0 License Code of conduct Code of conduct Stars 1k stars Watchers 13 watching Forks 20 forks Releases 1 v1.0.2 Latest Apr 12, 2022 Used by 1 * @bloomberg @bloomberg / pytest-memray Contributors 5 * @pradyunsg * @godlygeek * @pablogsal * @subbrammanian * @lkollar Languages * Python 63.4% * C++ 25.5% * Cython 4.2% * JavaScript 3.5% * HTML 1.9% * Makefile 1.2% * Other 0.3% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.