[HN Gopher] Pyston-lite: our Python JIT as an extension module
___________________________________________________________________
Pyston-lite: our Python JIT as an extension module
Author : kmod
Score : 82 points
Date : 2022-06-08 16:48 UTC (6 hours ago)
(HTM) web link (blog.pyston.org)
(TXT) w3m dump (blog.pyston.org)
| simonw wrote:
| I love how easy this is to try out!
|
| I use pipenv, so I ran this: pipenv shell
| --python=python3.8 pip install pyston_lite_autoload
|
| And it worked: I got a small but material speed improvement from
| a tiny benchmark I ran against my own project:
| https://simonwillison.net/2022/Jun/8/pyston-lite/
| drcongo wrote:
| My very unscientific test (just running the test suite on the
| Django project I happen to have up at the moment) proved to be
| very slightly slower with it installed: 11.2s - 11.3s without
| Pyston, 11.7s - 11.8s with Pyston.
| kmod wrote:
| As someone else mentioned, test suites are a bit of a tough
| case for JITs. That said, we really don't want to slow down
| any workloads, and I suspect that something might be going
| wrong if you are getting a measurable slowdown.
|
| Is this project public? I'd love to investigate
| danuker wrote:
| A test suite would be designed to call every routine in a
| project a limited number of times. This means the JIT
| compiler will be compiling each routine, for it to be run a
| few times then thrown away.
|
| Realistic loads might have the same routines run more
| intensely and might see more benefit from the JIT compiler.
| miohtama wrote:
| See also upcoming Python 3.11 performance improvements
|
| https://news.ycombinator.com/item?id=31642793
| metadat wrote:
| Pyston is neat, but I'm a bit unclear. Why should we invest time
| and effort into more proprietary shenanigans?
| nomel wrote:
| Because it's been decades, and the Python team has repeatedly
| made it clear that interpreter source simplicity is more
| important than performance. They've admitted it many times.
| kzrdude wrote:
| It might finally be changing, though! We'll have to see how
| the following releases go
| nomel wrote:
| Yes, it might be, but until them, some of us have to use
| options that exist.
| colatkinson wrote:
| Is it proprietary? According to their blog, Pyston was open
| sourced with version 2.2 [0], and the LICENSE file in the repo
| [1] appears to be the same as upstream CPython.
|
| Not a rhetorical question BTW. A pluggable JIT for Python could
| be a boon for some projects at my dayjob, but if it's
| proprietary that would put a bit of a damper on things.
|
| [0]: https://blog.pyston.org/2021/05/05/pyston-v2-2-faster-and-
| op...
|
| [1]: https://github.com/pyston/pyston/blob/pyston_main/LICENSE
| metadat wrote:
| Thanks, I didn't know Pyston had been de-proprietarized and
| open sourced.
| Nasreddin_Hodja wrote:
| Why not to just write entire project in cython?
| postfck wrote:
| postfck wrote:
| vsskanth wrote:
| Can someone ELI5 how this works as an extension module ? Is it
| possible to do a similar thing with pypy ?
| woadwarrior01 wrote:
| I haven't read their source code, but the API extensions
| defined in PEP 523[1] lets extension modules (written in C or C
| level languages) to replace the default interpreter loop[2]
| with a custom one. I suspect they're using this mechanism to
| replace the default interpreter loop with their custom one when
| the module is imported and the enable function is called.
|
| [1]: https://peps.python.org/pep-0523/ [2]:
| https://github.com/python/cpython/blob/main/Python/ceval.c
| simonw wrote:
| You inspired me to take a look at their code.
|
| The autoload module at https://github.com/pyston/pyston/blob/
| 96d5d33186b81f96ce3d9a... just calls:
| __import__("pyston_lite").enable()
|
| It took some digging, but it looks like that enable() method
| is defined in the C code here: https://github.com/pyston/pyst
| on/blob/96d5d33186b81f96ce3d9a...
|
| It calls jit_start() which I think is here and does more of
| the interesting work: https://github.com/pyston/pyston/blob/6
| 9b190003f14dfd2f6d276...
| goodpoint wrote:
| Any comparison with
| https://mypyc.readthedocs.io/en/latest/introduction.html ?
| make3 wrote:
| With the large performance increases brought by the more recent
| versions of Python, it's not clear to me that installing this is
| faster than just upgrading Python. It is more simple though I'll
| give you that. If it also faster with more recent versions?
| jorl17 wrote:
| I believe the article answers all your questions!
|
| (Also, IMO, it is much easier to install Pyston-lite than
| upgrade a whole Python version, which is an absolute nightmare
| on some of my specific scenarios with somewhat embedded
| hardware)
|
| They are currently stuck on 3.8, but believe they will be able
| to support multiple versions easily because pyston-lite is much
| easier to develop for than pyston (because it is an extension
| and not a whole CPython fork).
|
| They also present benchmarks at the very end comparing Pyston,
| Pyston-lite and the latest CPython 3.11.0b3, all compared to
| Ubuntu's default Python 3.8.10. It shows that while CPython
| offers ~8-15% improvements (macro-micro benchmarks), Pyston-
| Lite offers 8-27% improvements and "OG" Pyston offers 35-65%
| improvements.
|
| So compared to regular CPython you might get an extra 10% or
| so, which might or not be worth it depending on your case.
| Excited to see what happens if they port Pyston to Python 3.11,
| though!
___________________________________________________________________
(page generated 2022-06-08 23:01 UTC)