[HN Gopher] Python's experimental JIT to be removed
___________________________________________________________________
Python's experimental JIT to be removed
Author : f311a
Score : 527 points
Date : 2026-10-27 11:17 UTC (1 days ago)
(HTM) web link (lwn.net)
(TXT) w3m dump (lwn.net)
| bratao wrote:
| This is pretty bittersweet to see.
|
| Python is my favorite language, but I'm honestly tired of having
| to rewrite performance bottlenecks in Cython or Rust all the
| time. When the Faster CPython project started with Microsoft's
| backing, I was really excited. Mark Shannon had that plan
| promising huge speedups across a few cycles, and while that
| clearly turned out to be much harder than anticipated, the team
| still kept pushing forward to deliver a real JIT.
|
| Seeing all of that get bogged down in committee bureaucracy now
| is just frustrating.
|
| I have a lot of respect for the core devs and I know maintaining
| CPython, and it is a time donation job. It isn't easy. But I feel
| that over-caution and red tape ends up discouraging serious
| contributions. They are setting an unreasonably high bar for
| something that would obviously help the entire ecosystem.
|
| Just think about the sheer amount of electricity and compute
| power being wasted every single day running inefficient Python
| across millions of machines worldwide.
|
| Python really deserves a proper JIT. I genuinely hope they
| reconsider this.
| mananaysiempre wrote:
| Unladen Swallow died for lack of upstream interest.
|
| PyPy lacks upstream interest, looked like it could have a shot
| after many years of development but recently NumPy removed
| support because they just couldn't fathom how a non-dead
| project could be a couple of versions behind CPython.
|
| Now the new JIT is dying because of what seems to be a lack of
| upstream interest. (The things TFA describes as being required
| are not unreasonable, but not realistic for a necessarily
| speculative long-running project without wild amounts of
| committed funding.)
|
| I'm not optimistic.
| pansa2 wrote:
| > _Unladen Swallow died for lack of upstream interest_
|
| IIRC Unladen Swallow was, despite the extreme hype, nothing
| more than the internship project of a couple of students. It
| died because their internship was over.
|
| Otherwise you're right, though. Lots of CPython alternatives
| suffer from lack of upstream interest - the Python language
| designers don't seem very interested in anything other than
| CPython. In particular, they appear to only consider CPython
| when adding features to the language. Two examples:
|
| - Requiring `dict`s to be ordered because that's how they
| work in CPython, without considering how they work in other
| implementations
|
| - Adding pattern matching syntax that's suitable for
| CPython's PEG parser, even though it's not LL(1) so can't be
| parsed by, say, MicroPython
| genxy wrote:
| Core Python is antagonistic to all other Python
| implementations. There is only one Python.
| ZiiS wrote:
| I know not everyone loves them; but `Claude rewrite the
| performance bottlenecks in Rust` works so well it is hard to
| see the security surface area and harder reason about
| performance as something that should be paid in core.
| greenavocado wrote:
| Done https://github.com/can1357/pon
| ddorian43 wrote:
| Seems unmaintained from July.
| greenavocado wrote:
| I think its a meme but an interesting proof of concept
| nevertheless
| skeledrew wrote:
| > committee bureaucracy
|
| That's what's kept the language and ecosystem improving and
| from becoming a hot mess. If something isn't properly specced
| then it really shouldn't be in prod. They allowed the
| experiment in the first place so it'd be easier to gather
| sufficient data to guide said speccing, but the ball was
| dropped by the proposers and so the experiment was an ultimate
| failure (unlike free threading) and prod is now being
| appropriately cleaned up.
| pjmlp wrote:
| > Just think about the sheer amount of electricity and compute
| power being wasted every single day running inefficient Python
| across millions of machines worldwide.
|
| Every single time I remember the first AI wave was done with
| Lisp and Prolog, both with tooling that supports machine code
| generation since about 50 years.
| samus wrote:
| It's not red tape, it's about making sure that the effort
| actually delivers what it promises and can continue to so in
| the future. From the state of the draft PEP it seems it doesn't
| and it won't ever. The SC merely put the nail into the coffin.
| xx_ns wrote:
| It's unfortunate that a technically exciting new feature is being
| killed partly because of what is essentially bureaucracy (and the
| lack of the developer's personal bandwidth). On the other hand,
| it's pretty safe to say that the strict adherence to the PEP
| policy and all of the bureaucracy that comes with it is what
| makes Python a great language, for better or worse.
|
| Not to say that the technical objections were without merit, they
| likely very much weren't, and the debugger and instrumentation
| support and maintenance work sounds fairly painful on its own.
| Good4boothee wrote:
| Looking at amount of JIT related (for example type confusion)
| security bugs in V8 I am not surprised other developers are
| vary of its inclusion.
| jeremyjh wrote:
| I think it would be a mistake to conclude this has anything at
| all to do with bureaucracy as a matter of inanimate policy that
| the SC was helplessly in thrall to. Bureaucracy was the tool
| the SC used to demonstrate that this feature lacked many
| important properties that they cared about, as mentioned:
| "maintenance, compatibility, and ... clear success metrics".
| For whatever detailed reasons, the PEP was withdrawn after it
| couldn't convincingly demonstrate those things to the
| satisfaction of the critics on the SC.
| bunderbunder wrote:
| Im not sure I even like calling it bureaucracy. It's a
| process, yes. But we're not talking many layers of abstract
| organizational hell, here. Were talking a fairly direct
| process of documenting what you are doing and how you are
| going to do it well enough that others working and depending
| on the project can understand it, coordinate with it, and
| have a reasonable degree of confidence that it won't cause
| undue problems.
|
| If having to put all that in writing and get a bunch of
| people to agree feels too burdensome to you, maybe that's a
| sign that you would be happier working on something other
| than core changes to the world's second most popular
| programming language.
|
| (edit: That wasn't meant to be a jab at the core developers
| working on the JIT. Based on what's in the public discussion
| threads, they have been handling things very well throughout
| the process. It's unfortunate that the SC let the project get
| this far before realizing the project had grown well beyond
| its original PIP's mandate, and I'm sure that is deeply
| frustrating to the people leading the project. I know I would
| not be handling the setback nearly as gracefully as they
| are.)
| vlovich123 wrote:
| The problem is that clearly documenting all details as
| required in the PEP is at odds with complicated new
| features like a JIT where all the details can't be known
| and have to be decided on as they are realized. Process /
| bureaucracy is at direct odds with experimentation.
| jeremyjh wrote:
| What are you basing this statement on?
|
| The experiments had long been done. The topic at hand was
| including it as a core feature.
| bunderbunder wrote:
| But they weren't asking for all the implementation
| details. They were asking for a clearly stated scope,
| success criteria, support plan, etc.
|
| As they themselves pointed out, the problem isn't that
| there's anything wrong with an open-ended experiment.
| It's just that open-ended experimentation doesn't belong
| in _main_ , of all places. They have an ongoing duty to
| ensure that a critical piece of global infrastructure
| remains stable and trustworthy to consider.
| cogman10 wrote:
| > clearly stated scope
|
| The scope of a JIT is the entire python runtime. Not at
| the beginning, it can be more narrow, but certainly it
| can evolve to cover everything. JITs touch everything.
|
| > success criteria
|
| Impossible to measure. JITs don't universally make things
| faster and unless the SC was willing to accept "some
| programs will see around x% improvement" then it would be
| impossible to actually give them the criteria they want.
| There's a reason C++ compilers and their optimizations
| aren't defined by the ISO committee.
|
| > support plan
|
| Like most JITs, support for a JIT is an ongoing and ever
| evolving process. What's needed depends entirely on
| what's needed today. You usually need more than 1 or 2
| dev working on it, which may be the issue here.
|
| > It's just that open-ended experimentation doesn't
| belong in main, of all places.
|
| Very strong disagree. That's simply the nature of
| performance optimizations for JITs. They belong on main
| because new optimizations and heuristics will always be
| generated. The role of doing a compiler is a long term
| experiment in performance, seeing what your runtime is
| currently doing and how it's being used and tweaking it
| to work better.
|
| pypy isn't in a worse place because it's been running a
| longterm performance experiment.
|
| > They have an ongoing duty to ensure that a critical
| piece of global infrastructure remains stable and
| trustworthy to consider.
|
| None of the things mentioned actually do that, other than
| perhaps a support plan. There are plenty of critical
| infrastructure projects who all can manage to develop
| JITs. Python is the special snowflake that has decided to
| do JIT development incorrectly.
|
| This is a runtime optimization/implementation detail.
| It's foolish to send it through a committee approval
| process. That's why you won't be able to find a single
| language with a JIT or advanced optimizations that has do
| it by committee.
| ModernMech wrote:
| In that case I can understand why it's not a fit for
| Python. The risks you're describing are more what a brand
| new language would face: universal scope, impossible to
| measure success, no support plan... sounds like a
| nightmare in a mature product. If that's actually what it
| means to add a JIT to Python, the SC is doing its job by
| steering Python around this quixotic task.
|
| > This is a runtime optimization/implementation detail.
|
| I view JITs as a critical architectural component. Like
| you said, it impacts the whole system. Optimization and
| implementation details don't usually do that.
| vlovich123 wrote:
| The problem is that "Python" the language includes native
| poorly isolated FFI interfaces exposed by CPython. So the
| language isn't "Python" but CPython. That's why alternate
| implementations like PyPy haven't seen adoption despite
| having a very mature JIT while languages like JavaScript
| have JITs that put CPython's performance to shame.
| bunderbunder wrote:
| It's also part of why Python has been so successful in
| many domains. I was there for Python's takeover of the
| data space, and I got quite used to porting things from
| Java (static, JIT, decent at multithreading) to Python
| (dynamic, interpreted, GIL) and realizing substantial
| performance improvements in the process.
|
| It's kind of wild how nice Python's C extension system,
| which enables basically zero cost interop with native
| code, is for certain purposes. It supports a situation
| that's comparable to the games industry's tradition of
| putting a Lua or Lisp tip on top of a C++ iceberg.
| bunderbunder wrote:
| And yet, not introducing unnecessary chaos into the
| CPython project remains important.
|
| Perhaps these priorities were always irreconcilable.
|
| Perhaps that's why the existence of competing Python JIT
| projects that weren't trying to commit directly into
| CPython's main branch kept coming up in the conversation.
| saghm wrote:
| > Im not sure I even like calling it bureaucracy. It's a
| process, yes. But we're not talking many layers of abstract
| organizational hell, here. Were talking a fairly direct
| process of documenting what you are doing and how you are
| going to do it well enough that others working and
| depending on the project can understand it, coordinate with
| it, and have a reasonable degree of confidence that it
| won't cause undue problems.
|
| Bureaucracy doesn't definitionally mean Kafkaesque. It's
| popular to hate on it, but good or neutral bureaucracy does
| exist.
| bunderbunder wrote:
| It doesn't have to mean kafkaesque, but it does mean
| governance through layered hierarchies and fixed
| procedures.
|
| In this case a single committee directly engaged with
| both core developers and the community, through a fairly
| open process, explicitly bent several of their own
| official rules to suit the occasion, and indicated cases
| where further flexibility might be needed depending on
| how things develop.
|
| I would call that a pretty stellar example of _non_
| -bureaucratic governance.
| saghm wrote:
| Fair enough. I was reacting more to the "organizational
| hell" part and didn't focus as much on the "layers" part.
| bunderbunder wrote:
| I have a hard time seeing it as organizational hell,
| either? From the outside, it looks like the process went
| smoothly and reached a decision in about half the time
| they originally estimated.
|
| It's true that that decision didn't involve everybody
| getting their way. But that's not organizational hell,
| that's just organization. If you don't want to contend
| with other people's competing interests, the solution to
| that is solo development.
| dcss_gardener wrote:
| HN when AI is used to bomb a school: technology is neutral.
|
| HN when an organization achieves its intended goals: this
| can't be bureaucracy because it's good.
| pphysch wrote:
| Large codebases are bureaucracies of their own, that is, the
| complexity of extra layers starts to bog things down at some
| point.
|
| In this case, it seems like the human bureaucracy made wise and
| measured decisions to contain the software bureaucracy. They
| made reasonable requests to clarify the scope and impact of a
| major, major core feature. The proposers didn't feel like that
| was doable, so a complexity nuke was avoided.
| kannanvijayan wrote:
| Gonna dump a technical hot take here.
|
| Python is a mature production language with a lot of users. Even
| before it was huge - like 20 years ago when I was in college -
| getting a small backwards compatible API change into a python
| extension module (gnu readline in my case) required convincing
| GvR personally that the API change was necessary.
|
| Every JIT attempt I've seen for python just tries to cut too deep
| in the first pass, and scope themselves too large.
|
| I believe any successful attempt at optimizing python performance
| from JIT will FIRST start with a 100% semantically compatible
| hidden type model for optimizing object property access and
| method dispatch.
|
| The single biggest performance delta in most dynamic language
| jits come from that, not from optimizing guards off of bytecode
| operations or from compiling to machine code. It's from
| converting hashtable lookups into constant-time guard + offset
| lookups.
|
| The hidden type model is also a hard nut to crack. It needs to
| perserve dict semantics which have very dynamic behaviour via the
| various `__*__` methods, and handle exotics scope objects since
| scoped name lookups are the most common ones which have their own
| exotic behaviours (I remember python providing some ability for
| callee frames to inspect caller scopes in some limited ways).
| Pxtl wrote:
| It still blows my mind that the big "we're breaking backwards
| compatibility Python 3000" release didn't do anything to lay
| the groundwork for this, or any of Python's other structural
| problems.
| jeremyjh wrote:
| Its possible this could have been the concrete pillar that
| broke the camel's back and set Python 3 on the path of Perl
| 6.
| ModernMech wrote:
| At the time, Python had a clear purpose as an easy to learn
| "glue" language between other, perhaps more performant
| languages. There was no reason to make Python as fast because
| that would muddy its purpose by introducing compromises and
| complexity. For better or worse Python sat pretty well in its
| niche.
|
| Today there are far more people using it, and everyone has a
| different perspective on what it is. No longer is it a niche
| glue language, now it's a backbone of AI technology at major
| tech firms, so it's understandable they'd like it to be
| faster on its own.
| 0xpgm wrote:
| For a long time Guido wanted CPython to remain simple, as the
| reference implementation.
|
| Probably also during the 2 to 3 migration, but that goal
| appears to have now been abandoned.
|
| I wonder how the authors of other implementations like pypy
| will be able to keep up as the reference implementation
| becomes more and more complex.
| Pxtl wrote:
| > the reference implementation.
|
| The problem was he exposed too much of the implementation
| into the language itself, destroying its ability to be a
| simple reference implementation that others could re-
| implement.
| mohamedkoubaa wrote:
| This is slowly being addressed now, each version of
| Python sees the c API become more opaque.
| twoodfin wrote:
| Are there any PEPs that describe this work?
| iqbal1980 wrote:
| They reference this one in the article
| https://peps.python.org/pep-0836/
| twoodfin wrote:
| I meant the trend of making the C interop more opaque.
| mohamedkoubaa wrote:
| See the stable ABI for Python 3.15 and PEP 793, 803, 820
| for recent movements in this general direction and the
| following draft: https://github.com/py-
| ni/peps/blob/main/peps/pep-08xx.rst (the py-ni project is
| pushing for this)
| jsmith45 wrote:
| That's at best a partial fix for the API side. Python
| side CPython sill exposes a LOT of implementation
| details, and as a result occasionally projects will
| choose to rely on some of these details. Then other
| implementations need to decide whether to also pick the
| same implementations details and expose them, or use
| different details, and lack compatibility with any
| library that depended upon them.
|
| The python documentation has historically also not been
| especially clear at distinguishing between what the
| language guarantees, vs what is just implementation
| details of the cPython implementation. So even if
| libraries wanted to try to avoid depending on
| implementation specifics, it isn't always clear what
| those are.
|
| The net result is that for the end user python the
| language is effectively: whatever cpython does. This
| really, really discourages creating alternate
| implementations, as the only way to be an alternative
| implementation with good compatibility is to be a living
| fork of cpython, constantly merging in all the changes
| from it, while also maintaining your improvements.
| mohamedkoubaa wrote:
| This is also slowly being addressed
| seabrookmx wrote:
| That question is already answered. pypy only supports 3.11
| today and is largely unmaintained.
| t43562 wrote:
| https://pypy.org/posts/2026/10/pypy-v801-release.html
|
| Last release on Oct 14. Beta support for 3.12. It's not
| that bad really and 1/10 of the people who complain about
| it did something then the situation would be better.
| jodrellblank wrote:
| They weren't rebuilding the whole language.
|
| https://peps.python.org/pep-3000/ - PEP 3000 says it will be
| a derived from the Python 2 codebase and not a complete
| rewrite (referencing Joel Spolsky's views on complete
| rewrites and Netscape Navigator).
|
| https://peps.python.org/pep-3099/ - PEP 3099 is changes Guido
| van Rossum rejected from Python 3 (including := as a binding
| operator), parallel iteration, programmable syntax, parser
| going beyond LL(1), changing the for/else construct.
|
| https://peps.python.org/pep-3100/ - PEP 3100 is things he set
| as goals for it, things like all strings are Unicode and byte
| arrays are a separate type, optional type annotations, 'exec'
| and 'print' change from statements to functions,
| None/True/False become keywords, old-style classes are gone,
| changing inequality comparison from <> to != , require C99,
| drop BeOS support, drop old standard library modules, and
| more.
|
| actually, more changes than I remember.
| adrian17 wrote:
| > It's from converting hashtable lookups into constant-time
| guard + offset lookups.
|
| Python already does it in a lot of cases, see opcode
| specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.
| kannanvijayan wrote:
| I haven't kept up with python development lately, so thanks
| for pointing that out. I looked this up and it does seem like
| some sort of property access is being optimized. What's not
| clear is what the hidden runtime type model is being used for
| this, or if it's a lightweight tracking that captures some
| subset of property accesses.
|
| More digging led to this: https://peps.python.org/pep-0659/
| which seems to be what the effort is organized around?
|
| Python's object semantics are broadly similar to Javascript,
| bar the prototypical inheritance chain, but with the extra
| complexity of potentially needing to handle dynamic
| `__*attr__` overrides. Optimizing for this would require a
| tree of hidden map objects much like JS JIT engines use. I
| dug around the codebase a while back looking for something
| like that but didn't find it. `OrderedDict` seemed promising
| but after more investigation that turned out to just be
| adding property ordering to dictionaries.
|
| The thing is, you can build that entire hidden type structure
| out, AND optimize against it, without ever having to mess
| with things like bytecode replacement. Once you have the
| hidden type model, you can modify the property lookup
| algorithm in the VM itself (as in cpython c code) to use some
| annotation structure hanging off of the bytecode to look up
| inline caches of type-guarded fastpaths. It would be an
| optimized fastpath that integrated directly into the C code
| that interprets the GETATTR opcode.
| adrian17 wrote:
| > Optimizing for this would require a tree of hidden map
| objects much like JS JIT engines use.
|
| I don't really think that's true or even that relevant , at
| least in context of classes (especially with slots - the
| class, by definition, _is_ the "hidden map") and not
| generic dicts.
|
| > The thing is, you can build that entire hidden type
| structure out, AND optimize against it, without ever having
| to mess with things like bytecode replacement.
|
| Pretty sure the motivation was exactly the opposite - that
| a simple "if a LOAD_ATTR encounters the same class with
| slots N times in a row, swap it out for LOAD_ATTR_SLOT with
| inline cached slot index, and swap back to generic one if
| something changes" is way easier (both to implement and
| reason about) than any alternative that'd interact with the
| object model. (Also it means you skip having to branch
| between a slow and fast path at all)
|
| > you can modify the property lookup algorithm in the VM
| itself (as in cpython c code) to use some annotation
| structure hanging off of the bytecode to look up inline
| caches of type-guarded fastpaths. It would be an optimized
| fastpath that integrated directly into the C code that
| interprets the GETATTR opcode.
|
| Sorry, that sounds a bit vague and not too different from
| what CPython already is doing.
| kannanvijayan wrote:
| Ah, the `__slots__` feature makes it make a lot more
| sense.
|
| > Sorry, that sounds a bit vague and not too different
| from what CPython already is doing.
|
| With slots being available, I can see where I was misled.
| I went looking for a type structure that modeled the
| dynamic properties on an object which would be used as a
| typeguard, but the class itself functions as that if
| you're restricting the optimization to slotted
| properties.
|
| Appreciate the clarfication!
| adrian17 wrote:
| (Note that this optimization can also trigger for plain
| classes (that's LOAD_ATTR_INSTANCE_VALUE), it just needs
| a bit more guards and is only useful if you don't abuse
| instances being dynamic objects. I just used slots as the
| most obvious example, but there are lots of different op
| specializations since like 3.11)
| kzrdude wrote:
| And we note that these specializations are part of
| production Python and will not be removed when the JIT is
| removed.
| graemep wrote:
| If you look at the reasons for this it seems like a good
| decision: https://discuss.python.org/t/pep-836-jit-go-brrr-the-
| path-to...
|
| There are already many way of addressing Python performance
| (leaving aside whether Python is a good choice if performance
| matters) such as Cython, Numba, Pypy and C extensions.
| coldtea wrote:
| > _There are already many way of addressing Python performance
| (leaving aside whether Python is a good choice if performance
| matters) such as Cython, Numba, Pypy and C extensions._
|
| And all of them are lacking in different ways, we ALSO, want
| faster pure Python.
| graemep wrote:
| They all have drawbacks, but they also reduce the case for
| needing to speed up CPython.
|
| Pypy does provide faster pure Python. Its a compatible
| implementation with JIT.
| v3ss0n wrote:
| Onlything is a lot of cextensions are super slow there.
| that is only bottleneck left which could be solved with
| resources that those guys spent on experimental jit.
| thayne wrote:
| The newest version of python that PyPy supports is 3.12.
| CPython is on 3.15.
|
| And from what I understand there are limitations with using
| libraries that use the CPython API to interface with native
| code.
| cpburns2009 wrote:
| The problem with PyPy, which I'm a huge fan of BTW, is it's on
| life-support. The only reason the past couple major versions
| were released was due to a few small companies or individuals
| directly implementing the updates themselves. That's my
| understanding at least. It's disheartening because I've been
| primarily using PyPy for 5 years, advocated for it, and was
| looking into contributing more to it before finding out about
| its state.
| t43562 wrote:
| Perhaps if the money which was spent on the built-in JIT had
| been spent on PyPy we'd be somewhere.
|
| PyPy wasn't great for me because its startup time interfered
| with the main usecase I wanted it for in one instance and in
| another there bugs that made it not quite compatible enough.
| Money could solve the bug issue.
|
| On the other hand I did get pygame working with it at one
| point and that seemed like a nice scenario.
| pbronez wrote:
| Interesting, makes sense. JIT in CPython main is a big deal, glad
| they're proceeding with caution. Overall looks like they added it
| as an experimental feature for two years, then decided to back it
| out. Perfectly reasonable outcome.
| samus wrote:
| It looks like it very much should never have happened like
| this. The actually hard and awkward questions have not been
| asked beforehand, and the SC doesn't like their hands being
| forced, and rightly so.
| anthk wrote:
| The Rust based Python3 port can keep it and it can still be far
| more secure than CPython. Not everything it's JIT'ted on
| RustPython, it's ondemand per function, you have to enable it
| explicitly.
| adrian17 wrote:
| Unfortunate. I've been following the work on it occasionally.
|
| On one hand, it seemed like JIT work was really progressing in
| high gear until SC requested a new PEP. Also unfortunate, as it
| seemed like part of the hope for the JIT was that it'd counter
| the performance penalty from free threading.
|
| On the other, it did feel (to me) like with all the fancy (and
| definitely cool) optimizations it got (automatic refcount
| operation removal, new type-based specializations, inplace math
| operations), the macrobenchmark benefits weren't _that_ exciting
| for a JIT. It definitely has potential for more, maybe with even
| fancier optimizations like unboxing ints, but the complexity just
| keeps going up.
|
| > It said that the PEP was "overly optimistic" about user-visible
| differences with the JIT enabled.
|
| The JIT is technically generated from the same source as the
| interpreter loop itself, so originally the differences were
| extremely unlikely, but I imagine the risk just kept going up as
| new JIT-only paths and optimizations were introduced.
|
| And politically, I feel like SC's reasoning was just "this side
| project has landed in main branch almost by accident and has been
| stewing without oversight ever since, with no stabilization in
| sight"; and the latest PEP was probably still too vague, with a
| lot of "here's more things we can add to the stew" - so I can
| understand if SC wasn't satisfied.
| cogman10 wrote:
| I think this is a major issue with requiring a PEP for what's a
| performance optimization.
|
| JITs are big nasty things that by their nature have to do a lot
| of things. You are basically saying "Give us a PEP for a new
| python interpreter and compiler".
|
| What they do is fuzzy by their nature because optimizations are
| very rarely universally beneficial. It's a game of measuring,
| looking for regressions, and tackling the biggest problem next
| for whatever sample applications you have available. You always
| run into the risk of overfitting on top of all that (You may
| have made django faster, but you've made flask slower).
|
| This is the sort of thing that, IMO, should not go through the
| PEP process. It's an implementation detail for the runtime.
|
| Imagine, for example, if the JVM needed a bunch of JEPs for JIT
| improvements. They'd never get done. Or if any javascript
| engine needed ECMA acceptance before they could do a new JIT
| optimization.
| adrian17 wrote:
| > Imagine, for example, if the JVM needed a bunch of JEPs for
| JIT improvements.
|
| FWIW, I definitely can find a bunch of both PEPs and JEPs for
| internal optimizations, random examples:
|
| https://peps.python.org/pep-0509/
|
| https://openjdk.org/jeps/147
| cogman10 wrote:
| You can find those JEPs, but frankly they are there for 1
| of 2 reasons.
|
| 1. An outsider to the regular OpenJDK dev team proposing an
| optimization.
|
| 2. Advertising for a cool new performance tweak with
| potentially large impact.
|
| You'll also notice that the JEP descriptions are very
| different from PEP descriptions.
|
| Probably the largest JEP for the JVM, Valhalla
|
| https://openjdk.org/jeps/401
|
| Does not have the same level of requirements being asked of
| the Python JIT. Further, it's well understood that this JEP
| will "live on main" for quite some time in preview before
| it's accepted and merged.
|
| What the JEP does, instead, is simply describe the feature
| and potential risks.
|
| The vast majority of JVM optimizations do not get a JEP.
| adrian17 wrote:
| Fair, I don't argue with that, I'm not knowledgeable
| about Java's processes. Just one thing:
|
| > The vast majority of JVM optimizations do not get a
| JEP.
|
| In Python too. Still, the JIT really is just a really big
| thing. I don't know if there could be a better way to do
| it, as either 1. You develop it off-branch and... make an
| enormous PR when you're done, with risks of getting
| rejected anyway? Or 2, you allow more experiments on main
| like Rust, but then you end up with hundreds of minor
| half-maintained nightly-only features, possibly
| conflicting with each other, and eventually go through
| bureaucracy to remove them anyway.
| cogman10 wrote:
| So, I'd grant that it'd probably be a JEP for java (the
| initial JIT) but not that every change to the JIT would
| result in a new JEP. Java's JIT, hotspot, was integrated
| right before they started the JEP process. The JEP for
| the JIT would likely be pretty boring like "It's a JIT,
| it does this, in some applications we see this sort of
| speed up". Much like the AOT JEP
| https://openjdk.org/jeps/514
|
| The Java governance allows for a lot more open ended
| projects with follow ups. Not that it's reckless,
| Valhalla is proof of that, but rather that once something
| is good enough they are pretty open to just shipping it
| as a preview feature and refining until it is worthy of
| being stable.
|
| That's why, for example, we saw ZGC get merged, and then
| JDK after JDK seeing continuing development and
| improvement on ZGC (Or G1 for that matter). These GCs
| were fairly rough on their first release and now they are
| pretty much best in class.
|
| I see a Python JIT as the same thing. Delivering the
| infrastructure for a JIT would need a PIP, but I wouldn't
| put in things like "acceptance criteria" beyond "doesn't
| break older code". I would expect such a project to get
| continual updates without PEPs.
| adrian17 wrote:
| > I would expect such a project to get continual updates
| without PEPs.
|
| Just for completeness: the original JIT PR was merged
| quite enthusiastically and only after the fact it was
| pointed out that it should have went through a PEP
| process [1]. The following PEP 744 was of the
| "informational" type, and the just-withdrawn PEP 836 was
| the first "standards track" PEP, which means that (in my
| understanding) it's technically the first actual PEP that
| the SC would formally comment on. So in a way it's not
| "can we update the JIT" PEP, but the first real "does SC
| want this JIT as proposed" PEP.
|
| That said, I do agree with most of what you said,
| especially the acceptance criteria part.
|
| The issue I have and find it hard to resolve is that
| since the code was already on main, the feature was
| already in "sunk cost" mode. It's hard when the code is
| already on main and another 3 optimization PRs are always
| there behind the corner, but it needs to be possible for
| SC to say "we decided it's not actually worth all this
| complexity for <some small number>% speed improvement"
| (I'm not commenting whether in this particular case it'd
| have been a right call), and the PEP (if not withdrawn)
| was the way to exercise that control.
|
| [1] https://github.com/python/cpython/pull/113465#issueco
| mment-1...
| pjmlp wrote:
| The original JIT certainly did not get a JEP, because
| JEPs replaced the previous JSR process.
|
| And even then, Hotspot did not got a JSR, because all
| that work was done before Sun started collaborating with
| industry partners for Java evolution.
|
| They "simply" ported the JIT compiler from the Strongtalk
| acquisition into Java.
| stickfigure wrote:
| > Imagine, for example, if the JVM needed a bunch of JEPs for
| JIT improvements.
|
| They do:
|
| https://openjdk.org/jeps/197
|
| https://openjdk.org/jeps/515
|
| Not every optimization requires a JEP, but certainly
| something as big as "add or remove a JIT" would 100% require
| a JEP.
| crabbone wrote:
| Oh, it's not just a performance optimization. But you've said
| it yourself: "a new python interpreter and compiler". A lot
| of Python code out there lives with assumptions of CPython
| interpreter built into it. Pretty sure pytest would've been
| affected, should JIT replace the existing interpreter, for
| example.
|
| Anyways. I don't believe that Python needs a JIT compiler. I
| don't believe that CPython can realistically replace its
| interpreter with JIT compiler without causing major problems
| with the community packages.
|
| The way I see it: Python users today _desperately_ want Java.
| They want it called Python because there 's some vague memory
| still around of Python being cool and Java not being cool.
|
| In Russian there's a children tale about a hungry soldier
| returning from a war and a greedy old woman who offered him a
| bed for the night. The old woman then pretends to have no
| food for dinner, and a soldier promises her a magic trick of
| making a porridge out of a hatchet. And so he tells the woman
| to put a pot on a stove with a hatchet in it, and step by
| step he makes her add all the ingredients necessary to make a
| porridge to the pot. The story gave birth to the idiom
| "hatchet porridge", which is used to describe anything that
| was created in a similar way: by deception, when another goal
| was allegedly being pursued.
|
| So, to me, the current state of Python is this "hatchet
| porridge". None of its users want the actual Python (the
| hatchet is still sticking out of the pot), and they keep
| adding Java ingredients to the pot in hopes to cook up Java.
| And so was JIT. It just turned out to be too difficult to
| throw away the hatchet. An the porridge still has some
| chipped wood in it as well as a metallic taste.
| ActorNightly wrote:
| I mean, I have had great success with prototyping stuff in
| python, then letting an llm translate it to C. My 3090 with
| Gemma4:31b does this very well with a few guidelines.
|
| Dunno how much investment is needed into language optimization
| anymore.
| mring33621 wrote:
| Agree with the general workflow: prototype with an easy-to-
| read language then port to lower-level, faster language
| 14113 wrote:
| That works well for greenfield code, but if working on
| compiler optimisations has taught me anything, it's that
| there is a _lot_ of legacy code out there in the world that
| would benefit from re-compilation with optimisations - but
| would be very difficult to re-write for speed (regardless of
| whether the work is done by a human, AI, or other machine
| tool). JIT compilation fits very nicely in this niche, as it
| means that existing Python projects can get (theoretically)
| better performance without re-writing.
| lenerdenator wrote:
| People have been calling Python a "prototyping" language for
| decades. I'd be interested to see how many Python projects
| ever actually get rewritten in a more performant language. My
| guess is, up until the last five years that was almost never
| done. Even with LLMs to help, is that the best use of your
| tokens and humans' time? It doesn't add a feature. It could
| conceivably cut costs after enough time, but only if done
| correctly.
| nrawe wrote:
| Doing it currently, Python to Go. Early indications are a
| 50-98% increase in speed (e.g. one testcase went from 26hrs
| to 30mins) for network/CPU heavy workload. Conversion via
| Qwem, lots of testing needed but compute savings will be
| worth it.
| mynegation wrote:
| Did you mean "50-98% decrease in running time"? Just
| calculating from your 26hrs to 30 mins seems to imply so.
| In terms of speed that translates to 100% to 5100%
| increase in speed.
| gf000 wrote:
| Why not go with something like graalpy first? That's much
| less risk in terms of "we end up slightly changing
| behavior"
| zahlman wrote:
| > My guess is, up until the last five years that was almost
| never done.
|
| Because having just the hot loop in the performant language
| is often deemed good enough. And there's tons of Python
| code out there where the "hot loop" is already
| automatically in C (the various operations implemented in C
| by the runtime) or where you can get close enough e.g. with
| Numpy. Consider that things like Pygame can exist; for a
| lot of the games people want to make, the only thing that
| really needs to happen anywhere near "bare-metal" speed is
| blitting.
| ActorNightly wrote:
| > My guess is, up until the last five years that was almost
| never done.
|
| It was done plenty. Just with extensions to python like
| numpy and opencv. There is no reason to translate
| everything versus just the critical pieces.
| grep_it wrote:
| I've had good success with just using mypyc
| (https://mypyc.readthedocs.io/en/latest/introduction.html) on
| typed python code.
| nomel wrote:
| > on typed python code.
|
| A fairly massive undertaking for existing code, which jit
| was supposed to be useful for.
| zahlman wrote:
| I would imagine that the LLM has a much easier time
| adding correct type annotations to old Python code than
| it does translating it to C.
| EdwardDiego wrote:
| In my experiments with this, maybe, sorta, it'll do some
| AST stuff then it'll go "dang this shit is dynamic AF"
| then suggest a static type inference tool, then a runtime
| type profiler.
|
| Which honestly, is the right approach.
| EdwardDiego wrote:
| You can use static type analysers and runtime type
| profilers to add type annotations to a legacy codebase,
| but the interesting bit would be if you're built on top
| of a large untyped codebase you don't control like, say,
| Django, how far it can go.
| bjourne wrote:
| > On the other, it did feel (to me) like with all the fancy
| (and definitely cool) optimizations it got (automatic refcount
| operation removal, new type-based specializations, inplace math
| operations), the macrobenchmark benefits weren't _that_
| exciting for a JIT. It definitely has potential for more, maybe
| with even fancier optimizations like unboxing ints, but the
| complexity just keeps going up.
|
| Zero chance of shoehorning that into a standards-compliant
| Python runtime. Some things are too fundamental to change
| without affecting the user-visible language: JITting, lazy
| imports, threading, copying gc, ref counting, etc.
| lenkite wrote:
| I am confused. Why can't they have an experimental, optionally-
| enabled feature for a few years while they iron out all the
| issues ? What is wrong with that ? The JDK keeps doing this all
| the while. They write a new JEP if there are changes across
| previews.
| cedarbyte482 wrote:
| I also dont get it
| Krishansubudhi wrote:
| I hope it does not lead to performance issues later. Python is
| already famous for being slower than compiled languages.
| FranOntanaya wrote:
| Being slower than a compiled language is not saying much for an
| interpreted language. But it's slower than PHP which is a
| closer comparison, and PHP has had JIT for six years now.
| pansa2 wrote:
| Ruby has the reputation of being a very slow language, but
| Python has been slower still since Ruby jumped ahead with
| version 1.9, almost 20 years ago. Ruby also has had far more
| success with JIT compilers than Python has.
| chao- wrote:
| _> Ruby also has had far more success with JIT compilers
| than Python has._
|
| Ruby is on its third (I think?) JIT now? I'm a little
| surprised, given Ruby's reputation as a wacky "anything
| goes" language with a very complex grammar, and it makes me
| wonder why that is? Does it have to do with the design of
| the runtime itself more than the language semantics? Or is
| it more incidental, e.g. many popular ibs are C wrappers,
| already generates bytecode files, and so on.
| samus wrote:
| Getting a JIT right is hard work because time spent by
| the JIT is time not spent doing actual work, and finding
| the correct tradeoffs is very much an empirical problem.
| byroot wrote:
| The difference is that Ruby follow the BDFL model, and
| Ruby's BDFL was willing to allow people to experiment
| with in-tree development of JITs.
|
| If the Python steering committee isn't willing to do the
| same, they'll probably won't get a working JIT any time
| soon. Developing a JIT externally, or even from a branch,
| is just too hard.
| skeledrew wrote:
| And PHP is quite the hot mess. Always has been, really. See
| the security issues over the years, including the most recent
| for example. Meanwhile I've never seen any serious issues for
| Python that's triggered mass required fixing. There's
| something to be said for moving carefully and ensuring things
| aren't broken to the best of one's ability, which is the SC's
| purpose.
| pjmlp wrote:
| https://app.opencve.io/cve/?product=python&vendor=python
| skeledrew wrote:
| Are you helping me to prove my point? I see 156 results,
| with 1 in 2023 and the rest before that. Meanwhile there
| are 801 results with at least a dozen in 2026 for PHP[0],
| and we still have 2 more months to go.
|
| [0] https://app.opencve.io/cve/?vendor=php
| pjmlp wrote:
| Hardly, one CVE is enough.
| skeledrew wrote:
| Not at all. I specifically stated "serious issues for
| Python that's triggered mass required fixing", ie. it was
| so bad that there were public notices for users to update
| ASAP. The closest thing I can find is CVE-2022-34265, and
| that's in Django, not Python itself.
| pjmlp wrote:
| Python is hardly used at PHP scale for Web development.
| mahboi wrote:
| and the biggest one, JS
| yxhuvud wrote:
| It is also slower than any other interpreted dynamic language
| in wide use.
| tweakimp wrote:
| Sad to read this. I had hoped that with the advance of AI
| capabilities, features like this could be prototyped, tested,
| implemented and shipped even faster. And I was really looking
| forward to see performance gains for my favourite language.
|
| Where do we go from here? Microsoft did not continue to finance
| the Faster CPython project and the current speed gains are good,
| but not what we hoped.
| Retr0id wrote:
| At this point if you have a python codebase where perf is an
| issue, the "AI-pilled" answer is to rewrite it in a faster
| language.
| tweakimp wrote:
| But I want to continue to understand the code. I dont know
| Rust (maybe AI can teach me?) and if I let Claude rewrite my
| current 50k LOC project in Rust, how can I understand what is
| happening and how can I work on it and think it further?
| Yossarrian22 wrote:
| You could try having it rewrite parts of the hot path into
| Rust with PyO3.
| jerf wrote:
| Rust is not your only option. Nor is it automatically the
| best option, though it is certainly a solid option for a
| certain set of requirements. Go is a pretty decent match
| for a Python rewrite; superficially it seems like you're
| giving up a lot but under the hood they're philosophically
| more compatible than meets the eye in a lot of ways. But
| any static language that you have some other local reasons
| to prefer is going to blow its socks off in terms of
| performance. And "blow its socks off" is just for the naive
| translation of a conventional Python code base over to a
| new language. If your codebase is amenable to threading and
| can take advantage of multiple CPUs somehow, the result may
| blow Python's socks off twice over.
|
| The problem is that Python just isn't going to go fast.
| There are some languages that may have had more resources
| poured into them to make them go faster in absolute terms,
| like perhaps Java. But I don't think there's a single
| language that has had so _many_ ways of trying to make it
| go faster given so much effort over so many decades. It 's
| really time to call it... if you have Python code and it's
| a performance problem, if one of the many many many
| different solutions aren't adequate for some reason, Python
| is not going to get any faster than it is today and you're
| going to have to move off of it.
|
| If you need high performance code, Python is simply not the
| right answer and you're going to have to skill up. Or
| accept what you have, which you'll have to decide if that's
| a viable option. If it's fast enough I don't advocate
| switching casually. But if it isn't, there isn't a
| realistic prospect that you're going to wake up in a year
| or two to a 30x-faster Python. I wasn't terribly convinced
| by that claim back in the early 2000s, but it hadn't really
| been tested yet. It has been tested now.
| anthk wrote:
| RustPython it's pretty mature.
| skeledrew wrote:
| Not at all.
|
| > RustPython is in development, and while the interpreter
| certainly can be used in interesting use cases like
| running Python in WASM and embedding into a Rust project,
| do note that RustPython is not totally production-ready.
|
| - https://github.com/RustPython/RustPython
| LtWorf wrote:
| Pypy is really fast. Have you tried it?
| skeledrew wrote:
| There are already many solutions to the performance issue
| from which you can pick. You don't need to understand the C
| in numpy to use it for example. And I'd say it's highly
| unlikely that you have a use case that, if it's applicable,
| isn't already addressed. Like it's generally pointless to
| optimize language performance for code that's doing a lot
| of network operations, because network lag will always
| dwarf such optimizations.
| mahboi wrote:
| There's so much Python code that uses numpy etc but still
| ends up having some hot part in Python that's nontrivial
| to translate to something faster. Same reason people get
| frustrated with the GIL.
| vova_hn2 wrote:
| > But I want to continue to understand the code.
|
| You don't have to throw away the original Python codebase.
|
| You can have:
|
| 1. An implementation in slow language (i. e. Python) that
| you maintain manually or semi-manually (with minimal AI
| assistance) and understand well.
|
| 2. A fast implementation that is fully generated and
| maintained by AI, using slow implementation (1) as a
| reference.
|
| You can, for example, run two "adversarial" agents, one
| writes tests that compare output of (1) and (2) and tries
| to find a discrepancy, the other tries to fix (2) so that
| its output never differs from (1).
|
| This way, you can be reasonably confident that (2) does
| exactly the same thing as (1), despite being a blackbox
| that you don't understand. While (1) remains a reference
| implementation that you understand well and can work with.
| lemagedurage wrote:
| It's already typical to call into a native library with
| Python bindings for performance critical work in Python. Even
| Pydantic's validation runs in Rust code.
| Retr0id wrote:
| This works well if there are obvious hot spots that can be
| easily isolated, but sometimes the perf issues are more
| uniformly distributed throughout a codebase.
| Eji1700 wrote:
| A biased opinion as I dev in F#, but it strikes me that AI is
| going to play best with functionally pure languages (strongly
| typed, no side effects, whatever we call that these days)?
|
| F# is on the softest side of that, but letting the compiler
| tell the AI "nope, here's why" seems like a huge win in saving
| cycles and time.
|
| The downside at first was simply, "how many examples of F# are
| there to learn from", and early on it showed, but I think
| that's been long overcome.
|
| And F# is obviously NOT the best example since it's both "soft"
| on functional purity and niche. I imagine something like
| haskell is probably doing even better, although I've yet to
| check that at all (not that I can code haskell but I haven't
| looked at anecdotes or anything).
|
| I realize this doesn't really help anyone who's grown to love
| python, and there's surely no perfect language, but just like
| how the web catapulted javascript, I see a future where the
| answer isn't "use AI to make python better" so much as "what
| already works super well with AI, and does it ease the user
| adoption"
| vovavili wrote:
| The biggest winner from AI indeed appears to be Rust.
| pjmlp wrote:
| 3GL code generation is only a transition step until models
| get more mature for direct machine code generation.
|
| AI needs formal verification, not a borrow checker.
| pjmlp wrote:
| We pick languages that embrace machine code like Julia, Mojo,
| Common Lisp, Scheme, and stop calling libraries that are
| written in C, C++, Fortran, Rust, "Python Libraries", when they
| are thin bindings.
|
| Clearly the community doesn't want PyPy, GraalPy, or even a
| CPython JIT.
| samus wrote:
| Mature projects simply don't like large unexpected code drops
| if it is not made clear beforehand what the blast radius is and
| how it is going to be maintained. On top of that it seems that
| the performance gains weren't as expected and require further
| invasive changes.
|
| Implementation effort has never been what has held these
| efforts back. LLM helps with squirting out code, but the hard
| tasks are benchmarking, profiling, community engagement,
| knowledge management, and dealing with the hard questions such
| as whether the change is actually worth the additional
| complexity.
|
| These concerns won't go away even if we give up on human
| developers being able to grok the code and instead completely
| relying on LLMs for coding.
| mhh__ wrote:
| It seems like such a failing and waste of human life that we
| spend all this time waiting around for python when the JS engines
| are so cutting edge.
| mhh__ wrote:
| Why is this controversial? We spend hours waiting for python
| code when it could be much faster if it was treated like a
| serious programming language
| mahboi wrote:
| I think it's mainly because of how easy C interop is in Python.
| Same reason no other interpreter replaced CPython.
| neya wrote:
| > Same reason no other interpreter replaced CPython.
|
| Source?
| btown wrote:
| From https://discuss.python.org/t/pep-836-jit-go-brrr-the-path-
| to... :
|
| > We want the PEP to be clear that it's ultimately the JIT team's
| responsibility to find solutions for these packages and tools,
| and how to deal with situations where a tool can't be supported
| or made to work
|
| Is it just me or is this absurdly impossible, to the point of
| willfully killing the project? Committing a team of brilliant
| compiler engineers to doing free consulting ad infinitum for
| every bit of tooling in the world, however hacked-together?
|
| The alternative would be to have the bravery to say: this is the
| way forward to keep this language relevant into the future, and
| as a steering committee we collectively own that some tooling may
| be out of date. Instead, this is a signal that Python is going to
| rest on its laurels for a very long time.
| lemagedurage wrote:
| Given that the speedup is modest (~10%) and the Python 3
| migration caused enough pain for users, I get it though. If it
| was a 2x speedup it would be a different story IMO
| skeledrew wrote:
| Python use is widely diverse. Anyone wishing to contribute to
| that which is published to such a diverse community, all of
| whom expect Python to continue working for them without any
| regressions, has to be ready to cater to that diversity in its
| entirety.
| pansa2 wrote:
| Surprising, even though (IIRC) the performance of the JIT was
| always fairly underwhelming.
|
| What about the other major change to Python's implementation:
| free threading? Has it lived up to expectations? I assume it's
| not also in doubt, despite Guido recently saying "Personally I
| wish we'd stuck with the GIL"? [0]
|
| [0] https://discuss.python.org/t/pep-805-safe-parallel-
| python/10...
| yxhuvud wrote:
| Getting a JIT from underwhelming to actually good is always
| going to be a whole lot of iterative work to remove blocking
| parts of the existing implementation. In Ruby-land we have had
| a good JIT for a while now, but it took many release cycles and
| a ton of resources before it was able to outperform the non
| JITed version when using Rails. Don't expect quick fixes that
| make great strides and any investments in it needs to be
| persistent and apply over time.
|
| It boggles my mind that it apparently is the case that such a
| popular language as Python isn't able to get the resources to
| do this.
| ksec wrote:
| It is more than that. CRuby have had multiple JIT attempt
| before YJIT landed. From what I can tell CPython didn't have
| that. And luckily enough we now have ZJIT replacing YJIT.
| bilsbie wrote:
| What's the latest on getting rid of the GIL? Did that work out?
| LtWorf wrote:
| It did. It's slower though.
| genxy wrote:
| And you get it back with threads, which can now scale.
| LtWorf wrote:
| I didn't need compute threads. Just I/O threads. So now
| it's just slower.
| pansa2 wrote:
| > _we don 't feel we have the personal bandwidth to carry the
| current version forward_
|
| Are the JIT developers the same people who were working on Faster
| CPython at Microsoft? No surprise that they don't have the same
| amount of energy to dedicate to the JIT now that it's gone from a
| full-time job to a hobby project.
| raverbashing wrote:
| What's the status of 'Faster CPython'?
|
| I wouldn't be surprised if they all went nodejs instead :/
| Ralfp wrote:
| While JIT is being dropped, the original proposer and leader
| of the initiative was hired by ARM and continues work on
| speeding up Python.
|
| His current focus seems to be speeding up GC and free
| threading.
| tweakimp wrote:
| Microsoft is not financing it any more and laid off most of
| the team working on it.
| ttul wrote:
| That team was mostly hired away by NVIDIA.
| claudiojulio wrote:
| It doesn't matter. Just write it in Rust. AI will make Python
| less relevant in the future.
| avadodin wrote:
| Other than the trolling, you couldn't be more wrong.
|
| AI literally turned Python from this silly esolang that PhDs
| seem to love into the Lingua Franca of Computing.
|
| All LLM tools are built Python-first and all LLMs are trained
| Python-first.
|
| If there is anything you can believe in beyond gravity and
| taxes is that an LLM's first tokens on a new shell will be py
| thon 3.
| mike_hearn wrote:
| A sad day for CPython but it doesn't have to be a sad day for
| Python. There is a fully maintained implementation of Python 3.13
| that has an advanced JIT compiler. Once your Python warms up it
| can be 4x faster or higher (depending on what your program is
| doing). That implementation is called GraalPy and it builds on
| the JVM:
|
| https://graalpy.org/
|
| A bit like Chrome there's a fully open source version and a
| proprietary version with a few extra bells and whistles, but the
| latter is entirely free for both open and commercial usage (you
| can't redistribute it for a fee). So you may as well use it.
|
| It has other interesting features
|
| * Scalable pauseless GC
|
| * Single binary distribution of Python programs (like Go)
|
| * Fancy and optimized interop with other languages including
| Java, JavaScript, Kotlin, Ruby and any other Truffle language.
|
| * Supports native extensions. Tested and works with common libs
| like SciPy, NumPy, PyTorch, Pandas etc:
| https://graalpy.org/python-developers/compatibility/
|
| * They're working on a GIL-free mode
|
| In recent weeks there has been a release of a new web framework
| for Python that only runs on GraalPy, called Pyronaut
| (https://pyronaut.io/). It takes advantage of JVM interop to
| expose the Micronaut framework, which is similar to Spring for
| those who know it. Thanks to the JIT and well optimized base
| libraries it runs extremely fast, being about 3x faster than
| FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many
| more features.
|
| https://www.youtube.com/watch?v=zo6L7bxn4OY
|
| (that was presented at Devoxx but the Pyronaut devs will start
| touring Python specific conferences soon)
|
| So although CPython may not have made it past the JIT milestone,
| that doesn't mean Python as a language or community has to fail
| it. It does require openness to alternative implementations, but
| historically Pythonistas have been fairly good about that (e.g.
| Pypy was embraced back when it was more active).
|
| This stuff does require you to accept that it's made by Oracle,
| but as it's open source, that should be OK.
|
| Disclosure: I sit next to some of the people who work on this
| stuff.
| titzer wrote:
| > Once your Python warms up
|
| How long does that take?
| mike_hearn wrote:
| The Pyronaut performance page has some graphs with the warmup
| parameters specified precisely:
|
| https://pyronaut.io/performance/
|
| _> Latency lines come from the load ramp (+25% per 15-second
| step); throughput from 45-second validation steps (+2% each)
| after a 180-second warm-up. Each framework ran twice and we
| show its best run. No profilers were attached._
|
| So that's a "realistic" web serving benchmark if you're
| willing to grant the use of a GraalPy-optimized framework
| rather than Flask/FastAPI.
|
| Warmup times for other benchmarks are currently quite
| unstable because there's a big focus on interpreter
| performance at the moment, which is one of the main
| influences on warmup. There's a blog post here from August
| about how the compiler is being taught to recognize and
| optimize bytecode interpreter loops, to get results that
| normally require handcrafted assembly:
|
| https://medium.com/graalvm/escaping-the-giant-switch-
| dec20b5...
|
| So as you know already, warmup was historically a weakness of
| PE based stuff - although in this case the fair comparison is
| with CPython which never warms up at all - but a fast
| interpreter will make a big difference. In some benchmarks
| the GraalPy interpreter is already faster than the CPython
| interpreter, which is heavily optimized.
| pjmlp wrote:
| GraalPy and PyPy are great technology achievements, the problem
| is that the Python community at large rather prefers their C
| bindings or nowadays Rust, sprinkle a bit of Python FFI on top,
| and call them "Python" libraries.
| mike_hearn wrote:
| Yes, well, even with fancy JITs Python will never be a speed
| monster. Interop with libraries written in Java can be very
| high performance though as Pyronaut shows. And the heavy
| lifters that are hard to replace like NumPy and PyTorch do
| work with GraalPy.
|
| I think if the Python community wants an implementation that
| has a proven JIT and where the remaining work is easily
| scaled compatibility testing, then there's one available and
| ready. Whether they're calling into C or Rust or FORTRAN or
| Java shouldn't make a big difference in principle.
|
| Now, I've met lots of people who think that binding to Rust
| will always be the fastest thing to do but that
| underestimates Java significantly, especially once you factor
| in the fact that GraalPy can optimize out interop overheads.
|
| Consider LLM tokenization. It's one of the most heavily
| optimized pieces of code on the planet. Yet, toknroll written
| in Java can beat tiktoken which is written in Rust:
|
| https://github.com/qxoticai/qxotic/tree/main/toknroll
|
| Moreover the Rust code has some unsafe{} blocks as Rust often
| does, whereas toknroll has no unsafe code anywhere.
|
| So the scripting language communities need to think hard
| about where they go next, IMHO. DHH already got up and
| announced at a Rails conference that Ruby is over, you should
| use Rust for everything. I don't think that's quite true, but
| it's going to be a meme that with agents you should just pick
| the fastest language and go. If Python/Ruby/etc want to stay
| relevant then they need to care a lot more about their
| performance story. Just being more pleasant to read and write
| for humans doesn't cut it anymore. CPython is clearly not
| going to happen in this regard, so alternative
| implementations need to start being taken a lot more
| seriously than they were in the past.
| famouswaffles wrote:
| >the problem is that the Python community at large rather
| prefers their C bindings or nowadays Rust, sprinkle a bit of
| Python FFI on top, and call them "Python" libraries.
|
| They are Python libraries. Don't make the mistake of screwing
| up the cause and effect here. C, Rust etc - those are the
| replaceable parts of the chain. There is an official C++
| pytorch library and virtually nobody uses it for ML Research.
| People want to code in python and build those libraries so
| they can.
| pjmlp wrote:
| Nope, they are Python _bindings_ to native libraries.
|
| PyTorch in C++ naturally gets little use, because most
| people don't want to use C++, and although possible, e.g.
| CINT, very few care about C++ REPLs.
|
| Also these libraries only exist because Python cannot
| deliver on its own, like Lisp or Scheme, and needs such
| libraries in first place.
|
| Even BASIC has better options to do machine code and system
| programming than Python, starting with the original
| Dartmouth BASIC.
| famouswaffles wrote:
| I know what bindings are. A front-end to C++ for pytorch
| was only released from 1.0, literally 2 years later. The
| relationship is very clear here. There was no native C++
| library. It was an afterthought. Even in instances where
| there was, like Tensorflow, functionally it was still an
| afterthought.
|
| They are python libraries because people reached for
| those languages so they could continue work in python.
| They are there to serve python and nothing else. It's the
| same with Rust. It's by no means a C++ thing.
|
| EDIT: Not saying this is always the case. Qt>Pyside is an
| example where it's both technically and functionally
| bindings, but the vast majority of the time they are
| serving python. Some don't even have a native library to
| speak of.
| pjmlp wrote:
| For Python libraries they would be written in Python,
| eventually ctypes for syscalls.
|
| Otherwise they are bindings to native libraries.
|
| We can keep discussing in circles.
| famouswaffles wrote:
| >For Python libraries they would be written in Python,
| eventually ctypes for syscalls.
|
| So if you can't call/use the 'library' from the 'native'
| language, it's still a library? Whatever floats your
| boat. Lol.
| samatman wrote:
| I think it's actually reasonable to say that a Blub
| library should run on every implementation of Blub. So,
| GraalBlub and BlubBlub, not just CBlub.
|
| Otherwise it's just a sparkling FFI.
| faangguyindia wrote:
| I just use Go and Rust for everything; most of my mobile apps are
| in Flutter. I don't see any point in using anything else.
|
| for me it's over for python
| O5vYtytb wrote:
| What a shame. The alternative is CinderX? It's slower unless you
| stick to "Static Python" which is not most python code in the
| wild.
| phonon wrote:
| https://github.com/facebookincubator/cinderx might be a better
| approach.
| O5vYtytb wrote:
| CinderX isn't generally faster unless you use "Static Python"
| which is a major change in how programs are written compared to
| typical python code.
| phonon wrote:
| Out of the box it's comparable to the current JIT, but yes,
| major gains are only possible with at least type annotations.
| But that may be the only real way forward for a JIT
| implementation for CPython with the ability to gradually
| speed up slow parts of large programs.
|
| https://facebookincubator.github.io/cinderx/#performance
| hedgehog wrote:
| I did some experiments translating a typed subset of Python
| to C for small system use, certain features like lists of
| primitive types, generators and comprehensions, frozendict,
| simple classes, generic functions, etc, map with little or
| no cost but are so much cleaner than the equivalent code in
| plain C. You also get to write test harnesses using all the
| tools available to full Python. The main issue is
| unsurprisingly memory management, Obj-C / Swift style
| automatic reference counting works but has drawbacks, and
| it gets much more complicated if you want to mix the static
| and CPython code.
|
| The static + AOT approach has a lot of promise now that
| type annotations are widely adopted, predictable behavior
| and good performance on code that is mostly idiomatic
| Python seem like a good alternative in many situations to
| the weight of Rust.
| phonon wrote:
| Yes, exactly. If you can get a 2-3x speedup in CPython
| just with type annotations, and then a possible
| additional 2-3x by dropping down to "Static Python," that
| may be an overall near optimal solution for the
| ecosystem, all things considered.
| hedgehog wrote:
| I didn't touch JIT (complexity and footprint way beyond
| what I can use or manage myself in any case) but the AOT
| path is pretty much what you'd get if you wrote naive C
| by hand. The emitted code is readable, in some cases you
| probably do better because more efficient algorithms are
| easier to code or reuse, but with the current refcounting
| that will have some overhead. I do want to try some ideas
| like supporting an @soa decorator for a List of DataClass
| type that keeps the visible interface but changes the
| memory layout. If someone wanted to sponsor tokens I
| could probably pure robot through a stage of fleshing out
| enough to port a few representative libraries and tune
| against the results.
| kibwen wrote:
| Assuming that the biggest bang-for-buck of JIT compilation is
| optimizing hot loops (which is also the biggest bang-for-buck of
| optimization in every other context as well), and assuming that,
| per the historical best practices, all hot loops in Python are
| already written in C (or Rust, these days), this outcome is
| somewhat what I would expect.
| procaryote wrote:
| The performance of python indicates otherwise. There are some
| performance critical libraries written in C or C++ but a lot of
| python is written in python.
|
| Python in itself is poorly suited to ever become performant as
| it lacks the types required. I imagine that causes some issues
| for effective jit-ing too.
|
| They had a chance to fix this when adding type annotations, but
| instead of making it "optional/gradual typing" they opted to
| make it "documentation or linting support", which is obviously
| the wrong choice, but there were cultural reasons
| mahboi wrote:
| Does optional/gradual typing even help with JIT?
| procaryote wrote:
| If you know the type you need fewer checks in the jitted
| code, and you can optimise to native types.
|
| If it helps python specifically, given that the types
| themselves are pretty dynamic, who knows. A python integer
| doesn't map well to a native integer etc. Python really
| isn't designed for performance, and retrofitting it is
| hard.
| rich_sasha wrote:
| Here's an idea instead (maybe it already exists). Allow a
| decorator for writing inline Cython. It solves, I think, 90% of
| the problem. On first run, or eagerly, compile the decorated
| functions. Basically, Cython is the JIT, and user enables it
| explicitly.
|
| This way I can easily write inlined functions with proper C
| speed, in a Python esque language.
|
| Reasons why I rarely use Cython without this:
|
| - unmodified Python code running in Cython is often faster, but
| not that much. 50% or so is my rule of thumb.
|
| - but the development style is very different for Cython and
| Python, adding a lot of impedance. Eg reloading doesn't quite
| work, profiling is different, you need a build directory and a
| build step at deployment etc.
|
| - Cython proper is different enough and I'd argue unintuitive
| enough that if I have to have all these things in place, I might
| as well wrap my function in Rust.
|
| Such a decorator would give you a low friction way to make things
| faster when you need it, with scope for making it much faster via
| all the funny Cython, while keeping Python-esque semantics, in
| particular "development semantics".
|
| I think it would require some language modification since Cython
| was definitely a superset of Python; I think some of that syntax
| has been streamlined, but maybe still a strict superset.
| ttul wrote:
| I built this in 2001 (https://pyinline.sourceforge.net/).
| Obviously, that ancient thing doesn't work anymore, but the
| idea worked then and it ought to work today. That being said, I
| think there are significantly superior options in existence
| now!
| adsharma wrote:
| This is largely the idea behind py2many + fquery, but we want
| to get rid of C-API as well.
| https://github.com/py2many/py2many https://github.com/ads
| harma/fquery/blob/main/tests/django_example_model.py
| guipsp wrote:
| This is numba?
| ajross wrote:
| Yeah, this is numba. I find it really frustrating the extent
| to which people don't know about it.
|
| Folks, numba is great. To first approximation: write your
| python function to "look like C"[1] and work over the numpy
| arrays you probably already have and understand, put a "@jit"
| decorator on it, and you're done. It JITs and runs your code
| natively at native performance.
|
| With some extra effort, there are features for SIMD
| vectorization and thread parallelism, and even GPU backends
| (that, full disclosure, I've been pretty frustrated by -- but
| they do work).
|
| [1] Which is to say, don't touch Python interpreter data
| structures. No dicts or objects or lists or strings, etc...
| If you're bound by the interpreter[2] you're bound by the
| interpreter and nothing is going to help you.
|
| [2] A corrolary here is that the kind of C code numba is
| _NOT_ good for is "following pointers between structs",
| which is something that can't be expressed in either python
| or numpy.
| sgarland wrote:
| I posted it as well before scrolling down and seeing yours.
| Deleted mine, as your post goes into much more detail.
|
| I don't know how people who've worked with Python and been
| frustrated with its performance don't know about Numba.
| ainch wrote:
| if you want GPU-backed/compiled numpy code then JAX is also
| a good option.
| bheztio wrote:
| Last I tried numba, admittedly a long time ago, it was a
| pain due to extensive system requirements or alternatively
| requiring use of conda, which I dislike (for many reasons,
| but it's besides the point - I want to use standard python
| tools). Did they start to bundle their dependencies or..?
| Dylan16807 wrote:
| > No dicts or objects or lists or strings, etc.
|
| That's a really intense restriction. Sounds like the name
| fits; it's not general purpose, it's jit for number
| crunching and little else.
|
| I'm not sure exactly what you mean by "bound by the
| interpreter" but there's nothing about touching those data
| structures that requires the interpreter to be involved in
| most cases. The problem of slow APIs can be solved by the
| JIT having intimate knowledge of python's internals, can't
| it?
| ajross wrote:
| > That's a really intense restriction.
|
| Is it? An object field access in python is a dynamic hash
| lookup followed by a method call as a fallback (itself
| involving hashed lookups up the class hierarchy). That's
| not an "interpreter vs. native" kind of thing, that's
| just the semantics of the language that have to be
| preserved. And that's just not going to be "fast" in the
| way that the user of a JIT would want. Code dominated by
| Python datastructures is going to be "slow" in ways that
| JITs can't fix, ever.
|
| Which is why Python appears to have given up on the
| effort in this space. PyPy too[1] really never delivered
| on the desired performance gains (like 5x or so on
| typical code) without forcing the user into careful
| optimization.
|
| Numba, to its credit, doesn't even try. It falls back to
| python if it must for python stuff, but is quite clear
| that it's providing no benefit if that's what you're
| doing. And the stuff it _does_ do is obvious and clean
| and extremely useful.
|
| [1] PyPy is also looking pretty moribund recently.
| Dylan16807 wrote:
| A hash lookup can be very fast. There's no reason for it
| to be slow when it doesn't need to invoke the fallback.
| JITs can't remove the slowest case but they can make the
| normal case very fast. Saying it can't be fast ever is a
| huge cop-out.
| ajross wrote:
| > Saying it can't be fast ever is a huge cop-out.
|
| Maybe, but the world is still waiting for the
| counterexample. Python is simply not a runtime well-
| suited to optimization.
|
| In the meantime, those of us who actually enjoy real HPC
| problems are using numba and rolling our eyes about all
| the confuffery around JITs. Maybe one will arrive that
| proves us wrong, but numba is fast _today_.
| Dylan16807 wrote:
| LuaJIT has the same dynamism with dictionaries and it's
| very fast at accessing them in JIT code.
|
| I don't have enough experience with PyPy to say much
| about it.
| rich_sasha wrote:
| Not really. Numba is great at optimizing numpy code but not
| necessarily other general Python code.
| O5vYtytb wrote:
| This is fine for your own application code, but it doesn't do
| anything for the libraries you pull in for your project. I
| don't think that sprinkling Cython code in all library code is
| going to be a popular solution, especially for those not
| performance focused. The idea behind a JIT is to be a general
| performance solution not requiring changes in existing code.
| rich_sasha wrote:
| Good point.
|
| Maybe also then, importing module as Cython?
| import django with Cython
|
| compiles on first run?
| somethingsome wrote:
| I like Julia approach, compile everything on first launch, then
| reuse without speed penality, if you change a function,
| recompile that.
|
| On another note, I'm working on a (big) PEP for python, and
| looking for maintainers and sponsors (as per python's rules).
| Do someone know where I could find such amazing persons? (I
| already have the code, works in cython,graalVM and pypy, and
| it's kind of a big change but I believe it will help a lot the
| language)
| orf wrote:
| The language forum is the place to start- you should already
| have a thread there gathering feedback before writing a pep?
| somethingsome wrote:
| Thank-you, it's a longtime idea, but I was finally able to
| implement it, but now I'm trying to learn the procedure
| mirashii wrote:
| There's an #internals-and-peps channel in the Python discord
| where some folks hang out.
| ModernMech wrote:
| However because of this, Julia has has persistent problems
| with "time to first plot", which has gotten better, but they
| cannot fully ameliorate due to the way it fundamentally
| works. So there are tradeoffs -- the performance gets better,
| but the interpreted REPL experience is markedly worse.
| pjmlp wrote:
| It is overly exaggerated, especially when taken into
| account how long it takes to compile native libraries that
| people keep calling "Python" libraries, if there isn't a
| suitable wheel for the current environment.
| ModernMech wrote:
| I think it's true that it's exaggerated, but that doesn't
| mean the tradeoff doesn't exist.
| pjmlp wrote:
| A tradeoff that doesn't make sense to complain about,
| when the alternative is waiting for native libraries
| being compiled, in a mix of C, C++, Fortran and more
| recently Rust.
|
| A pain point that is big enough that at least two
| programming languages, Julia and Mojo, were created to
| move beyond it.
|
| Which would not even be an issue, if Lisp, Scheme and
| Prolog had not lost their aura among AI devs.
| ModernMech wrote:
| Julia and Mojo are attempts at solving the "two language
| problem", which not every Python user actually has --
| they're fine doing work exclusively in Python, not even
| using any of the libraries you're talking about.
|
| Mojo in particular recognized to compete with Python
| they'd have to be a superset, then they backed of that
| goal when they realized it would be too hard.
|
| So at the end of the day, Python has a value proposition
| and it cannot be entirely subsumed by other languages,
| because those languages make tradeoffs that move away
| from Python's value.
| pjmlp wrote:
| Python's value is being a portable scripting language, a
| better Perl, unfortunately some try to do full blown
| applications with an interpreted language.
| necovek wrote:
| Perhaps it's worse that some even succeed in building
| very complex systems that can handle a lot of load too
| (see eg. Instagram)? ;-)
|
| Python has a known set of trade-offs, mostly related to
| loop performance. But if you understand the trade-offs
| and available mitigations, it should not stop you from
| building good software. You know, just like any other
| ecosystem.
| FacelessJim wrote:
| By the way, the current nightly (especially if considering
| the optimized build) has a very substantial TTFX drop
| compared to the current release [1]
|
| [1] https://perf.julialang.org/?tab=overview
| pjmlp wrote:
| The Julia approach is the same as BASIC (as originally
| designed), Lisp, Prolog, as added info.
| klaff wrote:
| I don't recall being able to inspect compiled machine code
| in basic like I can in Julia.
|
| [EDIT - I assume you are commenting on Dartmouth BASIC.]
| pjmlp wrote:
| And? That doesn't change the fact it was compiled into
| machine code before execution, nor that with exception of
| 8 bit home computers, there were plenty of compiled
| BASICs in mainframes and later 16 bit home computers.
|
| Since we have to prove everything on Internet forums,
|
| https://archive.org/details/bitsavers_dartmouthB_3679804/
| mod...
|
| => Saving of Compiled Programs
| cpburns2009 wrote:
| That's basically how Python already compiles source to byte-
| code.
| jey wrote:
| Julia gets a lot more benefit out of it because it's
| compiling type-specialized version of the function all the
| way down to machine code. So for example, a function that's
| being called with a `Vector{Int64}` ends up dealing with an
| array of unboxed ints, not going through dynamic dispatch
| on a generic array for every operation.
| cpburns2009 wrote:
| I have no doubt Julia's far better with this. I just
| wanted to point out Python does have a similar precedence
| for ahead-of-time compilation, albeit with byte-code
| only. If the Python steering council ever gets its act
| together, maybe Python could have real AOT and JIT.
| zahlman wrote:
| > I like Julia approach, compile everything on first launch,
| then reuse without speed penality, if you change a function,
| recompile that.
|
| Python already compiles to bytecode (the .pyc files are a
| cache thereof). It's just that interpreting even the simple
| bytecode instructions can take dozens of clock cycles each. I
| imagine that a lot of this is _not_ in the interpreter loop
| (which has seen quite a bit of optimization, IIRC up to the
| level of "predictive" gotos within a big switch statement
| within a loop) but just unwrapping objects and moving values
| to and from registers, dealing with the fact that there is
| basically never any cache coherency for anything etc.
|
| Basically I'm not convinced that writing out the set of
| machine-code instructions needed to interpret the bytecodes
| would be a big win, because it takes too many instructions
| for simple tasks.
| ponco wrote:
| The main reason I don't see it in production code is you
| significantly increase your chance to write seg faults that
| kills the runtime / behaves unexpectedly to surrounding systems
| lcnmrn wrote:
| We just need Mojo to have 99% Python friendly syntax and we can
| forget about CPython.
| odo1242 wrote:
| Numba is basically this.
| __mharrison__ wrote:
| Numba does this. Generally as fast as Cython, but much easier
| to write.
| mpweiher wrote:
| I think the idea is correct, but hard to retrofit, you sort of
| have to have it there from the start.
|
| Objective-Smalltalk runs on the Objective-C runtime, so
| Objective-Smalltalk methods look like Objective-C methods
| accessed via objc_msgSend() to the entire rest of the system,
| including to other Objective-Smalltalk code.
|
| That means it doesn't really matter to the rest of the system
| if any particular method is interpreted, AOT-compiled or JIT-
| compiled. They can't tell, except for execution speed.
|
| And of course the execution model includes machine primitives,
| which the interpreter also deals with but doesn't make faster,
| but the native compilers do.
| stefanos82 wrote:
| If you want to use the same code unmodified *and* your code
| uses type hints, then you should use mypyc [1].
|
| There are times that it generates faster code than Cython! Of
| course with Cython you can tweak it and push its limits with
| various optimizations and other techniques, but for using your
| existing code as it is and gain massive performance
| improvements, mypyc it's a must!
|
| Code sample: import time def
| fib(n: int) -> int: if n <= 1:
| return n else: return fib(n-2) +
| fib(n-1) startTime = time.time()
| fib(35) duration = time.time() - startTime
| msduration = duration * 1000.0 uduration = msduration *
| 1000.0 print(f'{duration:.6f} seconds, {msduration:.2f}
| ms, {uduration:.2f} ms') $ python3 mypyfib.py
| 1.452477 seconds, 1452.48 ms, 1452477.22 ms $ #
| the same code compiled with `mypyc mypyfib.py` produces the
| following: $ python3 -c "import mypyfib"
| 0.063701 seconds, 63.70 ms, 63700.91 ms
|
| This is huge, if you think about it!
| 95.61% reduction in time 2180.15% increase in
| performance 22.80x faster
|
| [1] https://github.com/mypyc/mypyc
| alerighi wrote:
| I would not like a JIT to be embedded in python for one reason.
| The sole reason why Python didn't have all the security related
| bugs of other languages (just think about all the exploits in JS
| engines, as well as the recent PHP vulnerabilities) is sorely
| because it's an interpreted language, that is python is compiled
| in a bytecode that runs in a VM.
|
| Under these premises is very difficult that something not
| implemented correctly is really exploitable, and moreover it's
| simple to verify the security of the whole thing.
|
| Opening a JIT, that would mean mapping memory pages as RWX, would
| open the door to all soft of exploits that now are not really
| possible.
| CGamesPlay wrote:
| > just think about all the exploits in JS engines, as well as
| the recent PHP vulnerabilities
|
| Can you name some of these JS engine vulnerabilities? And the
| PHP one had nothing to do with JIT.
| futuraperdita wrote:
| Not OP, but JS engine JIT bugs are pretty common.
|
| https://phrack.org/issues/70/exploiting-logic-bugs-in-
| javasc...
|
| https://www.sentinelone.com/vulnerability-
| database/cve-2026-...
|
| https://projectzero.google/2020/09/jitsploitation-one.html
|
| https://www.zerodayinitiative.com/blog/2021/12/6/two-
| birds-w...
| xx_ns wrote:
| While I don't disagree with you in principle, the context for
| JS JIT vulns and potential Python JIT vulns is pretty different
| and worth taking into consideration.
|
| For JS, your machine running untrusted JS code is the
| _intended_ use case. In this scenario the JS is sandboxed in
| whatever JS engine (V8 etc) and does not have access to
| "dangerous" methods, and the main use case for a JIT vuln is to
| escape this sandbox to execute code on the host. This applies
| to regular internet browsing (and visiting potentially
| malicious sites) and also, e.g., using the built-in browser in
| a locked down device (such as game consoles) to gain arbitrary
| code execution.
|
| For Python, this security boundary doesn't exist. I can't
| really think of a scenario where you would be running
| potentially malicious Python code in CPython that would need a
| JIT vuln to escape whatever sandbox it is in instead of, say,
| calling the functions in Python directly (or using the C FFI to
| do whatever). I'm sure that in some niche use cases, this
| scenario might exist, but to me, it seems like the potential
| attack surface is fairly minimal.
|
| EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT
| writes code to a writable mapping and then mprotects it to W^X?
| I don't think it leaves RWX pages lying around at all.
| stickfigure wrote:
| Who is running sandboxed untrusted Python code?
|
| JS has a completely different threat model. And PHP has just
| been a basketcase from inception.
| mohamedkoubaa wrote:
| I would want a python implementation with a JIT and the ability
| to choose JIT or no-JIT when defining a python environment. And
| I want all my libraries to work either way.
|
| I'll get what I want eventually but I might be retired by then.
| incrudible wrote:
| This argument would be more convincing if, due to performance
| reasons, most Python code didn't use C extensions, where
| exploits are highly prevalent.
| adsharma wrote:
| It's strange that typescript has 5 different LLM written
| competing static translators to compiled languages and python is
| killing JIT because the PEP was not accepted.
| LtWorf wrote:
| cPython is killing it. You are very free to vibe code a new
| python interpreter that does JIT.
|
| In the remote eventuality that it should actually work and be
| maintained up to date and secure, I'm sure it will acquire
| users.
| adsharma wrote:
| More interested in creating a variant of python that's
| practical.
|
| Wrote a parser with a rust compatible pattern matching syntax
| here:
|
| https://github.com/py2many/spy-ast
|
| Should help both JIT and static translation.
| pjmlp wrote:
| This is why we cannot have nice things.
|
| Why had Python to succeed to Lisp in the AI wave? (rhetoric
| question)
|
| Since PyPy and GraalPy will never get the love they deserve, I
| guess there is hope that Julia, Mojo, and the whole Rewrite in
| Go/Rust keep pushing along then.
| nromiun wrote:
| I don't know why they don't just adopt Pypy as a faster Python
| implementation. Just let CPython develop slowly as the reference
| and give support to Pypy. It is already much faster than CPython.
| yje07039 wrote:
| Hope the copy-and-patch work gets a proper write-up before it's
| removed... The technique is interesting well beyond CPython.
| moralestapia wrote:
| Lol, it seems then they want to have a crippled ecosystem on
| purpose.
|
| I actually expected something good to come out of this, my
| mistake for trusting Python "engineers". Never again.
| Qem wrote:
| Sad news, I put a lot of hope in this project. At least we keep
| the base interpreter improvements they brought along, and we
| still have PyPy. CPython JIT is dead. Long live PyPy!
| PathOfEclipse wrote:
| How many Python JIT experiments have failed? I find it
| interesting because Ruby is probably harder to optimize than
| Python but Ruby has had a JIT for a long time. Maybe base python
| is more optimized than base Ruby so the JIT is incrementally not
| changing much? Also, I'd expect javascript to be harder to
| optimize but it has JITs. That could just be a funding issue,
| though. There's probably much more money and incentive to
| optimize Javascript.
| gopalv wrote:
| Python is harder to JIT because of the way __dict__ works.
|
| If all python had was __slots__ then we would not be struggling
| to JIT python.
|
| Tight numerical loops is where the JIT adds performance.
|
| Most performance sensitive features are already available in
| python between numpy, numba and Cython without really leaving
| the language core.
|
| You'll mostly be speeding up the slow parts of python, not
| supercharging the fast parts.
| cpburns2009 wrote:
| I've never bought this argument. Underneath all of the magic
| inheritance in Python is just the MRO (method resolution
| order). That's not characteristically any different from
| JavaScript's prototype chain. There's no reason a Python JIT
| couldn't optimize everything JavaScript does, and if you do
| specifically access `self.__dict__`, that can be a cold,
| unoptimizable path.
| gobdovan wrote:
| I'd say that's about right. The way people usually code in
| Python, mro is quite stable and most lookups would be cache
| hits. You can still get more weird invalidation in Python
| cuz they let you change stuff like `__bases__`, or you get
| some weird `__getattribute__` change in some place that
| would force you to invalidate more than you'd initially
| think should be necessary.
| hinkley wrote:
| One of the tricks here is to treat the happy path as a
| quasi-invariant and leave a guard that branches to slow
| code if the invariant didn't hold.
|
| I saw this presented as new tech at either OOPSLA or OSCon
| almost exactly 20 years ago.
|
| CPU branch prediction is still pretty good even with the
| loss of function in recent years.
| Sesse__ wrote:
| That works fine as long as nobody actually uses the
| unhappy path. If major libraries and frameworks do it all
| the time, you lose. So there is a certain aspect of
| culture outside what the language itself is capable of
| doing.
| pjmlp wrote:
| See Self.
| hermitdev wrote:
| > Python is harder to JIT because of the way __dict__ works.
|
| Heh, how about how `__class__` works? Not that I recommend
| actually doing this, but you can literally change the type of
| an already instantiated object by reassigning `__class__`:
| >>> class Dog: ... def hi(self): ...
| print("woof") ... >>> class Cat: ...
| def hi(self): ... print("meow")
| ... >>> dog = Dog() >>> dog.hi() woof
| >>> dog.__class__ = Cat >>> dog.hi() meow
| >>> isinstance(dog, Dog) False >>>
| isinstance(dog, Cat) True >>>
| pjmlp wrote:
| Smalltalk, Self and Lisp Machines enter the bar, with live
| image changes.
| samatman wrote:
| Lua has the same property with metatables. LuaJIT, which is
| very fast, has a solution: if you change a metatable after
| assigning it, or reassign a metatable, Mike Pall turns his
| back on you. Your code dies slowly, in the outer darkness
| of the interpreter loop, never to be JIT'ted upon again.
| sakesun wrote:
| I've been doing things in Python for over 20 years. Only
| recently I learn that I can do this. >>>
| Dog.__bases__ = (Cat,) >>> isinstance(dog, Dog)
| True >>> isinstance(dog, Cat) True
| >>>
| gregdeon wrote:
| This is wild. Is it part of Python's specification, or an
| implementation detail of CPython?
| sakesun wrote:
| As far as I understand, CPython implementation is THE
| specification. Some billions dollar ERP system firmly
| rely on this behavior, btw.
| esprehn wrote:
| That's also possible in JS with __proto__. The prototypes
| of the classes in the chain can be mutated at runtime too.
| How to handle these cases is well understood for JITs
| though and python could do the same.
|
| See also the warning here:
|
| https://developer.mozilla.org/en-
| US/docs/Web/JavaScript/Refe...
|
| And this related article:
|
| https://mathiasbynens.be/notes/prototypes
| mirashii wrote:
| The thing that actually makes it harder in Python is
| threading, because it breaks all ability to reason
| locally about what might happen and optimize. In Python
| you have to assume that another thread may do these
| things at any point mid-execution.
| lefra wrote:
| I see no issue here.
|
| Modifying a variable that is read by another thread needs
| special care in all multithreaded languages I know. If
| you don't use a synchronization primitive to tell the
| other thread "hey, this thing here may have changed, read
| it again from memory"; the other thread is free to use
| its old value for as long as it whishes.
| rurban wrote:
| Same as for the common lisp or raku mop. You check for the
| default, and use the fast path if so. And if you can
| guarantee that nobody changed those keys or values, lift that
| check to the front.
| necovek wrote:
| The type of dynamic language gymnastics done with Python is
| huge: any sufficiently well known library basically employs
| some sort of meta-programming of the language:
| * Typing is a non-obligatory add-on to the syntax *
| SQLAlchemy and other ORMs override basic operators like "=" and
| employ metaclass programming heavily to create Python-like SQL
| query builders * Pytest rewrites all the assertions as it
| side-steps bytecode generation (so your .pyc files are
| different than what they would be without pytest) * Many
| libraries depend on monkey-patching live library calls,
| including ones to external dependencies * The way async
| was added in is another bolt-on where the same code behaves
| differently in different contexts * Decorators can
| introduce any side-effect you imagine, and nobody (including
| core libraries) follow the golden rule of "don't do, only
| decorate" * Many debugging and profiling mechanisms
| (including REPL) depend on these properties to function --
| including observability tooling like Sentry, Datadog etc.
| ...
|
| JavaScript never had that level of flexibility (for better and
| for worse), and by extension, it's easier to build an
| alternative implementation and keep it compatible with the rest
| of the world that claims to be JavaScript.
| xorcist wrote:
| > The type of dynamic language gymnastics done with Python is
| huge
|
| Have you _seen_ Ruby?
| KerrAvon wrote:
| Ruby is essentially Smalltalk (see also Objective-C).
|
| I don't know Python's innards well enough to understand why
| it's so especially difficult in this regard, but I gather
| around the edges that its object model may not be as well
| founded.
| jashmatthews wrote:
| JIT compiling Ruby or Python in a way that speeds up code
| using existing libraries and frameworks is harder. We
| built up decades of code that does stuff that it would be
| a lot easier for JIT performance if we didn't do, like
| using C extensions instead of defined FFI.
|
| https://arxiv.org/html/2609.01502v1
| cyberax wrote:
| Operator overloading is a non-issue, it's just another form
| of method calls.
|
| JavaScript also has monkey-patching, and it's also very
| common. E.g. SolidJS replaces objects with dependency-
| tracking proxies.
|
| There are really no fundamental language reasons why Python
| should be more complex. From what I heard from a JIT
| developer, the main issue is not the code itself but the
| module system and code loading.
|
| JavaScript was designed from the start to operate inside a
| sealed environment with only few external handles, with the
| code loaded upfront or added on-demand via "eval". Not so for
| Python, where native code modules can (and will) result in
| code loaded dynamically and it's hard to track what depends
| on what.
| pjmlp wrote:
| See Self and Smalltalk for such kind of highly dynamic
| environment with code being loaded, and completely changing
| the running image on the fly.
|
| Yet both languages are on the genesis of modern JIT
| compilers, going beyond what was already being done for
| Lisps, Prolog, BASIC.
| exyi wrote:
| > JavaScript also has monkey-patching, and it's also very
| common. E.g. SolidJS replaces objects with dependency-
| tracking proxies.
|
| And the proxies make it slow as Python. Plain objects
| cannot suddenly become proxies, so the runtime can treat
| them as a special case.
| pjmlp wrote:
| The common excuse that ignores Smalltalk, Self, Lisp Machines
| like Interlisp-D, and more recently Ruby.
|
| All just as dynamic as Python, and with quite usable JIT
| compilers.
|
| Or for that matter, PyPy and GraalPy do exist.
| necovek wrote:
| Obviously, I am not saying making a JIT for Python is
| impossible: both PyPy and GraalPy prove otherwise. We are
| talking about full ecosystem support, not just the base
| language and stdlib.
|
| Can you just replace CPython with PyPy or GraalPy in every
| project? And why not? :)
| pjmlp wrote:
| Because the Python community writes libraries in C, C++,
| Fortran and Rust, to make up for the CPython slowness,
| using CPython C API, calls them Python libraries, and
| they naturally only work in CPython.
|
| PyPy and GraalPy then never get to use those pseudo
| "Python" libraries, because their API for C based
| extensions differs enough that you cannot take a random
| shared library and load them into either.
|
| They came up with a common C API proposal, but has hardly
| taken that much adoption.
| necovek wrote:
| Indeed, mixing all the language features and anti-
| features with non-native Python libraries is a big reason
| too -- both because it mitigates many of the performance
| issues a JIT would solve, but mostly because it
| complicates or makes the implementation unfeasible
| altogether.
| cdavid wrote:
| The common excuse is indeed wrong, as pypy proved. IMO the
| real issue is the python C API.
|
| The python C API is the "killer app" of python, and what
| gave its edge against otherwise similar languages
| (ruby/js). That's makes it unique (the whole scientific
| stack). But a lot of early choices leaked into this API,
| and this (+ preference from guido for simple
| implementation) is at odds with doing JIT: lots of global
| state (instead of explicitly passing interpreter state),
| ref counting, no tagged pointers meaning boxing/unboxing
| everywhere, etc. Some of those are getting gradually fixed,
| but that's a slog, though maybe AI can accelerate a lot.
|
| I had high hopes for HPy but that unfortunately fizzled
| out.
| mistrial9 wrote:
| > Many libraries depend on monkey-patching live library
| calls, including ones to external dependencies
|
| someone please think of the monkey patchers!
| yxhuvud wrote:
| > Maybe base python is more optimized than base Ruby
|
| Not since before Ruby 1.9, no.
| benrutter wrote:
| > How many Python JIT experiments have failed?
|
| Wait, how many? I'd say just this one. The other that I know
| of, pypy, was massively successful for a single maintainer
| project. And definitely showed the possibility/potential of JIT
| python.
|
| I wouldn't call it a failure for not over taking mainline
| CPython.
| tonymet wrote:
| what features would need to be removed to help python hit a
| baseline level of performance? adding typing? Some of the
| "functional" stuff. Is the performance a legacy issue or
| essential to the language's design?
| mahboi wrote:
| From my limited understanding, the C interop is the biggest
| obstacle to both JIT and GIL. It's also the feature that makes
| Python fast enough for so many use cases. And the dynamic
| typing is the reason it even has to be JIT instead of just pre-
| compiled.
| manvillej wrote:
| This seems to be all about drift from the main implementation of
| Cpython.
|
| If JIT is different enough in maintenance, security best
| practices, and consumer use, we don't have two compilers, we have
| two different languages.
|
| And that's the crux of it.
| quotemstr wrote:
| This would never have happened with a BDFL
| niltecedu wrote:
| ffffffff.... damnit I love python but why is it so difficult like
| this. JIT would have been so perfect so the language,
| increasingly overtime with AI code coming through I am not sure
| not focusing on making the performance better is going to be
| better for longer term. Even though a lot of the other operations
| done in python are done faster in libraries I would still like
| faster startup times, or faster loops.
| mwkaufma wrote:
| The poison-pill requirements had their intended effect:
| https://discuss.python.org/t/an-announcement-from-the-steeri...
|
| It would have been more honest to just cut the feature instead of
| haranguing a developer until they withdrew it.
| 12_throw_away wrote:
| I mean, if your multi-year-old project is ever asked to just
| take a moment to write out a quick formal roadmap and
| justification ... it is doomed, 100% of the time. The actual
| content of the request doesn't really matter, it's dead.
|
| To be clear, such a request doesn't even mean that someone is
| consciously attempting to kill the project (although in the
| corporate world, ime it probably does). It just means that
| something went critically wrong. In this case, it appears like
| python's development process really _required_ a PEP for this
| sort of work, especially since it was happening on the main
| branch.
| mwkaufma wrote:
| The sus bit is that the PEP they requested greatly increased
| the scope of the feature.
|
| >> For example, rather than proposing one single concrete JIT
| implementation, it may make more sense for the PEP to
| describe a JIT infrastructure that can support multiple
| implementation strategies.
| v3ss0n wrote:
| They should had just port everything to pyPy.
| shevy-java wrote:
| Python is no doubt a successful language. But, boy - this level
| of bureaucracy is annoying to no ends. Do people feel eager to
| propose a PEP? There are so many more enjoyable things to do in
| one's short life ...
| snovymgodym wrote:
| It clearly isn't hurting Python's popularity, but man the
| performance of the reference implementation is embarrassing.
|
| V8 made JavaScript fast in 2008, soon that will be 20 years ago.
|
| Perhaps I live in a bubble, but so much of the production Python
| code I see is fully type-hinted and makes used of an aggressive
| linter & type checker to the point where it might as well be
| written in Go/C#/Java/TypeScript so that it can actually take
| advantage of the performance gains instead of being loaded into a
| runtime that turns everything into heap-allocated PyObject
| pointers.
| cpburns2009 wrote:
| This is exactly how I write my Python. I'm currently debating
| between going in deep and transitioning to either TypeScript or
| Rust. Go's object system and error handling is too weak. Java
| is too verbose, but I have a soft spot for it.
| pjmlp wrote:
| Modern Java is not that verbose, plus you can use Scala,
| Kotlin, Clojure instead.
| throwaway7783 wrote:
| New java is not very verbose.
| seabrookmx wrote:
| C# is worth a look if you want less verbose Java with
| async/await like you have in other languages, and better
| performance than Typescript.
| _old_dude_ wrote:
| If you want V8 like optimizations, you have to introduce a
| strict mode.
|
| Python interpreter is known to leak its implementation
| (__class_ modification after creation, monkey patching with
| __getattr__, sys._getframe, eval/exec, deterministic call to
| __del__, no proper python API for libraries, etc.)
|
| You have to plug all of those holes before having a chance to
| get a decent JIT. Otherwise it will be just another JIT
| attempt, like Psyco, Unladen Swallow, Pyston.
| snovymgodym wrote:
| Python is not uniquely dynamic to a degree that would prevent
| an efficient implementation.
|
| V8 predates strict mode, and handles all the exec/eval,
| monkey patching and object system abuse you can do in
| JavaScript just fine. Common Lisp is one of the most dynamic
| languages in the world and allows you to do ridiculous things
| in terms of meta-programming and essentially building domain-
| specific languages into your program, yet it too manages to
| have ahead-of-time compiled implementations that are very
| performant. Also PyPy exists.
|
| There are definitely some technical aspects that make it hard
| to make Python fast, but they are not insurmountable. The
| reason CPython is slow today is simply because performance
| has not historically been prioritized by its developers.
| joerick wrote:
| I only just saw it mentioned in this thread, but CinderX might
| suit you well.
| benrutter wrote:
| I work in Python every day and think about this a lot - people
| really don't seem worried about its performance. Sure people
| _want_ it to be faster, but I don 't see very many people
| changing their language choice for a faster language, despite
| Python being a lot slower than most alternatives.
|
| If I think about where I see python being used though, this
| kind of makes sense, the two _big_ use cases are:
|
| - Web APIs and backends, where for the most part either the
| network or the database is the bottleneck
|
| - Data science work, where for the most part, work exists in
| libraries like polars, pytorch etc that ship the main work out
| to other languages.
|
| So from that perspective, although we'd all like python do be
| faster, it's speed _isn 't_ the bottleneck in the use cases
| where its popular (that's probably also the reason why we _don
| 't_ see Python getting used in other speed critical places).
|
| That's a really different story to JavaScript, which _doesn 't_
| have a handy CFFI or otherwise you can just reach out to. There
| are _a lot_ of cases where JavaScript speed improving _would_
| speed up the reactivity of a web app, even now.
|
| I'm not arguing against a faster CPython - I'd love one! But I
| think there's a clear reason why it hasn't been prioritised
| historically in the way JavaScript's performance has.
| callamdelaney wrote:
| Python needs less features, not more.
| bbor wrote:
| To me, goals without plans behind them aren't really goals at all
|
| Ok but that's not how specs and plans work -- they are very
| intentionally two distinct artifacts.
|
| As usual, though, I might just be biased by default towards
| defending the Python aristocracy; they seem to pretty much always
| be on the right side of history, even in the little ways!
| incrudible wrote:
| Python is a poor target for compilation, even the best JIT is
| never gonna make fundamental design flaws (that some may consider
| beloved features) disappear. It's never going to give "good"
| performance.
|
| In my view, the right course of action is to specify an official
| strictly typed subset of the language with compilation and static
| analysis as the justification, then implement both JIT and AOT on
| top of that. Legacy code can be rewritten or thrown out in
| phases, C extensions can be ported.
|
| Of course, any such proposal would invite monumental amounts of
| bikeshedding, which is why it wouldn't get far, much like the JIT
| proposal.
| tkcranny wrote:
| That's effectively what Mojo is. Hardly catching the world on
| fire but pretty good for what it does.
| lioeters wrote:
| > strictly typed subset of the language with compilation and
| static analysis
|
| This is a great idea for popular dynamic languages with
| existing users and codebases, particularly the focus on
| "subset" to limit the scope and emphasizing that full backward
| compatibility is not a goal; and allowing for gradual,
| incremental adoption. That makes it a lot more practical to
| implement and adopt.
|
| I think we'll see more of this. Recent examples include
| strictly typed TypeScript subset designed for ahead-of-time
| compilation.
| SuaveSteve wrote:
| 1. Python is slow, therefor we use C extensions. 2. We cannot
| make Python faster because of C extensions. 3. Go to 1.
| impossconstrain wrote:
| This is what happens what the defacto implementation, CPython and
| its external API, is effectively the specification. That includes
| all the incidental side effects that were not explicitly
| designed, but inherited as a quirk of historical implementation.
| It's next to impossible to retrofit a JIT given these
| constraints. ECMAScript has a proper specification and several
| implementations.
| henry700 wrote:
| bureaucracy and politics kills another innovation, fork found in
| kitchen
| rienbdj wrote:
| Maybe Python isn't a language for when performance is a big
| concern (at least in the user code) and that's OK?
| rurban wrote:
| That's why I never want to work in CPython core. The 2 jit PEP's
| were totally fine, the implementation solid and stable, and even
| easy to understand for the SC manager types (unlike all other
| JITs which are much more convoluted), the future impact promising
| (already faster than Luajit in the Lua port). But then they got
| on their power trips. profiling frames? Jit templates are jit
| templates for micro ops, what's not to understand? The very same
| architecture and ops they use in core already.
|
| As we see over and over again in python core development. It's
| the worst.
| sc077y wrote:
| This makes me think that mojo could actually become a python 4 of
| sorts meaning a python with more emphasis en performance and
| support for native gpu programming
| pythonRon wrote:
| Have people been finding bugs? Having trouble with it? It does
| work, right? Surely, this is just bureaucratic nonsense.
___________________________________________________________________
(page generated 2026-10-28 12:02 UTC)