https://github.com/samuelcolvin/pydantic/issues/2678 Skip to content Sign up Sign up * Why GitHub? Features - + Mobile - + Actions - + Codespaces - + Packages - + Security - + Code review - + Project management - + Integrations - + GitHub Sponsors - + Customer stories- * Team * Enterprise * Explore + Explore GitHub - Learn and contribute + Topics - + Collections - + Trending - + Learning Lab - + Open source guides - Connect with others + The ReadME Project - + Events - + Community forum - + GitHub Education - + GitHub Stars program - * Marketplace * Pricing Plans - + Compare plans - + Contact Sales - + Education - [ ] [search-key] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up Sign up {{ message }} samuelcolvin / pydantic * Sponsor Sponsor samuelcolvin/pydantic * Notifications * Star 6k * Fork 541 * Code * Issues 245 * Pull requests 48 * Discussions * Actions * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Security * Insights New issue Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community. Pick a username [ ] Email Address [ ] Password [ ] [ ] Sign up for GitHub By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails. Already on GitHub? Sign in to your account Jump to bottom IMPORTANT: PEP 563, PEP 649 and the future of pydantic #2678 Open samuelcolvin opened this issue Apr 15, 2021 * 7 comments Open IMPORTANT: PEP 563, PEP 649 and the future of pydantic #2678 samuelcolvin opened this issue Apr 15, 2021 * 7 comments Labels Meta Comments @samuelcolvin Copy link Owner @samuelcolvin samuelcolvin commented Apr 15, 2021 PEP 563 "Postponed Evaluation of Annotations" was introduced in python 3.7 behind the from __future__ import annotations switch, it basically meant all annotations are strings, not python objects. Pydantic has tried to supported Postponed Annotations since v0.18.0, see #348. The problem however is that trying to evaluate those strings to get the real annotation objects is really hard, perhaps impossible to always do correctly, see #248 #234 #397 #415 #486 #531 #545 #635 #655 #704 #1298 #1332 #1345 #1370 #1668 #1736 #1873 #1895 #2112 #2314 # 2323 #2411 The reasons are complicated but basically typing.get_type_hints() doesn't work all the time and neither do the numerous hacks we've introduced to try and get fix it. Even if typing.get_type_hints() was faultless, it would still be massively slower than the current semantics or PEP 649 (see below). In short - pydantic doesn't work very well with postponed annotations, perhaps it never will. The Problem The problem is that postponed annotations are set to be come default in 3.10, features of which will be frozen in about three week. Even worse, there's no way to switch back to the current behaviour. The Solution The solution to this is PEP 649 developed by Larry Hastings. This basically means annotations evaluation is lazy - the work of building __annotations__ isn't done until you access __annotations__. As far as I can tell this is the best of both worlds. The Second Problem The sad reality however is that it seems very possible that PEP 649 will get rejected and when python 3.10 is released it will break much of pydantic, and thereby FastAPI and all the other libraries that rely on pydantic (as well as other libraries that use annotations at runtime like enforce and typer presumably). See this very long discussion of the issue on python-dev about whether PEP 649 is a good idea. --------------------------------------------------------------------- The Point So why am I saying all this apart from whinging about something I don't agree with? 1. This is fair warning That pydantic might break in a big way if python's core developers continue value principle over pragmatism. 2. You can help Thousands of developers and organisations big and small rely on pydantic. Type hints might have been conceived to help readers and static type checkers, but they're now used to great effect at runtime by lots and lots of people - they make python better. In the end I don't think python's core developers and steering council want to make your experience of python worse. They just haven't realised how important this is. (Even Larry hadn't heard of pydantic or FastAPI until yesterday when he emailed me, I guess they don't read the python developer survey ) If you value pydantic or FastAPI or other libraries that use annotations at runtime, please (constructively and respectfully) let the python steering council know that you would like PEP 649 to be accepted. I understand the decision on PEP 649 will be made over the next few days, so if you're going to do anything do it today. The text was updated successfully, but these errors were encountered: 134 7 @samuelcolvin samuelcolvin added the Meta label Apr 15, 2021 @samuelcolvin samuelcolvin pinned this issue Apr 15, 2021 @mathematicalmichael Copy link @mathematicalmichael mathematicalmichael commented Apr 15, 2021 * edited Could you please provide some suggestions for the best way to accomplish this ask?: If you value pydantic or FastAPI or other libraries that use annotations at runtime, please (constructively and respectfully) let the python steering council know that you would like PEP 649 to be accepted. Is there an email I can write to? Some forum to post on? What's the most impactful course of action? @samuelcolvin Copy link Owner Author @samuelcolvin samuelcolvin commented Apr 15, 2021 Not really, it's precisely because I don't know how to make my concerns clear that I created this issue. I've posted to python-dev supporting the PEP, but I suspect that more people chiming in to just support what I've said might do more harm than good. I've you've spoken to core devs in the past and they know you, perhaps prompting them might be the best thing to do. @notatallshaw Copy link @notatallshaw notatallshaw commented Apr 15, 2021 * edited As a long time reader of the Python Dev mailing list I would say that it is not necessary to contribute to the mailing list beyond what @samuelcolvin has already posted, unless there are specific technical questions that you are highly qualified to answer. The mail is already extremely forthright and I have no doubt it will generate a lot of discussion. One thing I would say though @samuelcolvin is it will probably help your case a lot to go in to more details about why typing.get_type_hints() doesn't work for pydantic, either because of specific issues, real world performance examples, or if there is some fundamental flaw in the approach itself. There's a lot of smart people in the Python core Dev team, so it makes sense to state the problem not the solution. It may be that typing.get_type_hints() can be fixed or some other method could be implemented that would breach the gap. My 2 cents anyway, hope it helps. 12 @samuelcolvin Copy link Owner Author @samuelcolvin samuelcolvin commented Apr 15, 2021 I take your point, but the problems with typing.get_type_hints() and PEP 563 are well described in PEP 649 and debated at length in the other thread, I didn't want to bring all that up again. 3 @notatallshaw Copy link @notatallshaw notatallshaw commented Apr 15, 2021 I'm going to be honest, I skimmed over many dozens of mails for this one. But to me it seemed the discussions were talking about problems in the abstract, without a real motivation or need for them to be fixed. Whereas specifics on actual problems implementing it, with an actual library, that's used by lots of users, is a more much compelling position. I would guess some core devs are going to ask questions about this though anyway, so responses to those questions will likely fill the gap. Best of luck! Don't use this tool on a day to day basis, but it's been great when I have 2 @gaborbernat Copy link @gaborbernat gaborbernat commented Apr 15, 2021 I think you should explain better why it's impossible to implement lazy evaluation of type hints. The long list of issues doesn't help get a better understanding of the problem in achieving this, and in many it's never explained why it's hard to do. @methane Copy link @methane methane commented Apr 15, 2021 I take your point, but the problems with typing.get_type_hints() and PEP 563 are well described in PEP 649 and debated at length in the other thread, I didn't want to bring all that up again. Even there is a long thread, I don't know what point is actually breaking pydanitc. Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Assignees No one assigned Labels Meta Projects None yet Milestone No milestone Linked pull requests Successfully merging a pull request may close this issue. None yet 5 participants @methane @gaborbernat @samuelcolvin @notatallshaw @mathematicalmichael * (c) 2021 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.