[HN Gopher] Adoption of Mypy for Python type checking: 45% alrea...
___________________________________________________________________
Adoption of Mypy for Python type checking: 45% already use it, 40%
don't plan to
Author : pansa2
Score : 95 points
Date : 2021-03-21 12:45 UTC (10 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| bravura wrote:
| I love mypy and use it in my job.
|
| I would love to use it for personal pytorch projects, but only if
| I could type-check that tensors are the correct shape, etc. Is
| that possible?
| ali_m wrote:
| Variadic generics should make this possible in future:
| https://www.python.org/dev/peps/pep-0646/
| IshKebab wrote:
| Good progress. I think you'd be mad to not use type hints, even
| for small scripts they're hugely advantageous. It's especially
| nice now that VSCode supports them well.
|
| The only problems I have with them are that they are completely
| ignored by Python itself - you have to run a separate type
| checker. That's a fairly stone-aged system. And also I with
| VSCode had something like Typescript's "no implicit any" so it
| was more obvious where you had to put types that couldn't be
| inferred.
| incrudible wrote:
| > The only problems I have with them are that they are
| completely ignored by Python itself - you have to run a
| separate type checker.
|
| What do you expect the Python interpreter to do, exactly? Type
| annotations are not supposed to affect execution.
|
| In my experience, you may want different type checkers with
| different configurations for different projects.
| IshKebab wrote:
| I expect it to report type errors by default with an option
| to ignore them. Same as Typescript.
| heavyset_go wrote:
| I just use the typing module from the standard library, and only
| use Mypy via an LSP client in my editor.
| pansa2 wrote:
| It's interesting how close together those percentages are. Python
| developers may well be split 50:50 or 60:40 on the issue of
| whether or not to use static typing.
|
| For the language that's famously supposed to have "one obvious
| way to do it", that's a significant difference of opinion on
| something so fundamental.
| toxik wrote:
| I'm what I would say part of the old guard. Type annotations
| and checking was weird to me, but okay, then walrus assignment
| := came and I felt that's maybe worth it (makes some loop
| idioms a lot cleaner!)... then the match thing, I'm really
| starting to feel like the shine has worn off. Python is a
| noncohesive mess design wise.
|
| Two Python developers today may well have such enormous
| differences of opinion on what constitutes proper Python, they
| could be talking about different languages
| pansa2 wrote:
| David Beazley agrees with your last paragraph in a recent
| tweet:
|
| "Python is at least three different programming languages at
| this point."
|
| https://mobile.twitter.com/dabeaz/status/1373296229711376387
| geofft wrote:
| In the old days we called that "multiparadigm" and said it
| was a good thing.
| cosmic_quanta wrote:
| For my part, I'm waiting for a convenient way to combine mypy and
| numpy. Numpy 1.20 started using type annotations in the most
| trivial way, but it's a start.
| sonthonax wrote:
| It's something I'm really keen to see. MyPyC can produce decent
| LLVM IR from basic python classes, to the point where accessing
| a builtin type attribute on an enum is just simple pointer
| without indirection. However that's only one side of the python
| performance story, if people got into the habit of using Numpy
| arrays with MyPyC, python might actually be fast.
| danjac wrote:
| TBH once you start using a fair number of dependencies (say, your
| average Flask or Django app) you end up with so many workarounds
| and flags like _ignore_missing_imports_ that the benefit of mypy
| is marginal compared with other tools like flake and unit tests.
| I tend to use it more with libraries and less with applications.
| junon wrote:
| Not to mention the performance. Perhaps it's improved since,
| but just a year ago it took an upwards of 10-20 seconds to type
| check a trivial repository.
| robbyt wrote:
| Compared to the 10-20 hours I could spend debugging some
| stupid problem with passing objects to a function in the
| wrong order...
| PeterisP wrote:
| For that specific problem, using named parameters (even
| when not strictly necessary) is more helpoful.
| incrudible wrote:
| I've had a similar problems, pyright scales much better in my
| experience.
| emptysea wrote:
| In my experience it's been worth the trouble of writing partial
| type stubs for your project's dependencies.
|
| And by partial I mean only typing the stuff you end up using,
| so if you just use one function from some huge, untyped
| package, just type that one part for now.
|
| The way I see it, you write the stub once, and then the API has
| less ambiguity and you get type checking.
| CraigJPerry wrote:
| This seems very positive to me - specifically that a sizeable
| population recognise they really don't need it.
|
| Used correctly - i.e. you're leaning on structural typing, so you
| invest your time in Protocols that go "quack" rather than hard-
| to-test, brittle, nominatively typed Ducks who quack - which
| incidentally is exactly the path the mypy getting started docs
| seems to focus on, why?! - that's just setting people up for
| failure, i digress.
|
| Mypy, pyright etc are fantastic tools in the toolbox, sure to
| save time but Cargo culting is a pain to deal with. See for
| example 2 decades of empty getters / setters in Java "in case we
| ever want to add some behaviour in future".
|
| Gradual typing of the structural variety is fantastic, you'll
| know if you're doing it right because you'll spend surprisingly
| little time thinking about types, you won't spend any time
| wrestling with mypy to convince it of your idea and when it does
| complain your reaction will be a legitimate "aha, yeah ok fair
| shout, you got me" not "oh ffs!" :-)
|
| One really cool thing about gradual typing - if you get it wrong
| and i have to take over your code, i can at least turn it off.
| One of the reasons i love tests so much - I don't have to accept
| yours if they're a bear to maintain, i can easily bring along my
| own instead.
| skohan wrote:
| I find it hard to imagine cases where gradual typing would have
| any advantages over a language with decent, modern type
| inference. In the latter case, you're still not typing type
| specifiers very often, but you get unambiguous function
| signatures, and can eliminate whole classes of runtime errors.
| mumblemumble wrote:
| Data science. Static typing is a complete PITA when you're
| doing exploratory data work. Type inference only fixes things
| on a superficial level.
|
| For data engineering, I haven't had a chance to try this due
| to lack of language options, but I strongly suspect that
| structural typing would be much more useful than nominal
| typing.
| exdsq wrote:
| I'm not sure I really get this argument - you need to know
| something about the type you're getting to explore it, so
| why does specifying that type slow things down more than
| any gains you would with run time errors from hacking
| around?
| PeterisP wrote:
| You're often reading and processing a data source that's
| explicitly not "well typed" (heterogenous 'columns',
| heterogenous deep nested json structures, null values in
| places where they shouldn't be, garbage rows, etc), where
| getting them to fit any reasonable type assertions would
| be a lot of work in correcting and/or filtering that
| data, which is something that you would write _after_
| your exploration is done.
|
| Also, it's not uncommon to not be sure about anything in
| the data structure you're getting; it's so often that the
| schema or documentation of that data is wrong or outdated
| or simply missing, so understanding the properties of the
| data types is a key part of that exploration. You start
| with eyeballing the data with your code; making some
| assumptions, and making code to process and analyze the
| data that doesn't meet your type assumptions, instead of
| throwing a runtime error as soon as your assumptions are
| wrong and conflict with the data.
| skohan wrote:
| But arguably some of that issue could be helped if you
| had good type systems on both ends. How much data is
| poorly formatted because somebody just hacked it out of
| some system with whatever scripting language?
| PeterisP wrote:
| Sure, it would be nice if everyone would have a pony, but
| in such scenarios you're often working with data where
| you have no control over the data sources and limited or
| no contact with the people maintaining them, if these
| sources are even maintained at all. In some cases (not
| all) it is possible to assign resources to improve the
| system from which you're getting data, but that generally
| is conditional on you being able to demonstrate that this
| use of data is valuable, so that would only happen
| _after_ an extensive data analysis is already done on the
| data as it is now.
|
| Exploratory data analysis (and much of data science as a
| whole) requires you to work with data it is, not data as
| you'd want or data as it should be; and a tool which
| can't conveniently work with bad, dirty data is simply
| not a suitable tool for the task.
| skohan wrote:
| Most modern languages have tools to work with "messy
| data". The difference is that strongly typed languages
| have the tools to provide more structure once you do get
| the data into a reasonable state - which is hopefully
| something you would want before you start doing serious
| analysis.
| mumblemumble wrote:
| There's a chicken and egg problem. To figure out the
| structure of the data, you often need to get it into a
| programming environment so that you can more easily
| interrogate it. Unless you're already working with data
| that are well structured and well documented, that is
| often much easier to do in a dynamic language, because
| dynamic languages make it much easier to shift your types
| around on the fly. It's often even convenient to be able
| to modify them at run time, which is something I often
| end up doing, and also something that is, by definition,
| incompatible with static typing.
|
| By the time you have the structure grokked enough to
| easily work with it in a statically typed language,
| you've already got a program that is well-typed and
| strongly typed. Porting it to a static language would
| sort of be missing the point, because you certainly
| aren't going to get any static type checking to help you
| make sure you've correctly ported the code. And data
| wrangling code is rarely complex enough that I'm
| particularly worried about safe refactoring.
|
| And I'm not particularly excited to jump to a static
| language before I start doing serious analysis, because
| there still aren't any static languages with top-shelf
| data analysis packages. Partially, I think, because
| static typing also becomes a hindrance on this end of
| things. It's really convenient, for example, to be able
| to tell your graphing package, "Give me a splot matrix of
| these five columns," and have it be able to do that. A
| hypothetical static language with two datatypes would
| need 32 different overloads of the function for drawing
| splot matrix, or it would need to operate on a
| specialized data structure that puts a lot of effort into
| Greenspunning a limited dynamic typing facility into the
| language, or alternatively (and this seems to be the more
| popular option), it would need to simply forego having
| useful things like splot matrices, and instead just stick
| to the basics.
| exdsq wrote:
| But isn't this caused by a lack of decent typing? If I
| had a well-typed API and a public schema, wouldn't that
| be the better option?
|
| Also you could just stick that in as an array or string
| (or var) to start with
| mountainriver wrote:
| I type all my data science/ml code and I absolutely love
| it. Now when I explore packages or data I have a more clear
| understanding of what I'm dealing with
| skohan wrote:
| I feel like moving toward types is like organizing a
| messy chest of drawers. At first taking the extra care
| feels like more work than just rifling through the
| drawers like you're used to, but once the structure is in
| place you can't imagine going back.
| skohan wrote:
| Yeah but even then you can usually deserialize a CSV or
| json into "plain" data structures like arrays and dicts to
| explore it, and then switch over to a more rhobist,
| automatically serializable one when you have a better idea
| what the data looks like
| mumblemumble wrote:
| You can, but, IME, the practical benefit of doing so
| generally isn't proportional to the cost. It's just
| something you do if you have strong feelings about type
| discipline and are willing to sink a bunch of time into
| soothing them.
|
| It may have to do with the types you're using. When I'm
| doing EDA, I'm usually zooming past the arrays and dicts
| as quickly as I can, and making a beeline for something
| like Pandas that lets me manipulate the data more easily.
| Pandas is fairly pretty resistant to static typing, and
| getting over to something that does play nicer with
| static typing tends to also necessitate a lot of wheel
| reinvention. Tends to also bring a performance cost, too,
| since I'm often losing a lot of nice fast multithreaded
| C++ implementation for a mess of pointer chasing and GIL.
| nicoburns wrote:
| Dynamic types are considerably more flexible than static
| types. It's not just about typings them out. There are lots
| of types that you just can't have in static languages (that
| exist so far), because the language doesn't have the
| vocabulary to describe them.
|
| Code in dynamic languages typically takes advantage of the
| flexibility to do meta-programming, which can significantly
| both the amount of code required to complete a task and its
| complexity.
| skohan wrote:
| Just anecdotally, I used to work with a lot of dynamic
| languages, and in static languages I don't miss the
| dynamism at all. But when I go back to dynamic languages, I
| miss the type checking _a lot_.
| nonbirithm wrote:
| From my highly biased experience, I miss the dynamism all
| the time, but only for the specific reason of code
| hotloading.
|
| I love Emacs for its extensibility and the capability to
| make changes to the system and see them _immediately_.
| Until programming languages like Mun catch up, I doubt it
| would be possible to replace Emacs Lisp with a statically
| typed language and retain the same amount of
| extensibility. Part of the requirements would be having
| the ability to replace the implementation of any function
| in the system at runtime, and that 's the kind of thing
| you have no choice but to design your language from the
| ground up to include.
|
| I guess what I mean to say is that it's not static
| languages that make me miss dynamic language features,
| it's that there's no statically typed language that
| supports the runtime hotloading that I can accomplish
| with a few Lua scripts (and has enough community traction
| to support something like LOVE 2D).
|
| Steve Yegge wrote an article related to this. He suggests
| that people who use static type systems are mostly trying
| to catch errors at compile time, errors which have no
| relevance after the typechecking is over with and the
| code is run (in the sense that machine code has no
| understanding of types). Static and dynamic languages
| will both ultimately output machine code that does the
| same thing, but one language incurs a massive penalty to
| flexibility of change. It's a rant, but it made me think
| that maybe you could have the type checking to catch
| errors at the time you're specifically looking for
| errors, and use the dynamism of the underlying
| implementation later on.
|
| http://steve-yegge.blogspot.com/2007/01/pinocchio-
| problem.ht...
| skohan wrote:
| Just in terms of my subjective experience, the main
| benefit of type checking is that it codifies intent into
| the implementation. In many cases it feels like you can
| obviate unit tests, because the compiler can now verify
| that your code can do what you intend it to do.
|
| But I do disagree with one of your assertions:
|
| > Static and dynamic languages will both ultimately
| output machine code that does the same thing, but one
| language incurs a massive penalty to flexibility of
| change.
|
| In my mind, this is a clear win for static languages. As
| you have said, it's all machine code in the end - but for
| static types the compiler can often make more precise
| decisions about the size of a stack frame, for example,
| where in the dynamic case it's not so easy. Dynamism
| doesn't come for free, and probably _most_ of the time we
| don 't need to do fancy type manipulation or dynamic
| dispatch at runtime, so it doesn't make sense to pay that
| fixed cost on all my code for something which is done
| only rarely.
|
| Also I have never really understood the lack of
| "flexibility" imposed by a type system. Sure if I add a
| field to a type it might break some code elsewhere in my
| program, but I don't see that so much as inflexibility as
| much as the compiler catching a bug I would have been
| scratching my head over at runtime.
| CraigJPerry wrote:
| >> In many cases it feels like you can obviate unit tests
|
| And that is the trap. So far each time I've come across
| someone who's fallen down the types all the way down
| hole, they've shared a view like this.
|
| In the times where i've dug deeper, they were applying a
| bad unit testing strategy where they aimed for 1:1(+)
| ratio of methods to unit tests - save that granular level
| of testing for really specific cases. The heuristic to
| use is "can i refactor my code without changing a single
| test?" - if your refactor commit has changes to tests
| then you're not refactoring your just changing. Your unit
| test suite is too tightly coupled and is going to cost a
| fortune to maintain, even more so in a statically typed
| language.
| skohan wrote:
| I don't know what the " the types all the way down hole"
| is, but you sure seem to be ascribing a lot of views to
| me which I don't hold. I also don't know what this
| tangent into unit testing strategies has to do with any
| of the prior points about type systems.
| nonbirithm wrote:
| > probably _most_ of the time we don 't need to do fancy
| type manipulation or dynamic dispatch at runtime
|
| The fact is, I really wish there were a way to have the
| ability to build systems that you can recompile at
| runtime that had types. The recompilation itself is the
| "problem," or at least an inconvenience. You have to
| interrupt your flow in order to test even the smallest
| change. Yes, this is a selfish programmer desire to make
| building things easier.
|
| The general consensus is that it's not worth having
| flexibility over the stability that types enforce, and
| for _most_ cases I would agree. It would be ridiculous to
| program critical systems like financial software with the
| expectation you 'd be able to walk right up to it and
| deploy changes whenever you want. There isn't a need for
| extensibility with that use case. But there are some
| cases where extensibility is a key part of the
| development cycle, like Emacs or Excel or game
| development. Iteration is important in those cases,
| seeing that you can make some changes and test them out,
| "teaching" the program what you want until it's just
| right. My experience is that this is an optimal
| development cycle, where I'm not prone to be distracted
| easily and lose more time than I needed to when I
| intended to just compile something.
|
| It's just that I'm not seeing why having types means you
| _must_ use a machine-compiled language, meaning you now
| have to reboot the program to see your changes. I wish
| there was a best-of-both-worlds solution that works in
| the weird way I expect. The closest thing I know of is
| Teal, which compiles to Lua, but it will be a while
| before it 's production-ready.
| c-cube wrote:
| I keep hearing that, but in practice one can have
| macros/code generation in a static language. Maybe meta
| programming takes a bit more effort, but it's the kind of
| thing best done in a well tested library.
|
| A lot of modern typed languages have got pretty decent meta
| programming: rust procedural macros, nim, D, crystal,
| Zig... Where's the equivalent of serde in python? It's
| mostly runtime type driven deserialization that is orders
| of magnitudes slower.
| CraigJPerry wrote:
| hyperjson? (Python bindings for Rust's serdes-rs).
|
| It's not really a cheat answer - what if you could
| iterate quickly and launch 5 startup ideas in the time to
| launch 1 written in Zig but when one of your 5 turns out
| to be the next Youtube / Instagram / Facebook / [insert
| other huge traffic site launched on a "slow" backend
| here] then you have the freedom to glue in some optimised
| native code in the hot path?
|
| I haven't gotten to macros yet in my Rust playtime so far
| but Rust does seem like a really nice complement to
| Python. Go/Swift strike me as a faster Python - nothing
| wrong with that, Rust strikes me as a different kind of
| tool, that appeals. Zig / nim are too bleeding edge for
| me, when i saw the cool kids start to decry Rust, that
| was my cue to finally go order the O'Reilly book and dig
| in (i'll admit to reading up about comptime to see what
| the fuss was about - cool idea, i'll stick with trying
| Rust for now).
| c-cube wrote:
| That's interesting. The recent post about a fast
| dataframe library in rust (polars, I think) and its
| python bindings is also telling. Py3O might be a
| rust+python kind of secret weapon in terms of speed and
| productivity... And it's based on the procedural macros I
| was mentioning :-).
| pansa2 wrote:
| What might "Python, but with modern type inference" look
| like? Are there other existing languages that come close to
| matching that description?
| ojbyrne wrote:
| I'm going to take a guess and say that the OP is referring
| to languages that implement the Hindley-Miner algorithm.
| According to Wikipedia: "Some languages that include type
| inference include C++11, C# (starting with version 3.0),
| Chapel, Clean, Crystal, D, F#,[1] FreeBASIC, Go, Haskell,
| Java (starting with version 10), Julia,[2] Kotlin, ML, Nim,
| OCaml, Opa, RPython, Rust, Scala, Swift,[3] TypeScript,[4]
| Vala, Dart,[5] and Visual Basic (starting with version
| 9.0)."
|
| https://en.wikipedia.org/wiki/Type_inference#Types_in_progr
| a...
| nicoburns wrote:
| Nim?
| joshuamorton wrote:
| Python, with a few type annotations and pytype.
| ipaddr wrote:
| Possible referring to php and the auto type casting?
|
| He couldn't possible mean javascript or rust. Perhaps
| golang because of the limited subset of types.
| skohan wrote:
| Swift/kotlin are better examples of languages which allow
| you to stay very high level with types to keep you honest
| CraigJPerry wrote:
| Time to market?
|
| Type inference solves a different problem - it doesn't help
| you here. For example type Money {
| amount: Amount, currency: Currency }
| Money(USD, 10) + Money(EUR, 10) # what type should be
| inferred here?
|
| A runtime exception maybe?
|
| (There's no "best" answer here, a runtime exception might be
| the pragmatic choice but i like the Map<Currency, Amount>
| solution to this).
| exdsq wrote:
| It would help catch the fact you wrote it as both
| <Currency, Amount> and <Amount, Currency>
| CraigJPerry wrote:
| :-) >>> class Money: ... def
| __init__(self, currency, amount): ... self.m
| = Counter({currency: amount}) ... def
| __add__(self, other): ... self.m += other.m
| ... >>> Money("USD", 10) + Money(10, "EUR")
| Traceback (most recent call last): File
| "<string>", line 1, in <module> File
| "<string>", line 5, in __add__ File "/var/conta
| iners/Bundle/Application/F8EBA093-47B6-4538-A3A5-553C9C22
| 34FA/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/co
| llections/__init__.py", line 805, in __iadd__
| self[elem] += count TypeError: unsupported
| operand type(s) for +=: 'int' and 'str'
|
| Even Python yells at me for that.
|
| In all seriousness though, how far do such mistakes
| survive? Not past compile here, but in other cases it
| could. Could it pass tests? Unlikely. Could it pass
| casually running the code, not impossible but again seems
| unlikely.
|
| How many PRs on Github for projects written in
| dynamically typed languages are things that would be
| caught by static types? Some, it's non-zero, it's not
| anywhere close to a significant proportion though.
| joshuamorton wrote:
| Fwiw there are type systems that can catch this. C++, rust,
| and pythons in fact both have support for erroring on this
| by default, unless you have overridden the addition
| operator for Money to always result in the lhs or the
| currency, or whatever.
|
| See, for example, the absl::Duration apis in c++.
| CraigJPerry wrote:
| use std::ops::Add; fn main() {
| let usd = Money { currency:
| String::from("USD"), amount: 10
| }; let eur = Money {
| currency: String::from("EUR"), amount: 10
| }; dbg!(usd + eur); }
| #[derive(Debug)] struct Money {
| currency: Currency, amount: Amount }
| impl Add for Money { type Output = Money;
| fn add(self, other: Money) -> Money { Money
| { currency: self.currency,
| amount: self.amount + other.amount }
| } } type Currency = String;
| type Amount = i32; // i know :-)
|
| This outputs 20 USD in rust, it's a design problem not a
| types problem.
| joshuamorton wrote:
| I'm not sure what your point is.
|
| It's glaringly obvious that this code is functionally
| wrong, in no small part bc of the type information.
| skohan wrote:
| Unless your application is implemented in under 100 lines,
| types will save you time in general
| CraigJPerry wrote:
| This is obviously false, it's just too general a
| statement. Type errors are such a small time loss as a
| developer - when i think of the things that lose me time,
| it's things like domain complexity, misunderstood
| requirements, not "ooops i passed a string instead of
| list of string".
|
| I wondered if there are any studies that support your
| assertion? The first few results in Google all refute
| your idea though, e.g. https://www.researchgate.net/publi
| cation/259634489_An_empiri...
|
| Or https://www.ics.uci.edu/~jajones/INF102-S18/readings/2
| 3_hane...
|
| And that intuitively makes sense. When i'm adopting
| types, time saving isn't really what i'm buying. It's the
| opposite - i'm saying "hey, i want to invest more time
| into this to help ensure correctness" - i.e. i am happy
| to wrestle with the type system if it means the
| delegation frees my brain for solving the problem.
|
| And this comes back to what i mentioned earlier about
| domain complexity - you can use types to help make that
| easier to manage, often though, it's better to push
| harder to eliminate complexity in the first place.
| Occasionally that's just not possible, it's a complex
| problem domain not another CRUD webapp, in those cases
| types are fantastic.
| skohan wrote:
| > Type errors are such a small time loss as a developer
|
| It's not about the errors themselves, it's about the way
| that having a strong type system improves quality of life
| in general.
|
| For instance, in a duck-typed language, you may have code
| which runs correctly 99/100 times, but with one specific
| case it fails due to type issues. These kinds of issues
| are incredibly frustrating and time-consuming to debug.
|
| Also requiring type designations in function signatures
| makes code self-documenting. You can just look at the
| code which someone else wrote, and know exactly what the
| inputs and outputs are. In a dynamically typed setting,
| there's often guess work involved in exactly what's
| accepted by a function, and unless it's very well
| documented you might have to go into the implementation
| to see exactly what is happening.
|
| I didn't read the articles you linked in detail, but at
| least the first I would take with a grain of salt since
| they use Java as the exemplar for typed programming. Java
| is notorious for having a cumbersome type system.
|
| In my experience, without types you might get a
| _slightly_ faster time to first result - but in the long
| term, once a codebase gets to even medium size, types
| save you much more time than they cost. Especially since
| they become automatic after a bit of experience and you
| don 't even think about it anymore.
|
| There's a reason Python and Javascript are trying to add
| types, and why none of the most interesting languages of
| the past decade have been weekly typed.
| CraigJPerry wrote:
| I think you might have missed my point, it's not a binary
| good/bad - both approaches have a useful place.
| skohan wrote:
| I think we just disagree. I understand your point, but I
| think that the benefits of having strong types outweigh
| any drawbacks in almost all cases. The benefits are huge
| once you get past a very small program (100 lines was a
| little flippant on my part) and the drawbacks rapidly
| approach zero for any sufficiently well-designed type-
| system
| CraigJPerry wrote:
| :-) FWIW, through my career i changed my views on this
| topic.
|
| I started off loving dynamic, then i spent years
| preferring static. Nowadays i don't actually have too
| strong an opinion on types (despite what this thread
| might suggest!), both approaches have value.
|
| These days i place value in something else - less lines
| of code. Expressive languages like Python & Clojure,
| these make a lot of sense to me because in my experience
| nothing correlates stronger with bugs than too many lines
| of code. By this logic i should be writing in APL but i
| suspect that goes too far the other direction!
| mumblemumble wrote:
| What seems positive to me is that there's a good mix. It speaks
| to the eminent pragmatism of the way Python decided to handle
| typing and type hints.
|
| I've found that there is no one best typing discipline.
| Different approaches work better for different problem domains.
| And I'm currently using Python in so many different problem
| domains that I find it convenient that Python now allows me to
| choose the best typing discipline for the job.
|
| It's true that this means that Python isn't the best at any
| particular way of doing typing. But that also seems
| appropriate. Being a jack of all trades and master of none is
| sort of Python's thing nowadays.
| BurningFrog wrote:
| Here is my thought on mypy:
|
| 1. If you have a good test suite, mypy adds less value than it
| costs.
|
| 2. If you don't have a good test suite, you _really_ need to get
| one.
|
| How am I wrong?
| skohan wrote:
| When I read the twitter replies, it doesn't make me optimistic
| for the future of typed python. I don't work with Python often,
| so I don't know what the "truth on the ground" is, but it seems
| like having a language divided between several different type-
| checking implementations is going to be a nightmare for things
| like interoperability of frameworks.
|
| I'm imagining a future 5 years down the road where some poor
| python developer has to use some "polyfill" solution because they
| want to import a library which was developed with a type-checking
| solution which is no longer main-stream, or because the type-
| checkin solution they're using has breaking changes with respect
| to previous versions.
| philwelch wrote:
| Python 3 type annotations are part of the language spec.
| Whether you use mypy or a different library for type checking
| shouldn't necessarily make a difference to the code.
| ali_m wrote:
| Type annotations may be part of the language spec, but the
| way they are enforced is not. Different type checking
| libraries have made different design choices (leniency vs
| strictness, type inference vs gradual typing etc.). Although
| these differences don't matter at runtime, they do mean that
| code which passes one particular type checker might fail with
| another.
| davnicwil wrote:
| Is the solution here just to skip type checking the code of
| dependencies, and just check the types at the interface
| with your own code using your tool of choice? Or is it more
| complex than that?
| jraph wrote:
| I think so. When typing checking a project, its
| dependencies don't get type checked as well as far as I
| know. Type errors in dependencies are kind of not
| relevant for you, and many dependencies are not type-
| annotated anyway.
| ali_m wrote:
| Well you probably do still want the type checker to at
| least look at the public interfaces of any libraries you
| depend on, since you want it to enforce that you're
| calling into them correctly.
| jraph wrote:
| Absolutely. It should probably not report errors that are
| not in your code, by default anyway, but still extract
| information about typing relevant to you as much as it
| can.
| neilparikh wrote:
| I think that mostly works, but it's still possible that
| different tools assign different semantics for the same
| syntax, which could then interact in a weird way if the
| library author wrote the interface annotations with Tool
| A's semantics in mind, while you ended up checking them
| against your code with Tool B.
|
| I think the type systems are simple enough that this is
| probably a theoretical concern, and won't come up often
| in practice.
| throwaway81523 wrote:
| There actually seems to be no story about inductive types
| (e.g. trees) in Python and this appears due to well known
| theoretical difficulties combining subtyping (required for
| OO) and type inference (?). I.e. it is a hard problem.
| There are a few long term open mypy github issues about it,
| and some hacky workarounds, but no real answer. This is
| also why languages like Haskell (with good support for
| inductive types) tend to not have OO. I'm not exactly
| familiar with the issues, but there is some PLT literature
| about it, by Anton Setzer and others.
| midrus wrote:
| The ecosystem is already pretty split on several fronts, not
| just typing. Python 2 vs 3, async vs blocking, packaging tools,
| etc.
|
| I left python 3 or 4 years ago and not planning to go back
| (basically the company I was working for mandated they were not
| going to ever upgrade to python 3 because of the effort
| required and thus stopped all migration related projects).
|
| The language is nice, and frameworks such as Django are great,
| but the mess the ecosystem has become is not something I miss.
| heavyset_go wrote:
| > _Python 2 vs 3_
|
| This isn't an issue in 2021. Python 2 reached its EOL over a
| year ago, and those who own old Python codebases have had
| known for literally _almost 13 years now_ that Python 2 would
| be EOL 'd when Python 3 came out in 2008.
| guitarbill wrote:
| I agree that the ecosystem is a bit messy, but it's more of a
| function of how long Python has been around, and how
| successful it has been. Newer languages have the benefit of
| hindsight.
|
| Take `go fmt` or `cargo fmt`. black is the de facto
| autoformatter, but it's still an optional dependency. Maybe
| like virtualenv it'll eventually be integrated into the
| standard library/distro. That would help. With other stuff,
| the split you describe could be a real problem, although for
| the majority of Python users, using mypy and poetry is fine.
|
| > basically the company I was working for mandated they were
| not going to ever upgrade to python 3 because of the effort
| required and thus stopped all migration related projects
|
| If you're actually serious, this is intriguing to me.
| Business-wise, I have never seen the math shake out this way.
| When we've done sizings/estimated developer hours and ROI,
| it's always been more economic to slowly migrate to Python 3
| vs a rewrite, even if the dev team is also already using and
| well-versed in another language. If you're able to provide
| more details, please do. It's so rare to have a high-quality
| discussion about this.
| AlphaSite wrote:
| Python 2 is pretty dead at this point.
| johnnycerberus wrote:
| I think your comment is a little bit alarmist.
|
| * Python 2 is dead.
|
| * pip is the most popular in the industry, conda serves a
| different purpose if your focus is mainly on scientific
| computing
|
| * what does async vs blocking even mean in this context? all
| languages have the same problem, just use the libraries that
| are non-blocking.
|
| * mypy is the static type checker that is the most popular
| (28.3k repositories, 1.9k packages), if you are using other
| type checkers then congrats, you will just be the
| CoffeeScript/Reason user of the JS world, refusing to believe
| that TypeScript won the mindshare
| joshuamorton wrote:
| Pytype (Google) pyre (facebook) and pyright (microsoft) all
| have more institutional backing than mypy. They aren't
| going away anytime soon.
|
| Otoh, development of all the major type checkers is fairly
| collaborative. Mypy usually gets features first, but not
| always.
| spicybright wrote:
| Why not just adopt one into the standard library, seems
| like a very obvious battery you'd want to include.
| maple3142 wrote:
| > what does async vs blocking even mean in this context?
| all languages have the same problem, just use the libraries
| that are non-blocking.
|
| Maybe Node.js? Although it does have some sync function to
| do io, most of them are still unblocking, and the ecosystem
| is unblocking by default too. It is harder to find a
| blocking library than finding a non-blocking ones.
| heavyset_go wrote:
| There is no semantic meaning given to type annotations by the
| interpreter. There's no need for polyfill nor are there issues
| with interoperability of frameworks because, again, type
| annotations have no semantic meaning.
|
| They're like comments, they're only there for developer
| convenience and can be completely ignored.
| [deleted]
| canjobear wrote:
| It's always possible to run code ignoring all type annotations.
| smitty1e wrote:
| It's excellent that Python lets one light off a REPL and work on
| solving the problem, then annotate the answer later for all of
| those good reasons.
| ExtraE wrote:
| From a twitter reply:
|
| > Surely there's a real statically typed language with similar or
| better ergonomics?
|
| Please, if someone here knows of one, let me know!
| carapace wrote:
| I recently had a lot of fun exploring Nim. I'd like to use
| Haskell but it's such a beast; Elm lang is very promising
| though (IMO.)
| yxhuvud wrote:
| Crystal could perhaps fit the bill. It is (a lot) closer to
| Ruby in style though.
| hannofcart wrote:
| There are people in both Twitter and HN comments saying "if you
| want types, just use a typesafe language".
|
| What I found though, is that 'typed' Python is a great
| language/dialect in its own right.
|
| There's a confluence of simplicity and ergonomics that I find
| quite appealing, even as compared to mainstream static typed
| languages.
|
| I had previously submitted to HN with my thoughts on this:
|
| https://news.ycombinator.com/item?id=25520411
|
| I think the primary difficulty I have seen people face is getting
| a buy-in from all members of the team. One or two members in
| every team seem to prefer to write Python the "way it was
| originally intended" and consider type annotations more of a
| nuisance. And this ends up hurting adoption.
| dehrmann wrote:
| > 'typed' Python is a great language/dialect in its own right.
|
| Like OO in Python, it feels bolted on. The syntax for generics
| is bad, at least compared to Java. Typing also exposes how
| poorly thought out Python's collections are. The hierarchy and
| methods make sense, but some things are in typing, others in
| collections.abc. It also falls over when using things like
| functools.partial.
|
| Then there's the frequent complaint that if you're going to go
| to the effort to use something typed, you should use something
| that can take advantage of those types during compilation to
| optimize the code.
| 127 wrote:
| I found myself writing this kind of code recently:
| https://pastebin.com/fpQzRypp
|
| Sometimes code is being plugged between two highly unstable
| things. It just needs to work and can't trust that any of the
| data it receives is valid or good.
|
| I should really be using types here.
| hannofcart wrote:
| Indeed, you'll find that type annotations and enforcing them
| with a type checker will eliminate all those `assert`s for
| `not None`.
| djohnston wrote:
| That seems totally reasonable. Progressive type safety is one of
| the major wins of Python IMO. I don't need type annotations to
| write some codemod script, certainly do to write a robust web
| service.
| stephenroller wrote:
| I run a fairly large open source project
| (https://github.com/facebookresearch/ParlAI/) and we use mypy.
| Our experience has been that it can be quite difficult to
| placate, so we usually treat it only as a warning. However,
| having our code annotated with types in many places has
| significantly improved developer productivity, just from having
| less ambiguity with what you're dealing with.
| redsymbol wrote:
| Important to realize glyph's twitter followers are not
| representative of all Python users.
|
| In general, people who follow him are going to be more "into"
| Python than someone who just uses it to get their job done, and
| thus more likely to use optional tooling with a large overhead
| like mypy.
|
| That will skew the results in favor of mypy in this poll. Though
| the poll getting on the HN front page will balance that somewhat.
| make3 wrote:
| One of the first things you learn in theoretical sampling
| classes is that polls where people self select to participate
| (like twitter polls) are garbage
| duxup wrote:
| Arguably that applies to all social media ;)
|
| I see a lot of sort of 'programming culture' for given
| languages, practices, even discussion of working conditions and
| etc that on social media are taken as standard operating
| procedure. Highly active and even highly knowledgeable folks
| seem to portray various practices and etc are a given ... that
| I suspect are actually outliers.
| spicybright wrote:
| I agree listen to the heavy weights on these sorts of things.
| But a poll open to everyone is pretty useless.
| croh wrote:
| Yup agree. People who follow him are more "into" python async
| stuff.
| watermelon0 wrote:
| I'm just using it to get my job done, and really couldn't
| imagine working with Python without it.
|
| I'd consider static type checker as optional if you are playing
| around with small or throwaway scripts, but for any half
| serious project, it should be a requirement.
|
| IME I don't consider it slow, and it's definitely not visibly
| slower than pylint, flake8, or similar tools for other
| languages.
|
| Dropbox, which has one of the biggest Python codebases, uses
| it, and most mypy maintainers including Guido work there.
| you_are_naive wrote:
| The biggest problem with mypy is that it's written in python.
| It's slow. It needs to be rewritten in rust or go to make it
| faster like how js ecosystem is moving in that direction. All
| the tooling is moved to faster compiled languages.
| watermelon0 wrote:
| mypy is actually compiled via mypyc, which should be a few
| times faster than interpreted Python
| https://github.com/python/mypy/tree/master/mypyc
| crazypython wrote:
| mypy has implementations in other languages - PyCharm's is
| in Java, Facebook's Pyre's is in OCaML
| https://github.com/facebook/pyre-check
| MapleWalnut wrote:
| Those tools are not re-implementations of mypy, but their
| own type checkers. Pyre doesn't necessarily have all the
| features of mypy and vice versa.
| wk_end wrote:
| Initial type checking with mypy is slow, but if you use
| mypy-daemon each check after that becomes pretty much
| instantaneous, even on 10M+ line code bases.
| Twirrim wrote:
| > Dropbox, which has one of the biggest Python codebases,
| uses it, and most mypy maintainers including Guido work
| there.
|
| Guido hasn't worked at Dropbox since 2019. He retired for a
| bit, but is now in the Developer Division at Microsoft.
| watermelon0 wrote:
| My bad, I just checked http://mypy-lang.org/about.html,
| which appears to be outdated.
| bostonsre wrote:
| Yea, it may slightly slow down the initial draft due to
| needing to type a little bit more, but it helps speed up
| maintainability and readability a ton and it helps eliminate
| a bunch of bugs so I think it has definitely increased
| productivity in my experience.
| lambda_obrien wrote:
| When my company upgrades to 3.8 or higher, I'll be happy to
| include as much typing as possible. I can't wait until I can
| refactor Python more easily, it's a huge pain right now.
| hsaliak wrote:
| The survey may not represent sentiment toward type checking in
| Python. Mypy is only one of the checkers out there. Out of the
| 40% who don't use mypy, many could be invested into other type
| checkers such as Pylance. This is certainly what many comments on
| the Twitter thread are about.
|
| So the % who refuse to use type checking is likely lower than 40%
| super_mario wrote:
| I personally hate typing in Python. Python now feels like
| Typescript, and when using pyright, you actually get something
| that looks very much like type file for JS when you step into
| library code, instead of library source code itself.
|
| Honestly, I don't like the direction Python is taking lately. It
| used to be easy, approachable, dynamically typed, strongly typed
| language. Now, syntax is getting polluted and there are
| definitely more than 2 ways of doing things.
| dehrmann wrote:
| Typescript at least has the excuse that you have to run JS, so
| the approach is more sensible. With types in Python, for the
| most part, you could choose a different language.
| ThePhysicist wrote:
| I gave a presentation on type annotations at the EuroPython 2017
| where I also investigated empirically how many open-source Python
| projects really use type checking, I present the results at the
| end (33 minutes in):
|
| https://www.youtube.com/watch?v=vM2Zoy4Sxhk&t=2181s
|
| My conclusion was that only a small fraction of projects really
| used them, there were a lot of projects that had type checks in
| their code but only in "homeopathic" doses. I started using them
| for some of my Python projects as well (e.g
| https://github.com/algoneer/algoneer) and while I find them
| useful I think they're not as useful as a "real" type system in a
| fully typed language like Golang. Still, they're very useful for
| discovering simple mistakes that would only show up in unit
| testing otherwise.
|
| You can also "misuse" them for other purposes, at the end of the
| presentation I e.g. show how you can implement software contracts
| with them. Of course this would wreck a type checker like mypy,
| so don't do that in your codebase. That's probably also one of my
| critiques as the annotation syntax can in principle be used for
| anything, but mypy and other tools are not able to deal with code
| that does that.
___________________________________________________________________
(page generated 2021-03-21 23:02 UTC)