[HN Gopher] Show HN: Koda, a Typesafe Functional Toolkit for Python
___________________________________________________________________
Show HN: Koda, a Typesafe Functional Toolkit for Python
Author : keithasaurus
Score : 75 points
Date : 2022-02-07 16:46 UTC (6 hours ago)
(HTM) web link (pypi.org)
(TXT) w3m dump (pypi.org)
| hackandtrip wrote:
| A similar "framework", very robust and used in production
| systems, is dry-python's one[0][1].
|
| The approach taken is a bit different I think, since they rely
| heavily on `mypy` plugins to reach type safety and functional
| constructs otherwise impossible to get, without runtime
| inspections.
|
| 0: https://github.com/dry-python 1: https://github.com/dry-
| python/returns
| keithasaurus wrote:
| dry-python has a lot of cool stuff. I encourage people to check
| it out. Koda has a slightly different approach in that it's
| meant to be a little simpler than returns, and to work within
| Python's constraints.
| impoppy wrote:
| Why would you write Python like it's Rust? I really fundamentally
| do not understand the purpose. Ain't `is Null` checks and try
| except good enough?
| rmbyrro wrote:
| Came here to say the same thing...
| keithasaurus wrote:
| I think the discussion is far enough along here to cover a lot
| of the bases. Just to add a bit of clarity, Scala's APIs were a
| bit more influential than Rust, but, yeah, it's similar to
| both.
| IshKebab wrote:
| Yeah this sort of thing is necessary in statically typed
| languages like Rust and C++ but in Python (and e.g. Typescript)
| you can just use anonymous type unions like `str | None`.
|
| There's no need to construct a static type to hold both
| possibilities because variables can already hold all types.
| hackandtrip wrote:
| Good luck checking for `None` equality and indent your code
| like 5 times... a sort of railway programming[0] avoids the
| awkward checks for values, making functions work with higher
| order types.
|
| 0: https://www.greenbird.com/news/railway-oriented-
| programming-....
| impoppy wrote:
| How is this different from match case ok case err or from
| if result.val?
| hackandtrip wrote:
| Well, the most basic pro is to avoid indentation and have
| pure function which are clear and have less assumption to
| take into account; if I know that my pipes always return
| me a result such as the one I want, and the "bad path" is
| discarded automatically, code is way cleaner.
|
| Imagine a code without `if result is not None` basically!
| ReleaseCandidat wrote:
| The original Railway:
|
| https://fsharpforfunandprofit.com/rop/
|
| And his 'Against Railway-Oriented Programming':
|
| https://fsharpforfunandprofit.com/posts/against-railway-
| orie...
| dragonwriter wrote:
| foo | None
|
| is not a tolerable replacement for Maybe(foo)
|
| When None is a valid value of type _foo_.
|
| Which is obviously true when _foo_ is NoneType, but more to
| the point is true when _foo_ is ( _bar_ | None).
|
| Maybe composes, "... | None" does not.
| btown wrote:
| I've definitely hacked my way around this with sentinels.
| `foo | sentinel | None` is a passable replacement for
| Maybe(foo), and it arises naturally from dict.get("key",
| SENTINEL). Absolutely not fun to need to check, but if
| you're using it within a single function e.g. an
| implementation of `deep_get` or something like that, the
| code is readable.
| dragonwriter wrote:
| Yeah, there's lots of things that work good enough for
| localized use to approximate something like Maybe or
| Result and a lot of times that's all you need, and (in
| Python) more convenient than the real thing when that's
| all you need.
|
| Wanting something that's ergonomic for free composition
| where you aren't anticipating all the potential uses
| where it is defined is where real monadic Maybe/Result
| types shine, and once you pull them in to a code base it
| sometimes makes sense to use them for the local cases,
| too.
| impoppy wrote:
| a) do Optional[Union[foo, bar]] b) why do you expect this
| from a duck typed language? None of options you listed or
| the library is offering saves you from mismatched types.
| Mypy is still as good as it can possibly be using only
| vanilla type hints
| [deleted]
| DandyDev wrote:
| Because you'd like use the Python ecosystem, for example for
| writing ML code, but you'd also like to write typesafe code (or
| rather typechecked, using mypy)
| impoppy wrote:
| I let myself do a little bit of thinking and I guess this might
| be helpful when you are doing functional style pipes/chains but
| you can wrap the whole chain into a try except block. But the
| question is still there: why would you do that? Why the one
| would write functional code in a language that ain't very good
| with maps and filters, has no lazy calculations and has no
| constants on language design level? Why would you set
| `Maybe[int]` type for a function argument? That's an obvious
| overcomplication imo. Go and write some F# or Haskell. Well,
| the variety of options and flexibility is always good, but the
| language is fundamentally not meant to do it. From what I can
| see, it only makes sense in recreational programming, making
| some eye-pleasing code, but it all is just a workaround on a
| language that would not benefit from that code style.
|
| All above is strongly imo of course since I only write Python
| and am just learning Zig.
| dragonwriter wrote:
| > Why the one would write functional code in a language that
| ain't very good with maps and filters, has no lazy
| calculations and has no constants on language design level?
|
| Python is fine with maps and filters, and has lazy
| calculations.
| impoppy wrote:
| Please provide links so I can read on this
| dragonwriter wrote:
| > Please provide links so I can read on this
|
| https://docs.python.org/3/library/functions.html#map
|
| https://docs.python.org/3/library/functions.html#filter
|
| https://docs.python.org/3/howto/functional.html
| muxator wrote:
| The map() function is already lazy (in python 3 it
| returns a generator, and not a materialized list).
|
| You can also have a look at "generator expressions",
| which are the lazy counterpart to a list comprehension
| (hint: use " ()" instead of "[]").
| deltaonefour wrote:
| Python syntax is very very well suited for functional
| programming. A python map and filter is done with list
| comprehensions and even flatmap is doable within the list
| comprehension ecosystem. Additionally the syntax transfers
| over to inputting values into n-arity functions and even
| dictionary comprehensions. Note that this feature Isn't even
| part of the python standard library... map and filter and
| flatmap are an inherit part of the CORE syntax. It makes it
| much better suited for FP then javascript imo.
| x = [i*2 for i in range(5)] #[0, 2, 4, 6, 8] y = [i
| for i in range(8) if i%2 == 0] #[0, 2, 4, 6, 8] z =
| [[1],[2],[3],[4]] w = [j for i in z for j in i]
| #[1,2,3,4] t = {str(j):j*2 for j in w} #{"1":1, "2":4,
| "3":6, "4":8} #reduce (note the generator (i
| for i in range(20))... that's a lazy list essentially)
| reduce(lambda acc, x: x + acc, (i for i in range(20)), 0)
| #sum(0 ... 20) #get max value from a
| bunch of numbers: f = max(i for i in range(30) if 30 %
| 3 == 0) #max(0, 3, 9, 12, 15, 18, 21, 24, 27, 30) = 30
| #basic recursion def myreverse(x: str) -> str:
| return x if len(x) <= 1 else x[-1] + myreverse(x[:-1]) #O(N)
| on each slice, (However x[-1] is O(1) unlike the haskell
| list). #qsort haskell style def qsort(l:
| List[int]) -> List[int]: return l if len(l) <= 1
| else qsort([i for i in l if i <= l[0]]) + qsort([i for i in l
| if i > l[0]])
|
| Python also supports pattern matching shown here:
| https://www.python.org/dev/peps/pep-0636/
|
| Additionally lambda syntax, aka python anonymous first class
| functions HAVE to be functional. f = lambda
| x: x + 1
|
| There is no procedural concepts like variable assignment
| allowed in a python lambda. This is done because they wanted
| to force the python lambda to be properly functional and
| short.
|
| The type hints that python uses are also quite good with
| support for generics sum types and even type level
| programming (however your type checker needs to support it
| too and basically none of them do right now) You can
| literally assign a type to a variable in python. See:
| https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html
|
| I would say the main weakness in python is that the
| IMPLEMENTATION is not well suited for functional programming
| (but the syntax VERY much is). Tail slices on lists like
| x[1:] cost O(N) and that's the biggest issue right now imo.
| It makes some valid solutions on leetcode exceed the time
| limit because it adds and additional N complexity on every
| slice when the head and tail functions in haskell cost O(1)
|
| I'm sure some library can easily provide the underlying
| functional primitives python needs to be faster. Actually on
| that note, does anyone know of a library that solves the head
| tail slicing problem I described above? Probably should use
| the python deque, but the api for that is inherently not
| functional.
| keithasaurus wrote:
| > There is no procedural concepts like variable assignment
| allowed in a python lambda.
|
| Just a small note that the walrus operator gives you some
| wiggle room here:
| add_5_to_square_if_greater_than_10 = lambda x: squared + 5
| if (squared := x * x) > 10 else x assert
| add_5_to_square_if_greater_than_10(2) == 2 assert
| add_5_to_square_if_greater_than_10(5) == 30
| deltaonefour wrote:
| yeah my bad. Not fully up to date with all the newer
| python 3 tricks. That walrus operator is a trap though
| imo.
| ok_dad wrote:
| > ain't very good with maps and filters
|
| Isn't a comprehension just a map/filter by another syntax?
| You can re-create them from a list comprehension pretty
| simply:
|
| `map = lambda data, map_fn : [map_fn(x) for x in data]`
|
| `filter = lambda data, filter_fn : [x for x in data if
| filter_fn(x)]`
|
| Then, why even have the lambda-translations above since the
| comprehension is already "Pythonic"? Also, you have functools
| (https://docs.python.org/3/library/functools.html) which you
| can use for several functional programming paradigms as well.
|
| Functional programming isn't _what_ you call the techniques
| (map /filter/etc.) it's _how_ you are programming that makes
| it functional. You can still use functional techniques in
| Python to make things a bit easier to reason about. A lot of
| the things people thing about functional programming aren 't
| inherent to the actual functional programming paradigm: like
| laziness or immutability.
| muxator wrote:
| Nit: in py3 map() and filter() are lazy (they return
| iterators instead of lists). You can better mimic their
| semantics using generator expressions "()" instead of list
| comprehensions "[]".
| btown wrote:
| Say you're given a Python function in a large codebase that
| doesn't have documentation but does have type annotations, say,
| to indicate it returns Optional[str]. Does this tell you that
| you just need to check for None, and otherwise it's a string?
|
| Nope, it could throw. What could it throw? We don't know. There
| are many applications where this magnifies the dangers of any
| existing technical debt. One could desire strong exception-
| proofing of results without needing to go "all in" to a borrow-
| checked regime like Rust.
|
| The Koda approach is lightweight and doesn't enforce this, but
| one could imagine a static checker that says "any function that
| returns a Result, and calls any function that does not _itself_
| return a Result or does any other risky operation, must have an
| exhaustive top-level try-except block, or be wrapped by a
| decorator that does the same. "
|
| Then, you could guarantee that all code explicitly handles
| exceptions, while still accessing the full Python library
| ecosystem - you can use non-Koda-wrapped code at any time, you
| just need to handle its exceptions.
|
| Is this overkill? Possibly. But I can think of a handful of
| bugs offhand that this would have caught in our codebase if
| enforced.
| leethargo wrote:
| I wanted to ask the same thing: Couldn't we just use Optional
| and None for this? So the point of using Koda is to
| communicate intent (functions that don't throw exceptions),
| and some utility methods?
| keithasaurus wrote:
| You can also nest `Maybe`s. Just(Just(nothing)) could
| convey that two levels of a computation "succeeded", but
| the third did not. This may not be something you see often,
| but it's a case that Optional can't convey.
|
| There are also cases where None is a valid value, and then
| Optional doesn't make sense. Maybe[None] can be a valid, if
| rare, use case -- Just(None) is different from Nothing. But
| Optional[None] doesn't make sense -- since it's None |
| None.
|
| You can also do other stuff with Maybe, like map, flat_map,
| apply. And there's more that can be added.
| impoppy wrote:
| Now this is an answer I was waiting for. Thank you for
| explaining the real benefits of sticking to koda style. I
| guess this would be obvious if I used statically typed
| languages since there is no try except clauses in most of
| them
| btown wrote:
| I'll add that caveats do apply to these benefits. Many of
| them are outlined in
| https://fsharpforfunandprofit.com/posts/against-railway-
| orie... - which is a response to an excellent functional
| programming talk (see slides or video linked at
| https://fsharpforfunandprofit.com/rop/) that walks through
| why you'd want a Result-type thing to exist in the first
| place... and eases you into the larger world of monads!
| ReleaseCandidat wrote:
| > and eases you into the larger world of monads
|
| Yes, just begin reading at page 29
|
| https://www.slideshare.net/ScottWlaschin/railway-
| oriented-pr...
| [deleted]
| fishmarmalade wrote:
| Really nice project! I've been looking for something like this,
| and typically end up building a very minimal version of it for
| projects. Are you looking for help or contributors? Would love to
| contribute if possible.
| keithasaurus wrote:
| Yeah, happy to have contributors. I'm not 100% certain on what
| the next features / changes will be, so also happy to have
| input, requests, questions, etc.
| pizza wrote:
| Nice. One question I have is does the compose operator preserve
| error messages, allow you to pinpoint which function raised an
| exception etc? That's the one issue I have w/ the `pipe` function
| from dry-python/returns; if you build a complex pipeline, the
| errors are somewhat inscrutable.
| hackandtrip wrote:
| Can't you 'compose' together the various errors inside the
| `Failure` and inspect it at the end of the pipe?
| keithasaurus wrote:
| So far I've been able to follow them. If you `raise` an
| exception during one of the functions that's used in a compose
| the traceback will mention the function that failed (I'm
| looking in 3.9 at the moment).
| zestyping wrote:
| I was pretty puzzled by the method name "map" in the examples.
| Why would you call it "map" when there are no arrays or lists
| involved?
| cheriot wrote:
| Are you thinking about Maybe.map in particular? It's the same
| name because it does the same thing: run this function on the
| value inside if there is one. It's a Functor :)
| dragonwriter wrote:
| A Maybe is effectively isomorphic to an immutable list whose
| cardinality can only be zero or one.
| CodeAndCuffs wrote:
| x.map(f) doesnt mean "apply function f to each element in this
| list that we called x"
|
| x.map(f) means "apply the function f to the value within x,
| where x is a container of some sort"
|
| [].map(f) would do nothing, because there is nothing to pull
| out.
|
| ['adam', 'bill'].map(f) takes 'adam' out of the array, applies
| f to it, and puts it back in the array, then does the same with
| 'bill'.
|
| Just(5).map(f) takes 5 out of the Maybe, applies f to it, and
| puts it back in
|
| nothing.map(f) returns nothing.
|
| So we can map functions to values in Arrays, map functions to
| values in Maybes and even map functions to values in Eithers.
| Each time the function doesnt care that the value is in a Maybe
| or an Array or an Either.
| RyEgswuCsn wrote:
| Because of the mathematical "map" [0] perhaps?
|
| [0]: https://en.wikipedia.org/wiki/Map_(mathematics)
| zmmmmm wrote:
| I want a similar toolkit that lifts over the Ruby/Groovy/Kotlin
| style functions like countBy, groupBy, sortBy, collate, etc etc.
| Programming in Python is such a chore without these higher level
| collections APIs and for some reason (I guess, list
| comprehensions being considered idiomatic, bare bones itertools
| being considered "good enough", and the lack of real inline
| closures?) it doesn't seem like they'll ever be part of Python
| proper. But I still want them.
| wswope wrote:
| With full acknowledgment that it would be better to have at
| least some of those in the stdlib, Pandas covers all those
| needs pretty well in practice, with the bonus of workarounds
| that sidestep CPython's usual performance limitations.
___________________________________________________________________
(page generated 2022-02-07 23:01 UTC)