[HN Gopher] Functional Python Programming
___________________________________________________________________
Functional Python Programming
Author : meatjuice
Score : 85 points
Date : 2023-06-09 12:40 UTC (10 hours ago)
(HTM) web link (docs.python.org)
(TXT) w3m dump (docs.python.org)
| tandav wrote:
| shameless plug: I maintain a small library to do functional
| pipes.
|
| You can write: ( range(10)
| | Map(lambda x: x * 10) | Filter(lambda x: x % 2 ==
| 0) | Reduce(lambda a, b: a + b) )
|
| instead of: x = range(10) x =
| map(lambda x: x * 10, x) x = filter(lambda x: x % 2 == 0,
| x) x = reduce(lambda a, b: a + b, x)
|
| and more. https://tandav.github.io/pipe21/
| etra0 wrote:
| This looks cool! I always wanted something similar for pandas,
| because I found it very elegant in dplyr in R. AFAIK they do
| have a `.pipe` method now but it could definitely be better in
| the future.
| sampo wrote:
| import pandas as pd import functools (
| pd.Series(range(10)) .apply(lambda x: x * 10)
| .where(lambda x: x % 2 == 0) .pipe(lambda s:
| functools.reduce(lambda x, y : x + y, s)) )
| melx wrote:
| Readability isn't the best. Also what you present here is
| method chaining and not functional pipes.
| heyoni wrote:
| Is the difference mostly syntactic?
| gnulinux wrote:
| What difference does it make? They're conceptually the same
| thing. You're mapping immutable data to map/reduce/filter
| like pure functions to get new data.
| melx wrote:
| Improved code readability. The pipe operator was a game
| changer for me even after 20 years of programming (I know
| I was probably touching wrong things).
| gnulinux wrote:
| Hmm, I've never seen code like the latter so it's unclear what
| problem this solves. I'd just write the pandas code snippet
| given below. Possibly with polars to make it lazy.
| tandav wrote:
| My library solves 2 problems.
|
| 1. It does not require to wrap your iterable into some
| wrapper to use functional methods. It takes an
| iterable/object and returns another iterable/object. You
| don't have to unwrap it after transformations.
|
| 2. it uses oneliners (library is 80LOC single file) for most
| of the methods. You can just copy-paste it to use instead of
| install and import. E.g map, filter, reduce is just:
| class B: def __init__(self, f): self.f = f
| class Pipe (B): __ror__ = lambda self, x: self.f(x)
| class Map (B): __ror__ = lambda self, x: map (self.f, x)
| class Filter(B): __ror__ = lambda self, x: filter(self.f, x)
| class Reduce(B): __ror__ = lambda self, it:
| functools.reduce(self.f, it, *self.args)
| mtreis86 wrote:
| It is a fairly common pattern in low level stuff like hashing
| algorithms or cryptography.
| haspok wrote:
| What blows my mind is that Python was designed for maths and data
| science by a maths and data science guy. It should be natural to
| work with immutable values and pipelines in Python, which are
| natural functional constructs. Yet everything in Python is
| mutable and pipeline-style programming is not really supported.
| ???
|
| (A long time ago I tried to teach programming fundamentals to a
| maths person. She was completely puzzled by x=x+1. X equals to
| X+1? That was nonsense in their eyes.)
| channel_t wrote:
| This is false. It was mostly designed as a tool for Unix
| hackers, with motivations that seem not too dissimilar from
| perl. It was the ease of use of the language that brought
| people in from the math/science community who usually weren't
| programmers. This has often historically led to terrible and
| non-idiomatic (but working) code, because if you have an idea
| in your head you can quickly hit the ground running.
| tgv wrote:
| > Python was designed for maths and data science
|
| I don't think so.
|
| > During Van Rossum's stay at CNRI, he launched the Computer
| Programming for Everybody (CP4E) initiative, intending to make
| programming more accessible to more people, with a basic
| "literacy" in programming languages, similar to the basic
| English literacy and mathematics skills required by most
| employers. Python served a central role in this
| constantcrying wrote:
| Mathematics is an extremely broad field, the mathematics of a
| numerical analyst and someone doing algebraic geometry are very
| different and a programming language for each would look quite
| different.
|
| In python the C and even C++ heritage is quite clear, but it is
| in no way designed like Miranda and Haskell, which actually
| target to some extent expressing particular mathematical
| constructs in code, something which neither C, C++ not python
| ever aspired to.
| mikebenfield wrote:
| > What blows my mind is that Python was designed for maths and
| data science by a maths and data science guy.
|
| Neither of these is true. I actually still find it a little
| weird that Python became so widely adopted by data science
| people, and that definitely doesn't seem to be the main user
| GvR originally had in mind when creating the language.
| lindbergh wrote:
| In their definition of programming languages types, I'm not sure
| I see how SQL which they cateogrize as declarative really differs
| from FP, especially with lazy evaluation. Both are a succession
| of functions applied onto a previously declared variable : just
| think of a long SQL query with many ctes modifying the previous
| ones; each of them can be thought of as a variable which takes
| its value from the application of a function on other variable (a
| bit like the let function in ocaml).
|
| I always thought of LaTeX as being the prime example of
| declarative programming.
| peterkelly wrote:
| SQL does not support higher-order functions or lambdas (except
| possibly in some special dialect I'm not aware of)
| lindbergh wrote:
| Right, good point. I just find the structure and the mindset
| to be similar.
| paolosimone wrote:
| I tried to preach functional programming style to my data
| analysts colleagues once... it ended poorly, but it was fun!
|
| If you want to smile, here is the presentation I made (warn: old
| memes ahead)
|
| https://paolosimone.github.io/python-is-functional
| gnufx wrote:
| Iterators are a fundamentally non-functional construct. Long ago
| Henry Baker wrote "Signs of Weakness in Object-Oriented
| Languages": https://plover.com/~mjd/misc/hbaker-
| archive/Iterator.html
| heyoni wrote:
| You just broke my brain. I will read it though, just give me 6
| months!
| raphaelj wrote:
| Sadly, Python is a pretty poor functional language.
|
| The core of functional programming is about _avoiding mutable
| states_ , not much about anonymous functions or passing functions
| as data.
|
| To do proper functional programming in Python, there should be
| IMO:
|
| - a way to enforce non-mutable variables/objects;
|
| - non-mutable collections;
|
| - proper support for recursion and tail-recursion optimization;
|
| - a better syntax for anonymous functions than single statement
| _lambda_ s.
|
| Generators are nice, but do not have much to do with functional
| programming.
| linkdd wrote:
| For tail recursion, you can use this snippet of code:
| class Recurse(Exception): def __init__(self, *args,
| **kwargs): self.args = args
| self.kwargs = kwargs class Terminate(Exception):
| def __init__(self, retval): self.retval = retval
| def tailrec(func): def wrapper(*args, **kwargs):
| while True: try:
| func(*args, **kwargs) except Recurse as
| r: args = r.args
| kwargs = r.kwargs except Terminate as t:
| return t.retval return wrapper
| @tailrec def fact(n, acc=1): if n == 0:
| raise Terminate(acc) else: raise
| Recurse(n - 1, acc * n)
|
| Of course, it will be slow because it relies on exceptions :P
| _aavaa_ wrote:
| You can do a similar thing (that's around 2-3x faster) with a
| trampoline: def factorial(n): res =
| fac(n) while callable(res): res =
| res() return res def fac(n, acc=1):
| if n == 1: return acc else:
| return lambda: fac(n-1, n*acc)
| [deleted]
| tayo42 wrote:
| Why do you consider anonymous functions so important? Regular
| functions can be used the same way?
| sodapopcan wrote:
| OP said anonymous functions are NOT as important as immutable
| data.
| tayo42 wrote:
| its in the list they provided of improvements python
| needs...
| profunctor wrote:
| It's not a list of improvements python needs. It's a list
| of things python already does / has.
| tayo42 wrote:
| do I have bots responding to me? whats hard about
| comprehending this?
|
| > To do proper functional programming in Python, there
| should be IMO:
|
| > ...
|
| > - a better syntax for anonymous functions than single
| statement lambdas.
| [deleted]
| sodapopcan wrote:
| Oh... either that was added in an edit or I had a brain
| blip that caused me to skip the bulleted list. Apologies.
| carapace wrote:
| Syntactically maybe, but I find it has a quite workable
| functional subset.
|
| Integers, floats, tuples, named tuples, and frozensets are all
| immutable, functions are values, etc. E.g.:
| https://joypy.osdn.io/notebooks/Derivatives_of_Regular_Expre...
| -or-
| https://github.com/calroc/xerblin/blob/master/xerblin/btree....
|
| It's not fantastic, but it's not that bad.
| gnulinux wrote:
| I love both python and functional programming and I _do_
| write functional style python routinely. One barrier I hit is
| there is no immutable dict type. There is MappingProxyType
| but it 's insufficient. I prefer using custom dataclass like
| objects built with pydantic and type checked with mypy. It's
| better than dict and can be made immutable but requires tons
| of boilerplate code which is a bit unpythonic.
| carapace wrote:
| > there is no immutable dict type
|
| Does the namedtuple not suffice? Apologies if I'm being
| dense.
|
| - - - -
|
| It's not an immutable dict type (for one thing, this has
| linear lookup, the BTree would be better) but it's fun:
|
| https://stackoverflow.com/questions/13708701/how-to-
| implemen...
|
| In Python: from functools import partial
| def empty_dict(key): raise KeyError
| def _dict_add(dictionary, key, value, lookup):
| return value if key == lookup else dictionary(lookup)
| def dict_add(d, key, value): return
| partial(_dict_add, d, key, value) d =
| empty_dict d = dict_add(d, 'key0', 23) d =
| dict_add(d, 'key1', 18)
|
| And then... >>> d('key0') 23
| >>> d('key1') 18 >>> d('keyn')
| Traceback (most recent call last): File
| "<pyshell#8>", line 1, in <module> d('keyn')
| File "/usr/home/sforman/tmp_fn_dict.py", line 7, in
| _dict_add return value if key == lookup else
| dictionary(lookup) File
| "/usr/home/sforman/tmp_fn_dict.py", line 7, in _dict_add
| return value if key == lookup else dictionary(lookup)
| File "/usr/home/sforman/tmp_fn_dict.py", line 4, in
| empty_dict raise KeyError KeyError
|
| - - - -
|
| > objects built with pydantic and type checked with mypy
|
| FWIW, after ~15 years of professional Python development,
| I'm not into types [in Python]. My attitude is that if you
| really need them you should switch to e.g. OCaml or
| something that does them right. I love Python but I
| wouldn't use it anymore for anything other than scripts and
| maybe prototyping.
| odyssey7 wrote:
| Agreed, Python is designed for something other than functional
| programming, so a functional programming enthusiast would get
| better mileage out of something else. Even if you were to write
| all of your Python programs against the grain in a functional
| style, you'd still need to operate in a community of modules
| that don't provide referential transparency.
| theLiminator wrote:
| Even worse when some modules have global mutable state like
| matplotlib and co.
| gnufx wrote:
| You want tail call optimization, not just tail-recursion, i.e.
| mutually recursive functions.
| pjmlp wrote:
| Not for Lisp or some ML derived languages, FP isn't whatever
| Haskell does.
| qbasic_forever wrote:
| I agree but it's not so dire, the tuple type (and variants like
| namedtuple) is pretty powerful, non-mutable and often my first
| choice for data structure in python. Everything else can be
| built on top of it if you're really motivated.
| amanj41 wrote:
| I would add frozenset as the other immutable collection. The
| other points are well taken though
| xwowsersx wrote:
| Fun to work on doing FP in languages that don't really support
| it, but in my view a language has to be built for FP for it to be
| a practical option in any real applications. Several obvious
| reasons for Python being a poor lang in which to do FP:
| - mutable data structures - no built-in function
| composition - limited support for HOF - no tail call
| optimization (AFAIK) - performance in general isn't great
| and I imagine it's even worse when relying heavily on recursion,
| etc
|
| Still, a fun project!
| pjmlp wrote:
| Scheme is one of the few FP languages with required TCO on the
| language standard, and alongside Lisp, Caml Light, Standard ML,
| OCaml, F#, Scala, supports mutable data structures.
| jgilias wrote:
| It doesn't have to be all or nothing though. I've definitely
| seen (and written) functional Python in 'real applications'
| sprinkled here and there in an otherwise normal codebase.
|
| In fact, I'd wager that any Python programmer worth their salt
| would suggest using a list comprehension and a lambda over a
| loop in most data transformation situations. Also currying
| comes in handy often once you have it in your toolkit.
|
| It's a bit silly to call two very mature libraries that are
| part of the Python standard library a 'fun project' just
| because the language itself supports multiple programming
| paradigms.
| Jeff_Brown wrote:
| What's so useful about iterators and generators. The article says
| how to use them but not why. If you already know how to make list
| comprehensions and use "for elt in coll", do they let you do
| anything new?
| jejones3141 wrote:
| It's the difference between evaluating all the elements of a
| list, consing them, and returning it all at once and evaluating
| the elements one at a time (i.e. lazy evaluation). Saves memory
| and time and it gives you a chance to quit early.
| crabbone wrote:
| You could probably find more uses, but here are some:
|
| Generators allow one to easily transform code from non-buffered
| to buffered. Consider this example: for y in
| x: do(z)
|
| Now, x may be a collection that was eagerly evaluated in
| previous steps, let's say a list, but then you've discovered
| that this list is too big to fit in memory, and you want to
| generate it in manageable fixed-size chunks, so you replace the
| code that created x to make x a generator. You don't have to
| touch the code above -- it will work the same way because
| generators have the same interface as collections.
|
| Another use: generators are used to implement async / await. I
| personally find this idea ridiculously stupid, but a lot of
| people (and especially those who don't understand what it does)
| like it a lot. Yielding mechanism, which is a feature of
| generators, is the one that's used to communicate / switch
| between co-routines (tasks) of asyncio.
|
| Another aspect, besides bufferization is that you might want to
| delegate control over how much looping you want to do to a
| separate chunk of code. I.e. you may want to separate the
| generation of elements (hence, generator) from eg. filtering
| them, or transforming them in some way, or reducing them etc.
| If you didn't have this ability, you'd have to generate the
| entire collection upfront, and if your computation takes
| multiple steps that you'd like to separate into different code
| chunks, you'd have to also generate intermediate collections,
| even though, potentially, you don't need some of the elements
| in those collections. Consider, for example:
| def powers(start, end, power): return (x ** power
| for x in range(start, end)) def flt(a, b, c):
| return a + b == c def disprove_flt(upto,
| upto_power): for power in range(upto_power):
| for a, b, c in zip(powers(1, upto - 2, power), powers(2, upto -
| 1, power), powers(3, upto, power)): if
| flt(a, b, c): return a, b, c
| return None, None, None
|
| Which is, of course not a correct way to search for the
| counterexample to Fermat's last theorem, but I tried to find a
| popular enough subject so that the example was easier to
| follow.
|
| An exercise to the reader: rewrite the code above in such a way
| as to eliminate "upto" and "upto_power", i.e. to search until a
| counterexample is found (or indefinitely).
| xapata wrote:
| Fundamentally? No, it's all just machine language in the end.
|
| Practically? Yes, generators are often more memory-efficient,
| and thereby often more compute-efficient.
| Jeff_Brown wrote:
| Is it only about efficiency, and not expressivity? If so, can
| the compiler figure out where to transform my equivalent
| structures into iterators and generators?
| stayfrosty420 wrote:
| @dataclass is the new final
| _dain_ wrote:
| Does the type system let you express that a class shouldn't be
| subclassed? I remember this possibility was mentioned in a PEP
| and got deferred. It would be really useful with the new
| match/case pattern matching feature, because then you could
| have proper sum types, and mypy could enforce exhaustiveness.
| AFAIK you have to do a workaround with a "assert False" or
| similar at the end.
| hfhdjdks wrote:
| It's not what you were asking for (class that can't be
| subclassed), but `typing` has an `assert_never` to check
| exhaustiveness:
|
| https://typing.readthedocs.io/en/latest/source/unreachable.h.
| ..
| nerdponx wrote:
| Thanks for this trick for exhaustiveness checking, I've
| just been putting assert False, "This is
| unreachable."
|
| in my code until now.
| nerdponx wrote:
| Yes, that's what @typing.final is:
| https://docs.python.org/3/library/typing.html#typing.final
|
| But that's only checked by the type checker. At runtime you
| still need to do something like this to prevent subclassing:
| class DontSubclassMe: def __init_subclass__(self):
| raise TypeError("Don't subclass me!")
| harveywi wrote:
| Shameless plug for Scala programmers who are marooned in Python
| land and grasping for something to help dig them out:
| https://github.com/miiohio/ziopy
| sammy_rulez wrote:
| For those who are interest I maintain a small open source library
| to use type safe monads in python
| https://github.com/sammyrulez/typed-monads
| FrustratedMonky wrote:
| How new is this? Seems like there is more functional tools in
| Python than I remember. Didn't see the new 'match' statement in
| this doc.
| rirze wrote:
| Is the match-case statement a functional concept? I see it in
| functional languages but I'm not sure it is inherently
| functional...
| agumonkey wrote:
| functional I can't say, but match was highly related to
| structural typing which was very much central in FP/LP
| culture
| FrustratedMonky wrote:
| Think this is answer. It isn't technically needed to be
| called 'functional', but it is so integral to type systems
| that it seems to be used in all functional languages. Maybe
| we can call 'match' statements an emergent phenomena of
| functional languages.
| agumonkey wrote:
| and i'm sure the idea floated around in non FP languages
| too, but
|
| 1) mutable programming shifts the idioms very very far
| away from passive destructuring, they like to deref
| pointers and reuse memory cells
|
| 2) it's also linked to parametric type systems I believe,
| which was missing until cpp/java5 in the mainstream
| (while FP had this since milner which was in the 70s)
| vintagedave wrote:
| Thanks for posting this. Many years ago (2006?) I was a just-out-
| of-university programmer and a friend of a friend who wrote
| Python for Canonical sat me down and tried to teach me functional
| programming using Python. I was a poor learner: I didn't 'get it'
| (functional programming in general) and, to my regret, learned
| nothing.
|
| Much time has passed and I hope I'm a more aware and open-to-new-
| things person today :) I use Python these days though in a
| traditional procedural / OO way. This post reminded me that maybe
| I should look into writing it functional-style. Thankyou!
| FrustratedMonky wrote:
| I find that learning functional style in a language that does
| not enforce it, is extra difficult. If the language does not
| provide guard rails, then new learners will fall back on non-
| functional methods, and not even realize it.
| qbasic_forever wrote:
| IMHO broaden your boundaries a bit and learn a lisp or clojure-
| based functional style, like grab a getting started in clojure
| book and work through it. You will learn a lot more than this
| guide goes into, it's really just looking at functions in a
| functional style but doesn't spend nearly enough on data
| structures and state which are really the core of functional
| programming (and what python doesn't do well out of the box for
| functional programming).
|
| Learning a bit of clojure will really open your eyes to what
| functional programming means, and you can take some of those
| learnings back to python.
| Mizza wrote:
| I thought I knew functional programming from knowing Python, but
| looking back I didn't really "grok" it until I moved to Elixir.
| Now I like it and prefer it. I don't run into any of the types of
| bugs I used to create in Python, mostly from being lazy and
| fiddling with data inside of a loop. I miss do miss early returns
| though.
| macintux wrote:
| I moved from Erlang to Python and was miserable. I still write
| most of my code in a functional style, but I dearly miss using
| a language that enforced those constraints.
| crabbone wrote:
| Worthless functionality in a worthless language. Only topped by
| the bizarre and unverified claims in its documentation:
| Most programming languages are procedural:
|
| Did the author count? -- The answer is a clear and resounding
| "No". The author pulled this factoid out if his rear end. Maybe.
| Maybe not. It's not even clear by what's meant by "all languages"
| -- all possible languages? all languages known to author? all
| languages used in Github? And why should anyone care about this
| kind of multitude? Lisp
|
| Again, people who've never seen any Lisp, or vaguely remember
| their college days when some course requested from them to write
| a function to figure out if a string is a palindrome in Scheme
| think that Lisp is a single language. The author just decided to
| demonstrate his blistering ignorance by including something that
| he thought would render him as more experienced than their
| readers.
|
| Later the author perpetuates all sorts of absurd myths about
| "functional programming" s.a. increased modularity or ease of
| debugging. Apparently, author had never used step-debugger nor
| had he wrote anything in any popular programming language that
| advertises itself as functional to experience first-hand this
| "ease" he's talking about. Needless to say that nothing in
| functional programming prevents programmers from writing long
| functions... In practice, however, some languages which advertise
| themselves as "functional" have pathologically bad / hard to read
| syntax (eg. Haskell), and functions longer than some 10 lines or
| so become too difficult to understand even to people who believe
| themselves to be proficient in those languages.
|
| Author is simply lying when he claims that generators are a kind
| of function, which can be simply verified: >>>
| def generate_ints(n): ... for i in range(n):
| ... yield i ... >>> type(generate_ints(1))
| <class 'generator'> >>> type(generate_ints(1)).mro()
| [<class 'generator'>, <class 'object'>] >>>
| isinstance(generate_ints(1), type(generate_ints)) False
|
| Generators and functions are unrelated. Neither is a kind of
| other.
|
| On top of that, author frequently violates Python's coding
| conventions (eg. capital letters in variable names, assigning
| lambdas to variables).
|
| ---
|
| But, bottom line: crappy language deserves no better
| documentation than this.
| eska wrote:
| Have a beer, bro
| amne wrote:
| can you make anonymous functions with more than 1 expression now?
| hsfzxjy wrote:
| Shameless plug. I've once made a library for this
| https://github.com/hsfzxjy/lambdex
| PurpleRamen wrote:
| Just curious, but from a pragmatic view, what is the advantage
| of using anonymous functions instead of named functions for any
| mildly complex code?
| raphaelj wrote:
| It's pretty common to define "higher-order" functions in
| functional programming. These will sometimes be applied to
| code blocks.
|
| For example, Python's _with_ statement could have been
| entirely programmed as a library function if lambas supported
| code blocks: with(open("file.txt"), lambda
| f: content = f.read() return
| sum(int(c) for c in content.split(" ")) )
|
| Functional languages usually have a pretty limited "core" and
| implement most of their features through their standard
| library, usually using higher-order functions, overloaded
| operators ...
| PurpleRamen wrote:
| But I can do all that with python already? Your example
| would work this way too: def _(f):
| content = f.read() return sum(int(c) for c in
| content.split(" ")) with(open("file.txt"), _)
| sodapopcan wrote:
| I don't think they were saying you can't do it in Python,
| just offering what is arguably a nicer syntax, but of
| course that is subjective. The anonymous function version
| lets you write things in the order they happen and also
| "binds" the operations together, ie, someone can't
| accidentally add a completely unrelated line between the
| `def` and the `with`. It ultimately doesn't matter,
| people just get used to the conventions of their
| preferred languages and we all trudge on.
| phailhaus wrote:
| Readability and scoping are two big advantages. Anonymous
| functions are often short and sweet, but also highly specific
| to a single section of the code. It's not meant to be reused.
| Having to name that one-off use case and hoist it into a
| function elsewhere makes it harder to maintain and
| understand. With proper anonymous functions, you can just
| read through it in one go.
| PurpleRamen wrote:
| But you don't need to name it big or maintain it somewhere
| else? Just use a throwaway name and write it where you use
| it. I mean python even allows overriding functions with the
| same name in the same scope. You could literally go with
| using just the same name everywhere.
| biorach wrote:
| True, but you're missing the point.
| PurpleRamen wrote:
| Then what is the point?
| phailhaus wrote:
| Readability. If you're using the same name everywhere,
| then your code is really hard to understand for other
| devs: val = my_list.reduce(foobar)
|
| What is even going on here?
| cmcconomy wrote:
| one option is to define your function inline, directly
| above the line where you would have liked to write the
| complex anonymous function. I guess if you are worried
| about namespace pollution you can 'del' the symbol
| afterward.
___________________________________________________________________
(page generated 2023-06-09 23:01 UTC)