[HN Gopher] A retrospective on Requests
___________________________________________________________________
A retrospective on Requests
Author : zdw
Score : 96 points
Date : 2024-02-25 22:31 UTC (2 days ago)
(HTM) web link (blog.ian.stapletoncordas.co)
(TXT) w3m dump (blog.ian.stapletoncordas.co)
| dplarson wrote:
| For folks that don't recognize the author (Ian Stapleton
| Cordasco), he's one of the core maintainers for python-requests
| (sigmavirus24 on github).
| raziel2p wrote:
| Yikes. I always had a weird feeling about Requests looking from
| the outside, but it's worse than I thought.
|
| If you're looking for alternatives: https://www.python-httpx.org/
| zdw wrote:
| Or just the built-in urllib is pretty usable as-is at this
| point.
| argulane wrote:
| Plain urllib3 staring from v2 is also pretty nice to use.
| globular-toast wrote:
| I use and have contributed to httpx but I'm scared of moths and
| had to block the image in my browser. Looks like they updated
| the image to be an even more detailed one :(
| noelwelsh wrote:
| It's good to see an API critique. I don't think enough attention
| is paid to their design.
|
| The problematic design is something I see fairly frequently from
| people who aren't familiar with sum types, aka modelling data
| that has logical ors. It's very common in the Javascript world as
| well, for example, and in Go it's baked in with functions
| returning four possible values (success and error, success and
| nil, nil and error, nil and nil) when only two are valid.
|
| A better design uses a combinator libraries / builders / fluent
| API. These are little finite state machines and the best
| libraries make it so invalid transitions cannot be compiled
| (which requires a type system).
| lenkite wrote:
| They should just go ahead and make breaking changes after a big
| announcement so that everyone is clear that will no backward
| compatibility at the next major release. Otherwise the majority
| of design, development and mental energy will be spent in
| compatibility issues leading to full-burnout with little to no
| feature gain.
|
| You only need to look at C++ papers where superhumanly smart and
| driven IQ 200+ people desperately try with all their will to
| define a language or stdlib feature/improvement that preserves
| ABI backward compatibility and then finally give up and throw the
| towel after two dozen iterations and rejections. And also suffer
| from severe burnout in the process.
| EchoChamberMan wrote:
| Pretty sure Deno for Node has similar problems. Sometimes you
| just gotta let the past go.
| LunaSea wrote:
| Only if the delta of the improvements is big enough which
| isn't the case for Deno
| EchoChamberMan wrote:
| Sorry what? I'm saying Deno suffers due to Node backward
| compatibility. A write up about it:
|
| https://www.baldurbjarnason.com/2024/disillusioned-with-
| deno...
| zokier wrote:
| Or just make new project and let community deal with requests
| if they want.
| meitham wrote:
| I don't see a market need for a new one. Python's own stdlib
| urllib is pretty good for most cases, except async, and
| aiohttp is covering that gap.
| simonw wrote:
| HTTPX is the new one, and it's very popular. I've been
| using it in place of Requests for a few years now.
|
| https://pypistats.org/packages/httpx - 2 million downloads
| a day (requests has 15 million
| https://pypistats.org/packages/requests)
| simonw wrote:
| Breaking changes in requests would be a disaster.
|
| The problem is that they're not just a library people use
| directly: they're a library that is often used by other
| libraries.
|
| The worst possible form of dependency hell is when you want to
| use libraryA - which depends on requests<3.0 - and libraryB,
| which depends on requests>=3.0, at the same time.
|
| At that point, you're simply stuck. You cannot use both
| libraryA and libraryB at the same time until they both upgrade
| to the same underlying requests dependency.
|
| Python projects are affected by this because it isn't currently
| possible to have two versions of the same library installed at
| the same time, because they clash with each other in the
| sys.modules global namespace.
|
| This isn't true of all modern programming languages - I've been
| meaning to dig around and figure out which ones are affected by
| this problem and which are not - I think Node.js and Rust have
| mechanisms for this, but I don't think Go or Java do.
| mixmastamyk wrote:
| It can be handled "kludgingly" by incrementing the name. e.g.
| foo --> foo3
| edflsafoiewq wrote:
| Not sure why that's "kludgey".
| vvanpo wrote:
| > I don't think Go or Java do.
|
| Go's module system was specifically designed with this
| problem in mind: https://go.dev/blog/versioning-proposal
|
| I think Java projects get around this when they have to with
| shading, but that's a bit clunky.
| zokier wrote:
| I feel this is bit of a recurring theme in Python world where
| nice simple things are dragged into serious (enterprise)
| production world and then they accumulate gazillion bits of
| cruft, hiding the originally nice simple thing. It's quite
| fundamental problem trying to please both the ad-hoc small script
| and X million line production system users; Python straddling the
| gap imho ends up pleasing neither.
|
| My preference would be to focus on excelling on the small-scale
| stuff and leave the complex stuff for ecosystems better aligned
| with that; basically let Java be Java and Python be Python.
| game_the0ry wrote:
| > My preference would be to focus on excelling on the small-
| scale stuff and leave the complex stuff for ecosystems better
| aligned with that; basically let Java be Java and Python be
| Python.
|
| Agreed 100%. Strongly prefer python for the simple scripting
| and conceptually simple mental model, and others for more
| complex stuff.
|
| Also, don't use tools that were not designed for the job from
| the beginning and then complain that it does not fit your use
| case.
| math_dandy wrote:
| Don't you think that ship has sailed? Python powers software
| of incredible complexity these days including back-ends for
| large companies and all of AI (yes, I know the "hard parts"
| are written in C++, but still).
| game_the0ry wrote:
| > Don't you think that ship has sailed?
|
| No, those ships are setting sail with anchors down, and
| complaining why they're moving so slow.
| itsoktocry wrote:
| > _No, those ships are setting sail with anchors down,
| and complaining why they 're moving so slow._
|
| It's 2024 and you still think the choice of language in
| which you build your product is what makes or breaks a
| company?
| The_Colonel wrote:
| It's quite rare for companies to fail because of one bad
| decision, but I believe that choosing python for large
| complex projects will have long term negative
| consequences, irrespective of whether it will on its own
| kill the company or not.
| jerf wrote:
| It's one of the stereotypical failure cases of dynamically-
| typed languages. I cite this not to bury them in general, but
| because it's something that people programming in dynamic
| languages need to be aware of. If object oriented languages
| seem to frequently end up with "God Objects" dynamically-typed
| languages end up with "God Functions", where Just One More
| Parameter at a time, a core function grows slowly and steadily
| until it is a monstrosity of several dozen parameters and there
| is simply no conceivable way to document how they all interact
| with each other, test all the combinations, conceptualize all
| the combinations, etc. It's very dangerous because it happens
| often just one parameter, or even fractions of a parameter
| ("let's make this string also able to be an array of strings")
| at a time, each step seeming sensible at the time but the
| totality becoming a nightmare.
|
| Statically typed can incrementally end up with too many
| parameters too, but the static typing generally helps reduce
| the complexity and there is generally more incentives to start
| binding the parameters up into meaningful other structs.
| Dynamically typed languages on the other hand have the tendency
| to start widening what types each parameter can take which
| makes the problems even worse. Is "targetURL" a string, an
| array of strings that will automatically turn the return into
| an array of results, an object that implements ".toURL()", a
| function that will be automatically called with some magic
| parameters, a None which will then invoke other magic to get
| the URL from somewhere else, etc.?
|
| (Don't worry, dynamic typing fans, static languages have their
| own characteristic failures, like the aforementioned God
| Object.)
|
| Dynamically typed code bases are not doomed to end up this way.
| It can be avoided with discipline, which starts with the
| awareness that it is a problem at all. And it's a very good
| idea to learn to avoid them, because they're terribly difficult
| to tear back apart once constructed. I've worked in a code base
| that darned near had a "JustDoEverythingWeCanPossibleDo(...)"
| function, where (...) doesn't represent the true horror because
| there were rather a lot of global variables getting set that
| had huge impacts on the process as well. Trying to pull that
| apart to do any real work was, ah, an experience.
| int_19h wrote:
| I would argue that Python itself is a case of this. It peaked
| early as an ultra-dynamic language, and was very consistent at
| it - e.g. all declarations are executable code, which is
| actually fairly straightforward conceptually and easy to
| explain - but this approach doesn't mesh well with things like
| class declarations and imports, which turns out you need for
| more serious development.
|
| More recently, this is the case with typed Python. Again, the
| dynamic-ness of Python - which easily exceeds that of JS - was
| part of the original design for type annotations. Turns out it
| doesn't work well with things like forward references or type
| parameters, so ugly hacks (like stringifying type names) were
| introduced to deal with that. Now there's yet another revamp to
| fix the resulting ugliness and inconsistencies (take a look at
| https://peps.python.org/pep-0649/ to see what I mean).
| blibble wrote:
| > I gave it but realized that most people only know that Requests
| does the right things for them for TLS
|
| as long as you don't work somewhere that has its own internal CA
| infrastructure
|
| requests decides to use its own built-in CA store instead of the
| carefully tweaked OpenSSL default (the location of which you may
| not know declaratively)
|
| meanwhile urllib works fine
|
| (yes I read further down where this point is sort of mentioned)
___________________________________________________________________
(page generated 2024-02-27 23:01 UTC)