[HN Gopher] Python HTTP library 'urllib3' now works in the browser
___________________________________________________________________
Python HTTP library 'urllib3' now works in the browser
Author : SethMLarson
Score : 72 points
Date : 2024-01-30 17:05 UTC (5 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| canadiantim wrote:
| Pyodide examples using urllib3 in browser still requires using
| fetch, so this still requires javascript.
|
| What does this practically mean?
|
| Does this ever point to a future where we can use urlib3 instead
| of fetch?
| SethMLarson wrote:
| Lead maintainer of urllib3 here: We're tracking all the options
| for how we can make the experience better for folks looking to
| use Python in the browser. Today our biggest blocker is that
| there's no socket or TLS APIs for Emscripten/WASI. If those
| were to become available we'd jump on them right away :)
|
| I think the biggest benefit of a foundational project like
| urllib3 getting browser support is that /many/ Python projects
| are built on top of urllib3 and many of them likely work in
| browsers now that urllib3 works in the browser.
|
| For example, Requests is built on top of urllib3 and now gets
| browser support for free! And this is recursive, how many
| projects use Requests (spoiler: many). Pretty exciting IMO :)
| canadiantim wrote:
| wow that's awesome. That we could even talk about potentially
| using just python to make requests in browser is pretty wild
| (of course, as you say still blocks in the way - lack of
| socket or TLS APIs for Emscripten/WASI). Really cool stuff
| tho. All the best eh and thank you VERY much for the
| incredible urllib3 library.
| SushiHippie wrote:
| Do you know if there are some plans for sockets or TLS APIs
| or are we far away from this?
| SethMLarson wrote:
| Browsers limit the ability for these platforms to use raw
| sockets, there simply is no API for it. The best that can
| be done /today/ is to use WebSockets, which are not the
| same thing and can't be used for HTTP requests without the
| server expecting a WebSocket connection:
|
| https://github.com/emscripten-
| core/emscripten/issues/5196#is...
| SushiHippie wrote:
| Sure, sorry if it wasn't clear from my question, it
| wasn't specifically about urllib3, but I meant if there
| are plans that browsers will make such an API available
| or if this is something that will never become real.
| sillysaurusx wrote:
| I doubt browsers would allow this, though gamedevs have
| wanted it for decades now. Particularly UDP.
| plopz wrote:
| isn't the udp problem fixed with http3 unreliable
| webtransport?
| sillysaurusx wrote:
| Is it? That would be cool. Under the hood is it actual
| udp?
| SushiHippie wrote:
| HTTP3 uses QUIC, it's not "pure" UDP, but it's a protocol
| based on it
|
| https://en.wikipedia.org/wiki/QUIC
| at_a_remove wrote:
| Thank you for your work on urllib3; I will probably never use
| Requests if I can help it. I'm glad that you're keeping up
| that "batteries included" tradition.
| mistrial9 wrote:
| except the browser context is being sandboxed with lots of
| access restriction, individual profiling, geolocation
| reporting, device identification, and more
|
| Big Tech does not like independent protocol operation it
| seems; therefore your interesting technical advance is likely
| to be a pawn in other, larger changes to the social contracts
| on the open net IMHO
| msoad wrote:
| Am I misunderstanding this? You need JS to load the WASM module
| anyways, what does "requiring JS" means here?
| ploxiln wrote:
| It means that urllib3 (python code) isn't actually making the
| http request (it's not making the tcp connection, tls auth,
| writing the request, or serializing the request headers ...)
|
| It may be better to describe this as a "shim". Any python
| code that depended on urllib3 is now transparently shimmed in
| browser context to use the fetch-api instead. Which is
| useful.
| zarzavat wrote:
| This is definitely out of scope for web browsers. The
| security implications of allowing arbitrary sockets are
| prohibitive.
|
| To do something like that then you're going to need a
| different environment with a different set of security
| guarantees and expectations, more like a shell where the
| user is responsible for vouching for the code's
| trustworthiness.
| canadiantim wrote:
| You're right. It just reflected a momentary misunderstanding
| of mine. WASM still requires JS.
| hpeter wrote:
| why would you want to run python in the browser? Anyone can ELI5?
| rmnclmnt wrote:
| Lots of use-cases but to name a few:
|
| - Interactive documentation (static website)
|
| - Jupyter Notebooks without a server, great for education
| (Jupyter-Lite)
|
| - Python app alternative deployment as static webapp
| cle wrote:
| There's a huge volume of existing Python code.
| dharmab wrote:
| Python in Excel currently requires a separate cloud server to
| run Python formulas. Imagine this could run in the Excel web
| view instead.
| pphysch wrote:
| For real demos of your Python libs: https://pygments.org/demo/
| Austizzle wrote:
| The film (especially vfx) industry runs on Python, and so much
| code is written for Python.
|
| Same reason you'd want to run your server in node.js so it
| matches your front end code ( and you can reuse code on the
| front end and backend). Pyscript lets us have the same
| programming language in the front end and the backend, and
| access to all of our internal packages on the front end.
|
| A lot of vfx software is written in QT, so at a previous
| company I actually made a wrapper around a subset of the QT
| interface and widgets, and replicated it in Pyscript, so we
| could write an app once and run it from inside vfx
| applications, or run it in the browser with no additional
| coding. It wasn't perfect, but it worked and made things
| simpler
| Extigy wrote:
| Since this is using fetch/XHR under the hood, I guess requests
| from the browser are restricted only to same-origin URLs or
| servers responding with permissive CORS headers?
| SethMLarson wrote:
| Correct! The examples we used were making requests to
| httpbin.org, but I also was able to query GitHub's API :)
|
| `data = urllib3.request("GET",
| "https://api.github.com").json()`
| Austizzle wrote:
| Incredible! Two years ago I was experimenting in porting some
| code at the company I was at to python in the browser (instead of
| QT desktop apps) and this was the biggest thing I had to work
| around! I had to manually monkey patch or proxy a few libraries
| to make them work, but I'll have to test out this update and see
| if everything magically works
| SethMLarson wrote:
| Oh wow, thanks for this story! Would love to hear more if you
| have time :) Good luck with testing it out.
|
| Note that we found an issue w/ emitting an
| InsecureRequestWarning by default. The request is perfectly
| secure, it's just we aren't telling the ConnectionPool that
| information (see:
| https://github.com/urllib3/urllib3/issues/3331)
___________________________________________________________________
(page generated 2024-01-30 23:01 UTC)