[HN Gopher] Writing a chat application in Django 4.2 using async...
___________________________________________________________________
Writing a chat application in Django 4.2 using async
StreamingHttpResponse
Author : ipmb
Score : 75 points
Date : 2023-06-16 16:28 UTC (6 hours ago)
(HTM) web link (valberg.dk)
(TXT) w3m dump (valberg.dk)
| Thaxll wrote:
| Do modern chat actually use websocket and the like? Discord /
| Slack on the web ( browser ) what do they use?
| vorticalbox wrote:
| The slack_bolt library has support for web sockets.
|
| https://api.slack.com/apis/connections/socket
| pmontra wrote:
| This implementation is probably a little different but we were
| using long poll in the late 90s and early 2000s. The problem was
| that you were committing one thread (or worse, one process) to
| each client. That obviously doesn't scale unless threads are
| extremely light on RAM and either the OS or the runtime support a
| large number of them. I remember that a way out was using
| continuations. Jetty was a Java application server that supported
| them (random link [1]) One thread -> many connections. I didn't
| investigate how Django is implementing this now but CPUs and RAM
| are still CPUs and RAM.
|
| [1] https://stackoverflow.com/questions/10587660/how-does-
| jetty-...
| klabb3 wrote:
| > The problem was that you were committing one thread (or
| worse, one process) to each client.
|
| This is mostly true today as well, although we do have beefier
| machines and more efficient thread pooling.
|
| I did some personal research for infrequent real-time
| notifications. My conclusion was that many stateful connections
| are poorly memory-optimized by language runtimes and reverse
| proxies. Even with lightweight tech like Golang, gRPC, nginx,
| etc, it's hard to push anything less than 30 kB for an idle
| conn, mostly from thread/goroutine stacks, user space buffers
| and (easy to overlook) a bunch of HTTP headers and TLS
| handshake state that often remain after establishing the conn.
| That's without any middleware or business logic wants their
| share of the cake.
|
| The only mature project I found that really took this stuff
| seriously is uWebSockets. It's extremely lightweight, around an
| OOM better than most alternatives. Highly recommend - they also
| have a socket library so you can implement other protocols if
| needed.
|
| Anyway, it's important to be aware that massive amounts of long
| running connections is not just about adding a websocket/SSE
| library. Chances are you'll need a mostly-separate serving
| stack for that purpose.
| kiraaa wrote:
| https://github.com/sysid/sse-starlette makes token streaming so
| much easier in python
| [deleted]
| Ralfp wrote:
| Just a heads up that currently Django is not cleaning up open
| PostgreSQL connections when ran in ASGI mode, leading to too many
| open connections error:
| https://code.djangoproject.com/ticket/33497
| pdhborges wrote:
| I'll have zero trust running async Django until all
| sync_to_async and async_to_sync calls are removed from the code
| base.
| lisasays wrote:
| Care to elaborate?
| pdhborges wrote:
| Look at the intended semantics [1], and then read the
| implementation [2]. Can you figure out if the
| implementation is correct? Can you infer the possible
| limitations of the approach at glance? Can your async
| library actually handle being called with multiple event
| loops installed?
|
| I have zero trust in this code and I have been bitten by
| fixes to this library that introduced deadlocks in my own
| code.
|
| [1] https://github.com/django/asgiref#synchronous-code--
| threads.
|
| [2] https://github.com/django/asgiref/blob/main/asgiref/syn
| c.py#...
| lisasays wrote:
| Doesn't look like something I'd want to have to debug in
| a pinch, that's for sure.
|
| Is there anything else in Python-land you would go with
| for the use case you have in mind? Or does pretty much
| all the async stuff turn on you like this, at some point
| or another?
| [deleted]
| tomwojcik wrote:
| Same here, but without these weird utils it doesn't get any
| better.
|
| I have 7 YoE with Django. Its great at so many things. You
| see some code, like middlewares, and immediately understand
| what's going on.
|
| Now, we also have Starlette. The base of all new, fancy asgi
| libraries. Here's the base middleware class.
|
| https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb.
| ..
|
| In the last couple of years I heard 'we're running fastapi on
| production. Wanna join us?' so many times... but the reality
| is that it's still not suitable for prod. Who wants to work
| with a code like that if you have a readable, stable Django?
| I'm clueless.
| baq wrote:
| Running Postgres without a connection bouncer is a huuuge no-no
| already, but if this breaks the default mode of pgbouncer it's
| basically a showstopper.
| winrid wrote:
| Your average django app running on one or two servers with
| four workers each (which use connection polling) does not
| need pgbouncer.
| etimberg wrote:
| Yeah, I've definitely seen odd things in Django ASGI. Had at
| least one instance where it looked like it leaked data between
| requests but couldn't get a clear enough reproduce. Switching
| back to WSGI worked fine though
| kolanos wrote:
| Reading that bug report it looks like Django hasn't been
| isolated as the source of the problem? Looks like Carlton
| Gibson closed it until it is reproducible. Problem could be in
| uvicorn, psycopg2, etc
| whalesalad wrote:
| that's kind of a dealbreaker
| thefreeman wrote:
| could this be addressed by something like pgbouncer or
| another connection pooler?
| verandaguy wrote:
| Going off the pooling mode feature map[0], it might be
| possible to mitigate that issue using transaction pooling,
| but not session pooling -- which means a restricted feature
| set and cooperation from the application (which would just
| not call restricted features up). [0]
| http://www.pgbouncer.org/features.html#fn:2
| whalesalad wrote:
| Perhaps yes, would be worthwhile to benchmark an internal
| (to your runtime) conn pooler versus external only.
|
| I'm using internal pools + pgbouncer but as I write this
| out I am beginning to wonder if that is even necessary.
| samwillis wrote:
| All the new asyncIO stuff in Django is awesome, they are doing a
| phenomenal job retrofitting it all to an inherently sync designed
| framework.
|
| One thing to note, it's been possible to build these sort of
| things with Django by using Gevent and Gunicorn for over a
| decade, and it works well.
|
| In many ways I wish Gevent had been adopted by Python rather than
| AsyncIO.
| btown wrote:
| Can second the Django/Gevent/Gunicorn stack - we use it in
| production and IMO it's much easier to reason about and code in
| than asyncio. Just write synchronous Python, including long-
| running `requests` calls, and the system will yield control
| whenever you're blocked on network or disk I/O, no matter how
| deep that is in your stack. The entire ecosystem of Python
| libraries just works. I also wish more people knew about gevent
| - it's truly magical, especially if you need to work with APIs
| with unpredictable latency!
| jononomo wrote:
| Where can I learn more about this? I've been thinking of trying
| to integrate Supabase Realtime
| (https://github.com/supabase/realtime) into my Django app
| (without the rest of Supabase), but I'd also like to keep
| things even simpler if possible.
|
| Also, what was the reason not to go with Gevent?
| lastofus wrote:
| >Also, what was the reason not to go with Gevent?
|
| The biggest downside of Gevent IMO is that it enables the
| magic of turning sync code into async code via monkey
| patching things like the socket lib. This lack of
| explicitness can make things a bit difficult to reason about,
| without a good mental model of what Gevent is doing
| underneath the hood.
| jwtnb wrote:
| Hi, thank you for your comment. I have been looking for correct
| ways to integrate websockets on my django app. Currently I'm
| using gevent gunicorn setup. Can you elaborate a bit more on
| how to make websockets work with an existing app? Thanks.
| bastawhiz wrote:
| I did this. If you're on a modern version of Django, it's
| actually pretty easy: swap out gunicorn for an async
| alternative and install django-channels. I even got it
| working with GraphQL subscriptions (using graphene) without
| much hassle.
___________________________________________________________________
(page generated 2023-06-16 23:01 UTC)