[HN Gopher] Django 4.1
___________________________________________________________________
Django 4.1
Author : j4mie
Score : 270 points
Date : 2022-08-03 09:13 UTC (13 hours ago)
(HTM) web link (www.djangoproject.com)
(TXT) w3m dump (www.djangoproject.com)
| WhatsName wrote:
| The most anticipated Django release for me personally. Async ORM
| access is a massive quality of life improvememt, same for async
| CBV. The later is also a blocker for Django Rest Framework going
| async.
|
| Sure most CRUD applications work perfectly fine using WSGI, but
| for anyone using other protocols like WS or MQTT in their app
| this is kind of a big deal.
| collinmanderson wrote:
| > for anyone using other protocols like WS
|
| Yes, if you do WebSockets, async really is a must if you don't
| want to have one thread per active connection. Each thread
| takes 8mb of virtual memory, not sure how much actual memory.
|
| It's also nice for a backend that's doing a lot of proxying
| apis to another backend.
| tgv wrote:
| Seriously: why use Django for a REST-like service? I inherited
| an application that (probably) evolved from form-based to REST
| with a JS frontend, and I fail to see the point. There's a
| model, a serializer, handlers to implement relations, and
| multiple views _per data type_. It 's so much overhead, and
| everything has to be kept in sync.
| theptip wrote:
| It really shines if you are doing simple CRUD APIs where you
| can lean on autogenerated routes, views, and serializers.
| Since you can get full CRUD for a model in something like 10
| lines of code.
|
| When you need to wire up custom serializers per model (eg
| specifying subsets of fields or making some read-only) and
| wire up non-CRUD actions then I think you can fall down a
| slippery slope where you write as much (or more) code than if
| you just used say marshmallow and flask to manually write
| your API.
|
| Also you get the Django model admin for free, so for internal
| services you have a nice CRUD admin for operators.
|
| I think it's a great tool for accelerating your first
| 100-200kloc of API code. Beyond that it can start to creak at
| the seams depending on your usecase. (I'd make the same
| assessment about Django itself FWIW).
| giancarlostoro wrote:
| I went with Django Rest Framework when I used it for a
| project because it provided things I would otherwise spend
| says implementing out of the box. The one thing I definitely
| did implement for myself was filtering, the built-in stuff
| was kind of mediocre for our needs, outside of that I want to
| say we left pagination and everything else as-is. It was a
| bit of a nightmare to look up documentation on though I will
| admit, if you don't know the Google Buzzword Soup you got to
| look up. Previously to that I was using CherryPy and building
| my entire kitchen sink myself, so it had nice moments and bad
| moments, but it definitely saved me on time and effort.
|
| You typically do Django + REST because you are already
| comfortable with Django or because you were already using it,
| although I could argue, you could consider making your API
| project stand alone and using CherryPy or even FastAPI for a
| much nicer experience. The downside there being you wouldn't
| be using Django's ORM. I really do wish Django's ORM would be
| spun off into a stand alone project that Django then imports,
| so anyone else can use it in other non-django specific
| projects.
| znpy wrote:
| I remember watching a beautiful talk from some pycon whose
| details I can't remember about using django as if it was a
| micro-framework.
|
| It basically boils down to rejecting the folder structure
| that django-admin sets up for you, importing stuff yourself
| and doing some basic initialisation/configuration.
|
| Once you do that, django kinda somehow behaves like a
| micro-framework, with the significant difference that you
| can import and use the advanced features if you want/need
| to.
| rsanek wrote:
| I guess I prefer to keep it light and use a true
| microframework like flask for this use case. I find I can
| similarly import Flask extensions without bringing in a
| full heavy framework
| ingenieroariel wrote:
| Here is an example of a single file Django project I
| started around eight years ago that does what you
| mention:
|
| https://github.com/planetfederal/registry/blob/149a2b958d
| d05...
| Evidlo wrote:
| PonyORM is sort of like Django's
| chpmrc wrote:
| If I had a dollar for every time I heard this I would be
| having lunch with Jeff Bezos right now.
| danmur wrote:
| I just make normal money and I don't want to have lunch
| with Jeff Bezos. Maybe in this scenario you lose money each
| time you hear it and you're so in debt that Jeff Bezos has
| basically purchased you :P.
| fernandotakai wrote:
| every python developer has accidentally made django at
| least once.
|
| you start with "i will do in flask because it's easier".
| then you need to add authorization/authentication. and then
| templating, ORM, maybe some RESTful flask library to make
| endpoints a bit easier... and now you have django, but in a
| different way.
| chpmrc wrote:
| This 1000x. It's such a slippery slope that you'd think
| by now most people would be aware but no, there's always
| the "who needs an ORM" guy :)
| metadat wrote:
| Gross, why would you want this? Improve your standards
| matey. What a sad thing to aspire to.
| collinmanderson wrote:
| Why do a REST-like service?
|
| It makes sense as backend for a native iOS/Android app, and
| if you already have that REST endpoint then it can make sense
| to re-use it for the web-version of your app.
|
| But if you're not doing an iOS/Android app (most websites?),
| then I generally agree it's a lot more work compared to form-
| based / server-side-templates (though sometimes can give a
| _slightly_ better user experience, _if_ done correctly.)
|
| HTMX seems to be growing as a "REST endpoint" alternative,
| but again not as useful if you're also doing a native
| iOS/Android app.
| pen2l wrote:
| In this day and age, mobile apps are _almost_ a must so
| REST is everywhere.
|
| A well-implemented REST app is pretty nice, business logic
| divorced from all UI and API lends itself nicely as a
| comfortable and maintainable programming environment. More
| often than not, being RESTful from the get-go tends to have
| positive returns down the line.
| acdha wrote:
| > In this day and age, mobile apps are _almost_ a must so
| REST is everywhere.
|
| This really isn't true. Mobile apps are far more
| expensive to develop (everything is doubled, you're
| dealing with periodic forced updates outside of your
| development schedule, etc.) and there are a ton of apps
| which don't need anything you can't do in a browser.
|
| One of the earliest things to do on a project is agreeing
| on what level of complexity makes sense for you. If your
| project doesn't have a known hard requirement for mobile
| apps and correspondingly large team sizes, a classic
| Django app is likely to let you iterate an order of
| magnitude faster at lower cost overall.
| sgt wrote:
| I don't find it has a lot of overhead if you keep it simple.
| No need to use class based views for REST services, just add
| function-based views with the @api_view decorator.
|
| You can then within minutes return JSON coming from your data
| model by also adding an additional decorator to make it
| render JSON output.
|
| If returning data directly from managed ORM objects, you will
| need to serialize it, but you can easily build an API without
| doing that and just return a dict.
| varispeed wrote:
| It all depends how it is written. While Django and REST is
| quite strongly opinionated, you can still make it difficult
| to maintain if you want to. That being said, Django REST
| allows you to move fast. It's very very easy to knock out a
| REST API and data models plus some basic admin panel. Once
| your API matures - you can then reimplement it in Go or other
| platform.
| mrits wrote:
| It's a really easy framework that just takes a few days to
| get proficient in. It is very well documented and because it
| comes with so much "overhead" you don't have to get your own
| team to make a set of different mistakes building the
| features as needed.
| larrik wrote:
| Proper Django Rest Framework should basically have zero view
| code. Treat Serializers as forms, point at them in the
| appropriate generic view (or viewset) subclass, and you are
| done.
|
| If you are writing view code in DRF, you are probably doing
| it wrong.
| physicsguy wrote:
| I don't think this is fair - the DRF views are a great
| starting point for a project but I don't think I've ever
| worked on one where they're totally sufficient without at
| least some modification in places if you don't want to make
| use of signals (which are very difficult to debug!).
|
| Even things as simple as adding logging to calls on certain
| end points will require you do provide your own view method
| (which might then call the mixin's provided method).
| WhyNotHugo wrote:
| Also, async views allow for better scaling. Say your view needs
| to make an API call to a third party service within a request-
| response cycle. Currently, this will block an entire
| worker/thread while doing a blocking request.
|
| With async, we can use async HTTP libraries and scale these WAY
| better.
| BozeWolf wrote:
| Or just run 20 workers and let the OS handle async. OSes are
| pretty good at this. Async is pretty hard to reason about.
|
| Async is beneficial when you make multiple blocking (http)
| request to a resource at the same time and later fold that
| into one response. Parallel stuff. Or when you are forced to
| run a single thread. Or when you are memory bound.
|
| Reality is that most requests depend on previous requests
| quite often. In the latter case there is no benefit in terms
| of speed for the user.
|
| I know this is unpopular opinion though :-)
| bpicolo wrote:
| Java's Loom model really does feel like the future of
| concurrency in many ways.
| chpmrc wrote:
| Why would you block a process that could use CPU cycles to
| do something useful just to wait on I/O? With the right
| primitives async is not "hard" nor inconvenient. Maybe
| you're referring to concurrency?
| BozeWolf wrote:
| No I am referring to async, really. Accidentally blocking
| operations like open() or psycopg or mongodb or some
| included library which (accidentally) uses any of that
| stuff. Basically any library requiring requests is not
| usable. Or worse: open().
|
| It can be hard to determine if something is doing
| something blocking and if it is, it is hard to debug.
| Especially in python world which, unlike javascript, was
| not async from the start.
| chpmrc wrote:
| I don't understand, are you saying async is hard or that
| a poor implementation is a source of issues (e.g. async
| function that calls sync primitives)? Because I totally
| agree with the latter but I have no idea what "hard"
| means. Very little changes in terms of syntax (assuming
| support for async/await ops) or reasoning.
| JimDabell wrote:
| > Or just run 20 workers and let the OS handle async.
|
| That just means you can only handle 20 concurrent requests
| before performance drops off a cliff. If the third-party
| service you are talking to is slow, then you'll just end up
| with 20 workers all waiting for the service to respond,
| while other requests pile up. With async, those workers
| could still be handling other requests. Adding more workers
| because you are blocking on i/o works for low traffic
| services, not for anything remotely busy.
| ElectricalUnion wrote:
| > Adding more workers because you are blocking on i/o
| works for low traffic services, not for anything remotely
| busy.
|
| Yes.
|
| That just means in very bad cases, you can only handle 1
| "concurrent request" before performance drops off a
| cliff. If the third-party service you are talking to is
| that slow, then you'll just end up with bufferbloat (high
| latency in hidden queues) waiting for the service to
| respond, requests piling up.
|
| With async, unless you know what you're doing and handle
| back pressure, work pile up even worse, the service stops
| working and you get 0 throughput instead.
| BozeWolf wrote:
| Agree! Up the number of workers to 40, fix your
| application in another way... or indeed go with async
| stuff! ;-)
|
| I was just saying that async quite often is not of much
| help and just complicates things. Of course it has its
| use! Your case is a great example of converting a django
| view into an async view.
| redox99 wrote:
| So now if the requests become slow because of the third
| party or whatever, and take 1 second each, you have a max
| performance of 20 rps.
|
| With async instead you would still be able to handle
| thousands of rps.
| ramraj07 wrote:
| Might be unpopular but still good to stick to it. I tend to
| associate excessive enthusiasm for pythons async await
| framework as a sign of immaturity and potential for doing
| things without understanding them fully. I'd never trust
| async code from anyone except the best python engineers.
| Most of them out there can't even debug linear code, adding
| async to the mix only makes it worse. At least you can
| trust the time stamps.
| BozeWolf wrote:
| Or when accidentally some blocking database driver is
| used, even when django's ORM is async, a query with
| mongodb's driver is not async. Or just the filesystem...
| This harms performance badly, especially if the idea was
| to run only a few threads.
|
| Other than that, it has benefits too! :-)
| mrmikardo wrote:
| And indeed, there's an important caveat in the Django 4.1
| docs;
|
| _Note that, at this stage, the underlying database
| operations remain synchronous, with contributions ongoing
| to push asynchronous support down into the SQL compiler,
| and integrate asynchronous database drivers._...
|
| https://docs.djangoproject.com/en/4.1/releases/4.1/#async
| hro...
| coldtea wrote:
| > _I tend to associate excessive enthusiasm for pythons
| async await framework as a sign of immaturity and
| potential for doing things without understanding them
| fully._
|
| Couldn't it be the inverse? I.e. your inexeperience with
| async code in other languages, and not understanding it
| fully, translating into "new thing aversion"?
| hansonkd13 wrote:
| I think my experience in async in other languages has
| shown how rough async in Python is.
|
| Unless async is the default python, it will always be an
| after thought and introduce needless complexity when
| working on large projects that require a lot of external
| libs.
| maxmalysh wrote:
| > Or just run 20 workers and let the OS handle async. OSes
| are pretty good at this. Async is pretty hard to reason
| about. def slow_sync_view(request):
| # these requests won't be executed in parallel;
| # async version could eliminate this extra latency
| foo =
| requests.get('https://www.google.com/humans.txt').text
| bar = requests.get('https://checkip.amazonaws.com').text
| return HttpResponse(f'{foo}\n{bar}')
| KptMarchewa wrote:
| Basic concurrent.futures.ThreadPoolExecutor solves this
| problem.
|
| https://docs.python.org/3/library/concurrent.futures.html
| #co...
| pdhborges wrote:
| I think you are being down voted because a straight port
| of this code to async would still run one request after
| the other.
|
| You would have to queue one task for each request in the
| event loop and then await for them both to gain some
| parallelism in the I/O section of the code.
| westurner wrote:
| ## _Asynchronous ORM interface_
|
| https://docs.djangoproject.com/en/4.1/releases/4.1/#asynchro...
| :
|
| _`QuerySet` now provides an asynchronous interface for all
| data access operations. These are named as-per the existing
| synchronous operations but with an `a` prefix, for example
| `acreate()`, `aget()`, and so on._
|
| > _The new interface allows you to write asynchronous code
| without needing to wrap ORM operations in `sync_to_async()`:_
| async for author in
| Author.objects.filter(name__startswith="A"): book =
| await author.books.afirst()
|
| > _Note that, at this stage,_ the underlying database
| operations remain synchronous, with contributions ongoing to
| push asynchronous support down into the SQL compiler, and
| integrate asynchronous database drivers. _The new asynchronous
| queryset interface currently encapsulates the necessary
| sync_to_async() operations for you, and will allow your code to
| take advantage of developments in the ORM's asynchronous
| support as it evolves. [...] See Asynchronous queries for
| details and limitations._
|
| ## _Asynchronous handlers for class-based views_
|
| > _View subclasses may now define async HTTP method handlers:_
| import asyncio from django.http import HttpResponse
| from django.views import View class AsyncView(View):
| async def get(self, request, *args, **kwargs): #
| Perform view logic using await. await
| asyncio.sleep(1) return HttpResponse("Hello async
| world!")
| aynyc wrote:
| I don't fully understand the _sync_to_async()_ operation
| here. So the underlying database is not async, then is Django
| just tossing the sync DB call onto a thread or something?
| Ralfp wrote:
| exactly, it fires database call in a separare thread with
| coroutine awaiting this threads result.
| aynyc wrote:
| Is the limitation on django ORM? Because I can run
| asyncpg/psycopg3 and sqlalchemy with async mode without
| any issues.
| megaman821 wrote:
| Yes, it is just being put on a thread. A future Django
| release will use async database drivers. Django is very
| incrementally adding support for async. I don't believe
| they have started work on async templates yet, which will
| be one of the last major areas before the framework is
| fully async.
| collinmanderson wrote:
| Yes, they're working from the top of the stack down, so
| first it was views+middleware, now it's the top layer of
| the database stack, and in future releases they'll work
| on pushing native async lower and lower in the stack.
|
| Basically every function call that does db-queries needs
| an "a"-prefixed version. aget(), acreate(),
| aget_or_create(), acount(), aexists(), aiterator(),
| __aiter__, etc. And in future versions they'll work on
| the lower level, undocumented api, like _afetch_all(),
| aprefetch_related_objects(), compiler.aexecute_sql(),
| etc.
|
| Eventually you'll probably need to switch database
| drivers to something that actually supports async.
| hansonkd13 wrote:
| Why do they need Async templates? When a template renders
| it should be pure data and fully on the CPU. It seems
| Async templates would encourage a bad behavior of doing
| n+1 queries inside the HTML.
| megaman821 wrote:
| My immediate thought would be custom template tags. Like
| `{% popular_articles %}` would have to connect to the DB
| and load the articles. I am sure there are other reasons
| too.
| simonw wrote:
| Django makes a lot of use of lazy evaluation. You'll
| often construct a QuerySet object in a Django view like
| this: entries = Entry.objects.filter(
| category="python" ).order_by("-created")[:10]
|
| Then pass that to a template which does this:
| {% for entry in entries %}...
|
| Django doesn't actually execute the SQL query until the
| template starts looping through it.
|
| Async template rendering becomes necessary if you want
| the templates to be able to execute async SQL queries in
| this way.
|
| Jinja has this feature already with the enable_async=True
| setting - I wrote a tiny bit about that in
| https://til.simonwillison.net/sqlite/related-content
| syrusakbary wrote:
| I completely agree. I have been using GraphQL in Django with
| WebSockets and ASGI and it was a bit of pain in the ass to get
| everything working, but with this changes things will get much
| smoother.
|
| Great work Django team!
| sirodoht wrote:
| I wonder how popular the async interface will become. I use
| Python and Django extensively but haven't touch any of the async
| operations.
| Piezoid wrote:
| I use the event loop to defer the (io heavy) post-processing of
| some requests without involving an off-process task runner like
| Celery. I use a custom implementation of this: https://django-
| simple-task.readthedocs.io/how-it-works.html It is way simpler
| when you don't need strong guaranties and checkpoints
| persistence.
| jonatron wrote:
| Once the underlying database queries become async, I can see it
| becoming popular. Many views do multiple database queries that
| aren't dependent on each other, which would be a quick win to
| make async.
| Chiron1991 wrote:
| Since ultimately anything you build in Django will eventually
| go into the ORM layer, it won't become anything unless the ORM
| is fully async. This release brings the async API to it, but:
|
| > Note that, at this stage, the underlying database operations
| remain synchronous
|
| That kinda defeats the whole purpose of an ASGI app.
| tecleandor wrote:
| But you could use it for load not tied to database, isn't it?
| Like CPU, IO or network bound stuff...
| aynyc wrote:
| You don't use async for CPU. Python async is actually
| asyncio, so I/O and network such as calling external API,
| etc.. It'll make integration much easier to reason about.
| buro9 wrote:
| Guess I best upgrade my production side-project from Django 1.5.9
| mafro wrote:
| No need, that's a good vintage.
| wellthisisgreat wrote:
| chef's kiss
| collinmanderson wrote:
| Python 2 or 3?
| buro9 wrote:
| There's a Python 3?
|
| It's 2... definitely 2.
| intrestingstuff wrote:
| you found the sweet spot
| ralmidani wrote:
| Django is single-handedly responsible for making me fall in love
| with programming. I am now using Elixir/Phoenix and 100% sold on
| the benefits of working with immutable data, but I still keep an
| eye on developments in the Python/Django ecosystem. Seeing Django
| edge closer to having a fully async stack is very exciting, and
| should make it a more viable platform for existing users as well
| as newcomers.
| brianbreslin wrote:
| as someone who is re-learning to code now after a 15 year
| absence, django is for sure making it feel amazing. Sometimes
| folks in this community overthink normal people's actual needs
| when it comes to programming tools, Django seems to support
| most everyday uses just fine.
| hansonkd13 wrote:
| Great to see improving support for async here. I am a big fan,
| but IMO python async code is just different from other async
| langauges. I used async extensively in python and compared to
| other languages it is a pain to use because unless 100% of the
| the libs you want support it you will get stuck on some sync
| library. Async in python is not even a 2nd class citizen. Its
| more like a 3rd class citizen. Its gets better every year but it
| is minuscule compared to regular sync code, so very few tutorials
| mention it and nobody teaches it as the default way of doing
| things.
|
| In order for async to be popular it needs to be the default way
| of writing python. Right now its an after thought for writing
| "high performance" python. For example asyncio version of redis
| got 72,114 downloads this month. The sync version got 26,825,663.
| Thats not even in the same universe.
|
| Historically, using something like gevent is a more drop-in
| solution for python if you want lightweight threads, but it comes
| with its own problems. Gevent gives python what Zig has that
| automatically turns all sync IO calls to async and your app is
| now magically using greenthreads.
|
| However I think gevent in the past was the crutch that prevented
| a lot of lib authors from writing async libs.
| traverseda wrote:
| > unless 100% of the the libs you want support it you will get
| stuck on some sync library.
|
| What do you mean by this? Do you mean like an actual deadlock,
| or just waiting on some kind of IO? Are you trying to use async
| for performance?
| collinmanderson wrote:
| Yes, though I do think the python async ecosystem is gradually
| moving from a "3rd class citizen" to a "2nd class citizen".
| There are sync+async libraries like HTTPX that are gaining
| support. I don't expect it to really ever be a "1st class
| citizen" (supported as well as sync in pretty much every
| library everywhere) because of backward compatibility and
| ultimately async is just harder to program. Sometimes developer
| time is more important than the performance wins of async.
| acdha wrote:
| > Sometimes developer time is more important than the
| performance wins of async.
|
| Also, a fair fraction of applications don't see a meaningful
| benefit compared to other things the developers could be
| spending time on. Async is not only harder to write in many
| cases but can force you to deal with new problems which might
| be harder to manage (e.g. if you use non-trivial amounts of
| memory, bounding your peak usage can be a challenge).
|
| I've used async a fair amount and it's worked really well in
| network-heavy uses (HTTPX is really great) but outside of
| those the savings have been a lot less than people thought,
| independent of the language used (I'm thinking of a colleague
| who spent a lot of time trying to beat classic Python with Go
| and ended up with about a 10% win after a month or so of work
| because the problem had enough CPU/memory contention to
| prevent greater savings -- that's not a trivial win but it
| definitely wouldn't have been enough to justify changing
| languages on its own).
| aobdev wrote:
| Hey, I wanted to chime in and say that I too am underwhelmed
| with the support for async in Python libs. Coming from a
| node.js background, it falls short because you constantly have
| to think about whether async code is calling sync code, or vice
| versa, and whether you need an executor, threadpool, etc. Just
| a mess that gets in the way of productivity IMO.
|
| However, things have been getting better, and in case you
| haven't seen it, AIORedis[0] appears to be the de facto
| standard for async in Python (a little better with 1,724,389
| downloads this month). It's popular enough that it's been
| merged with the official redis-py driver[1].
|
| [0] https://aioredis.readthedocs.io/en/latest/
|
| [1] https://github.com/redis/redis-py/releases/tag/v4.2.0rc1
| codethief wrote:
| > In order for async to be popular it needs to be the default
| way of writing python. Right now its an after thought for
| writing "high performance" python.
|
| Might this have to do with the fact that the (lack of)
| performance of asyncio is exactly the issue here?
|
| I remember reading a blog post by zzzeek (the SQLAlchemy author
| who's also participating in our discussion here) about
| precisely this issue a while ago. Not sure but it might have
| been this one:
|
| https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...
| ac130kz wrote:
| Let's see how long it will take for DRF to support async...
| BiteCode_dev wrote:
| This is going to make django-ninja even easier to use, so it's
| very welcome.
|
| And if you like django and never tried django ninja, it's like
| DRF and FastAPI had a baby: https://django-ninja.rest-
| framework.com/. You define your endpoints using pydantic type
| hints, and it generates the API, validators and docs all in one
| go.
| godtoldmetodoit wrote:
| +1 on Ninja. Been using it on a number of projects the last 6
| months, and its been a joy to use.
| pen2l wrote:
| Django is supposed to be opinionated and batteries-included,
| Django Software Foundation ought to shepherd some REST package.
| There's ninja, DRF, etc. and a few more on the horizon. I agree
| with the other poster, RESTful projects with views one
| encounters: often complicated and not easily maintainable. DSF
| should take over django-ninja. :)
| game_the0ry wrote:
| Am I bad SWE if I do not care for for async in django?
|
| I am sure a a lot of django devs will like it, but if I need the
| performance that I would get from going async, I would not being
| using django as my web framework.
|
| FWIW I like django / python for a lot of things, just not for
| performance.
| bastawhiz wrote:
| Async doesn't make your code faster, though there are cases
| where it lets you handle more requests with a single server
| (especially if you have code that is async). But moreover, if
| you have libraries that use async code, it's really not ideal
| if the thing that runs those libraries isn't async. You don't
| have to like or want it, but it's there for the folks who want
| or need it.
| froza wrote:
| it seems most useful for those who want to add simple Websocket
| support to the existing Django codebase and use the ORM without
| too much pain
| ralmidani wrote:
| Enabling async means those already invested in Django can do
| more on one machine without a complete rewrite, and those
| considering it can adopt it with fewer reservations. There will
| always be more scalable platforms out there, but enabling async
| with Django helps it serve the "sweet spot" in terms of both
| ergonomics and scalability.
| roflyear wrote:
| Probably not. If you need async for DB "stuff" to take
| advantage of your resources, I think it is a rare thing, and
| likely you would not want to use the ORM anyway, and then can
| just do whatever you want without it.
| rossdavidh wrote:
| If you have one part of your app that needs async, but 90% of
| it is in the sweet spot for Django, then this could be
| important.
| roflyear wrote:
| No. Async in Python is... eh. Rarely is it useful. Sometimes it
| is, though! Use it when it is useful...
| Spivak wrote:
| Twisted is a pretty dope though. Wouldn't migrate a non-async
| app to async but would absolutely look at async-by-default
| for new projects.
| jmconfuzeus wrote:
| I don't care about it either. Who's actually doing a ton of
| querying in their views? We have materialized views for that.
|
| I wish Django followed Ruby on Rails a little by focusing on
| improved developer experience instead of niche features that
| only a few mega corps need.
| Demiurge wrote:
| No, I agree with you. It's 2022 and I have thought only once
| that it would be neat to have async methods from Django, but
| only as some 'cool thing to try'.
|
| I don't actually actually know if there would be any
| substantial benefit. It's like when I tried PyPy as my django
| interpreter. It gave a few percent speed increase in "real
| world" but created extra work for deployment.
|
| I've been using Django non-stop from 0.96, and it just works,
| and you can build around it. If I was running a trading
| clearing house, or Instagram, I'd probably care about marginal
| improvements, and then use another language. But for Django,
| just tuning the workers per server node, and using django-rq
| jobs works really well.
|
| Kudos for the enthusiasts working on this though, because
| 'async' is a new Python feature/meme, and supporting it is just
| being a good ecosystem player.
| nilsbunger wrote:
| > If I was running a trading clearing house, or Instagram,
| I'd probably care about marginal improvements, and then use
| another language.
|
| Ha, Instagram was written in Django!
| collinmanderson wrote:
| Right, though they're stuck on 1.8 (upgraded to 1.8 only so
| they could use python3), and really just use it for
| request/response/views, not ORM or anything else.
| Demiurge wrote:
| Thats why I mentioned it, lol. I figured a few percentage
| points improvements in perf for Instagram == $$$
| znpy wrote:
| > Am I bad SWE if I do not care for for async in django?
|
| In a way, yes.
|
| Async in django means that with relatively minor adjustments to
| your codebase you may significantly increase your requests over
| time throughput.
|
| And as someone else pointed out, if you have database queries
| that do not depend on each other you can effectively wait for
| their results in parallel, which might this time not only
| increase throughput but also lower the total request time.
|
| It's a very valuable development that brings significant gain
| for relatively minor changes.
|
| If you are a professional django developer you should not
| dismiss such developments like that.
| nickjj wrote:
| If anyone is interested I updated my example Django Docker app to
| 4.1 at: https://github.com/nickjj/docker-django-example
|
| It includes running Django, Celery, gunicorn, Postgres, Redis,
| esbuild and Tailwind through Docker and Docker Compose. It's set
| up for development and production.
| akx wrote:
| That seems to be using WSGI, not ASGI, though, so no natively
| `async` views..?
| nickjj wrote:
| > That seems to be using WSGI, not ASGI, though, so no
| natively `async` views..?
|
| One does not simply go async. Changing your entire app to go
| full async has serious implications, requires every library
| you use to support it and requires a completely different
| mode of thinking.
|
| The example project includes an ASGI file if you want to use
| an async app server like uvicorn with async views instead of
| gunicorn. That would involve changing a couple of lines of
| code. That's very much a "per user" decision and in my
| opinion shouldn't be the default for the project when 4.1
| only landed a few hours ago. It's going to take a long time
| until apps are ready to go async by default, both from
| waiting on libraries to support it and the community to shift
| towards this style of building apps.
| akx wrote:
| I know one does not simply go async, sure, but I thought it
| should be noted (especially considering how async-centric
| this release is) that the repo doesn't immediately allow
| for async.
|
| Also, "4.1 only landed a few hours ago" isn't the perfect
| argument in my books since support for ASGI and async views
| in general was added back in 3.0 and 3.1:
|
| * https://docs.djangoproject.com/en/4.0/releases/3.0/#asgi-
| sup... * https://docs.djangoproject.com/en/4.0/releases/3.1
| /#asynchro...
| VWWHFSfQ wrote:
| > use an async app server like uvicorn with async views
| instead of gunicorn
|
| I believe gunicorn with the uvicorn worker class is the
| recommended production server.
|
| > For production deployments we recommend using gunicorn
| with the uvicorn worker class.
|
| https://www.uvicorn.org/#running-with-gunicorn
| nickjj wrote:
| > I believe gunicorn with the uvicorn worker class is the
| recommended production server.
|
| Yep for production that works. In development tho I don't
| think the gunicorn --reload flag works with that worker
| type.
|
| There is an issue open for this at
| https://github.com/benoitc/gunicorn/issues/2339 and since
| I last tried about a year ago I guess they have an
| unofficial "workaround solution" in one of the comments
| but it involves making a custom class to support
| reloading but that requires copying in some code into
| your project and also using different worker types in dev
| vs prod.
|
| I think this is a good example of why I don't default to
| uvicorn and async in the project with Django 4.1. The
| whole package of "async everything" isn't ready for prime
| time in my opinion. It will improve over time but it's a
| waiting game at this point. Early adapters can easily
| swap it in if they want.
| collinmanderson wrote:
| Interesting looks like it might actually be a python bug.
| Somehow just changing from sys.exit(0) -> os._exit(0)
| apparently fixes it.
|
| "There is a problem with using the sys.exit() in a forked
| process (child processes or workers) instead of using the
| os._exit() that is mentioned in python documentation."
|
| https://github.com/benoitc/gunicorn/pull/2820
|
| I've always just needed to manually reload the gunicorn
| async server in development. Super annoying.
| blitzar wrote:
| Never go full async
| kolanos wrote:
| > CMD ["gunicorn", "-c", "python:config.gunicorn",
| "config.wsgi"]
|
| Looks like this example is still WSGI?
| nickjj wrote:
| Yep but it includes "config.asgi" which you can use with
| gunicorn's uvicorn worker or uvicorn if you want if you want
| to use async views.
| ydnaclementine wrote:
| Thanks for this. I've been learning django and have been
| struggling to find a good example of a way to integrate modern
| js into django. Unfortunately not as much support as rails +
| esbuild
| fredsmith219 wrote:
| This is very cool. Thank you.
| omarhaneef wrote:
| I was going to say you should offer a tutorial around how this
| can be used. It is much more useful to have a walk through when
| people put out such products.
|
| And then I decided not to say that because it is up to you how
| to spend your time and why should you make a tutorial for
| people if you don't want to.
|
| And then I realized, you have actually made dozens and this is
| the work you do. I think I took a Flask-docker course of yours
| on Udemy a few years ago. It was pretty good, and "forward
| leaning" for the time.
|
| I know you're not allowed to shill your product on here, but
| presumably I can.
| nickjj wrote:
| Thanks a lot!
|
| By the way the version of the course on Udemy doesn't get
| updated. The full version of the course is at
| https://buildasaasappwithflask.com/ which includes lifetime
| free updates (it has over 10+ hours of extra updates and new
| features not on the Udemy version). Udemy is a really bad
| platform for instructors which is why I've been trying hard
| to decouple myself from them over time.
|
| As for a tutorial on using this repo specifically, did you
| find the readme file lacking in any way? I tried to write the
| readme to be fully self contained. It would expect prior
| Docker knowledge tho. I did give a talk at DockerCon going
| over a bunch of Docker best practices at
| https://nickjanetakis.com/blog/best-practices-around-
| product.... That talk live demos a Flask example app not
| Django but everything still applies to Django. The same
| patterns are used in both projects. It can act as a mini-
| tutorial on this repo.
| soperj wrote:
| I was going to say that you should really camelcase your
| url, because it's hard to see what it's saying, but even
| when you do that... https://buildASASSAppWithFlask.com/
| doesn't really help that much hahaha.
| omarhaneef wrote:
| I have not tried it, but I believe I could download it and
| set up the app based on the readme.
|
| I would not be confident using it, but I'll give it a try.
| zzzeek wrote:
| looking at the release notes and the source, it looks like the
| asyncio interface is still using the old drivers, like psycopg2
| for postgresql, and using a threadpool.
|
| we at SQLAlchemy came up with a way to interface asyncio frontend
| and backend (like asyncpg for the driver) while maintaining all
| the "in the middle" code as synchronous style. the dark secret is
| that for this approach you have to use greenlet to propagate the
| "await" operations. It's been in our release for 18 months now
| with 1M downloads a day and there have been no problems reported
| with it, so i continue to wonder why nobody else seems to want to
| look at this approach. one downside which nobody has complained
| about yet is that it makes profiling tools like cProfile harder
| to use, but seems not to have come up yet.
|
| there's also another approach, which is that you rewrite all your
| "in the middle" code as pure asyncio, then create a dummy task to
| implement your synchronous API. dark secret for that one is you
| had to rewrite everything and I'm not sure if there's other
| performance implications for having awaits all throughout code
| that's running only one task per thread.
| baq wrote:
| thanks for sqlalchemy. it's a shining star in the Python
| package ecosystem.
|
| > interface asyncio frontend and backend (like asyncpg for the
| driver) while maintaining all the "in the middle" code as
| synchronous style. the dark secret is that for this approach
| you have to use greenlet to propagate the "await" operations.
| It's been in our release for 18 months now with 1M downloads a
| day and there have been no problems reported with it, so i
| continue to wonder why nobody else seems to want to look at
| this approach.
|
| do you have a blog post or some other kind of writeup to read
| about it? maybe nobody else had the same idea and it works so
| well nobody is aware that it can be done...
| tmpz22 wrote:
| Is it a shining star or a rough dirty pragmatic get-the-job-
| done tool with a heart of gold thats approachable to
| newcomers and grizzled veterans alike. The trenches are too
| dirty for shining stars.
| zzzeek wrote:
| I was a bit ranty but the gist at https://gist.github.com/zzz
| eek/d9c98c43553e43e76600f03dfb5e1... lays out pretty much
| everything that happened.
| syastrov wrote:
| Someone mentioned it on a Django forum thread about async ORM
| [0]. It seems to have been ignored. Super cool idea and
| execution! I hope it gets some more exposure.
|
| [0] https://forum.djangoproject.com/t/asynchronous-orm/5925/51
| snapcaster wrote:
| Just want to say thanks for SQLAlchemy work! Love the package
| and use it in every one of my work and personal projects
| bredren wrote:
| Mike, thanks for this reply and your work on SQLAlchemy. My
| employer uses your framework, but not yet the newest versions
| featuring async.
|
| Previously, and still in my personal projects, I have been a
| Django and Django ORM person.
|
| I got away with by and large avoiding SQL statements in
| building web apps and have sometimes struggled to understand
| why SQL statements seemed much more forward-facing in
| SQLAlchemy in comparison.
|
| Then I listened to your recent Talk Python podcast episode with
| M. Kennedy [1] and it helped me understand where the framework
| is coming from with regards to expectations for SQL capability
| from a developer that wants to use SQLAlchemy.
|
| I believe the focus was much more so on the dark secrets you
| mention above, though and colored in how you discovered those.
|
| If anyone wants to learn more about async or SQLAlchemy, I
| highly recommend listening.
|
| As an aside, it was also cool to hear you talking about your
| project because for me, it humanized you as the maintainer. You
| sounded pretty chill, which was for some reason not what I was
| expecting.
|
| I hope you can make it back on to that podcast with less of a
| gap between appearances next time.
|
| [1] https://talkpython.fm/episodes/show/344/sqlalchemy-2.0
| syastrov wrote:
| There are a lot of comments about async ORM, but to me the
| support for application-level (e.g. if you use forms, admin, DRF,
| Graphene) validation of DB-level constraints is a super cool
| feature that solves a real pain point - having to duplicate this
| logic at both levels, or forgetting to do so.
___________________________________________________________________
(page generated 2022-08-03 23:01 UTC)