[HN Gopher] The Practical Guide to Scaling Django
___________________________________________________________________
The Practical Guide to Scaling Django
Author : rbanffy
Score : 58 points
Date : 2024-11-15 18:49 UTC (4 hours ago)
(HTM) web link (slimsaas.com)
(TXT) w3m dump (slimsaas.com)
| feydaykyn wrote:
| The snippets are not false, but there's so much context missing
| it's easy to worsen the situation, especially for beginners which
| seem to be the target audience.
|
| First, this guide should emphasize the need to measure before
| doing anything : django silk, django debug toolbarsm, etc. Of
| course, measure after the optimizations too, and measure in
| production with an apm.
|
| Second, some only work sometimes : select_related /
| prefetch_related / iterator will lead to giga SQL queries with
| nested joins all over the place, and ends by exploding ram usage.
| It will help at first, but soon enough one will pay any missing
| sql knowledge or naive relationships.
|
| Third, caching without taking the context into account will
| probably lead to data corruption one way or another. Debugging
| stale cache issues is not fun, since you cannot reproduce them
| easily.
|
| Fourth, celery is a whole new world, which requires workers,
| retry and idempotent logic, etc.
|
| Finally, scaling is also about code: architecture, good
| practices, basic algorithm, etc
|
| I'll end by linking to more complete resources : -
| https://docs.djangoproject.com/en/5.1/topics/performance/ -
| https://loadforge.com/guides/the-ultimate-guide-to-django-pe... -
| https://medium.com/django-unleashed/django-application-perfo...
| vldmrs wrote:
| At first I wanted to criticize the post, buuut after finishing
| reading it I actually liked it. Very concise and practical
|
| ps - I didn't know about template "cache" directive
| vundercind wrote:
| Probably 80% of notable performance problems I've seen in the
| kinds of systems that things like Django and Ruby get used for
| have been terrible queries or patterns of use for databases (I've
| seen 1,000x or worse costs for this versus something more-
| correct) and nearly all of the other 20% has been areas that
| plainly just needed some pretty straightforward caching.
|
| The nice thing about that is that spotting those, and the basic
| approach to fixing them, if not the exact implementation details,
| are cross-platform skills that apply basically anywhere.
|
| I actually can't recall any other notable performance problems in
| those sorts of systems, over the years. Those are so common and
| the fixes so effective I guess the rest has just never rated
| attention. I've seen different problems in long-lived worker
| processes though ("make it streaming--everything becomes
| streaming when scale gets big enough" is the usual platform-
| agnostic magic bullet in those cases)
|
| A bunch of TFA is basically about those things, so I'm not
| correcting it, more like nodding along.
|
| Oh wait I just thought of another I've seen: serving large files
| through a scripting language, as in, reading it in and writing it
| back out with a scripting language. You run into trouble at even
| modest scale. There's a magic response header for that, make
| Nginx or Apache or whatever serve it for you, it's a fix that's
| typically deleting a bunch of code and replacing it with one or
| two lines. Or else just use s3 and maybe signed URLs like the
| rest of the world. Problem solved.
| megaman821 wrote:
| I have had to combine files into a zipped file on demand
| before. It is hard to avoid the inherent slowness of that.
| jonatron wrote:
| The magic header is probably X-Accel-Redirect
___________________________________________________________________
(page generated 2024-11-15 23:00 UTC)