[HN Gopher] Django 4.0 will include a built-in Redis cache back end
___________________________________________________________________
Django 4.0 will include a built-in Redis cache back end
Author : stanulilic
Score : 117 points
Date : 2021-09-17 14:44 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| crb002 wrote:
| Django also needs to pin a default Postgres container image, and
| default NGINX image. The default stack should be extensively end-
| to end tested and optimized, including an optional patched
| kernel, cgroup2 settings, and untrusted nsjail worker process for
| handling media files.
| spapas82 wrote:
| This is a welcome addition but isn't such a big deal... There are
| great third-party packages (i.e
| https://github.com/jazzband/django-redis) for whoever needs a
| redis cache backend in a project.
|
| Myself, I just use good ol' memcached for all my caching needs (h
| ttps://docs.djangoproject.com/en/3.2/topics/cache/#memcache...).
| It is rock solid and has never failed me till now!
| JonathanBeuys wrote:
| What is being cached?
|
| I don't understand what there is to cache in a web framework.
|
| Is it to cache rendered pages without dynamic content?
|
| If so, I think it is better to let a CDN handle that. Instead of
| the server that runs the web application.
|
| What am I missing?
| asmosoinio wrote:
| The data being cached does not need to be related to the web
| framework as much as the application logic.
|
| E.g. results from external services, calculations, anything
| that an application might want to cache.
|
| Am working on a somewhat large Django system, and there is a
| ton of different things we cache in Redis.
|
| Also HTML template results as another comment mentioned, but
| mostly something else entirely, related to the application
| logic.
| TylerE wrote:
| Generally the approach is fragment caching... so you're caching
| some block of HTML that would otherwise hit the DB (Think
| something like a block of configurable sidebar links, or a
| "latest posts" block.
|
| HTML, not assets.
|
| Some such setups (like one I wrote for a busy Drupal newspaper
| site way back when) can support clearing relevant cache entries
| on DB write, so you're essentially never serving stale data,
| but can save _substantial_ DB load - and possibly webserver
| load too, depending on how computationally intensive the HTML
| generation is.
| jayd16 wrote:
| I guess this is easier than read replicas? I can't imagine
| there's too much work to be saved by skipping the component
| building.
| TylerE wrote:
| Yes, if the queries involve lots of fiddly calculation (for
| instance, the newspaper system did a lot of fancy stuff
| around calculating good line breaks and wrapping, or the
| queries themselves are complicated, involve subqueries,
| many joins, etc.
| zepolen wrote:
| DB lookup + rendering vs cache lookup + concatenating. Yes
| it adds up.
| MattGaiser wrote:
| Database queries are a big one in some cases.
| matsemann wrote:
| Anything.
|
| What's annoying with Python though is how hard it is to just
| cache stuff in some dict or other data structure, since the GIL
| basically forces each request to have it's separate process
| (instead of separate thread). So non-trivial to share some
| state application wide without having to store it outside the
| application.
|
| Of course a shared cache is sensible when there are multiple
| nodes running the application. But not everything needs a
| global cache either, or maybe the point of the cache was
| exactly to avoid a round trip to something else.
| BiteCode_dev wrote:
| First, there are people that don't want to use a CDN (don't
| trust them with censorship), or can't (local network).
|
| But you can cache other things that are not the entire page
| itself:
|
| - database queries
|
| - user sessions
|
| - api calls
|
| - page fragment
|
| - function memoization
|
| What's more, cache backends have the ability to expire things,
| which can be helpful for all sort of stuff.
| nicwolff wrote:
| Nuts, I was hoping that this was an evolution of
| https://github.com/Suor/django-cacheops which uses Redis to cache
| and invalidate ORM result sets by an abstracted representation of
| filter values. It works, but has a lot of magic (as is the Django
| way) and takes some tending-to in production.
| rufugee wrote:
| Interesting...I recall that Django always set itself apart from
| Rails by _not_ having a lot of magic. Has this changed?
| benatkin wrote:
| At surface level, Redis isn't currently on this page, but will be
| there when the 4.0 version comes out:
| https://docs.djangoproject.com/en/3.2/topics/cache/
|
| However, it seems this will enable the community to establish
| using redis as a best practice. So it will be a bigger deal than
| adding just another cache backend.
|
| This is good because the default is local-memory caching, and
| redis can easily beat that.
| https://docs.djangoproject.com/en/3.2/topics/cache/#local-me...
| whoknowswhat11 wrote:
| My understanding is Django 4.0 would include SUPPORT for a redis
| cache backend using a subclass of
| django.core.cache.backends.BaseCache
|
| Now they are shipping a built in Redis cache back-end?
|
| That would be so totally crazy that I am having trouble believing
| this headline.
|
| How about "Django 4.0 will include built-in Redis cache support"
| cat199 wrote:
| > Now they are shipping a built in Redis cache back-end?
|
| no. actual link as you surmised states: This
| PR aims at adding support for Redis to be used as a caching
| backend with Django. As redis is the most popular caching
| backend, adding it to django.core.cache module would
| be a great addition for developers who previously had
| to rely on the use of third party packages.
|
| so just a bad link title
| tyingq wrote:
| I was somewhat surprised to see that the django.core.cache
| module didn't support an mmap based cache. The default cache
| is per-process, so it's somewhat limited in effectiveness.
| Something like Redis solves that, but adds another running
| process to manage.
|
| In the PHP world, for example, acpu is a popular user-facing
| cache that uses mmap so that it works cross-process, then you
| can move to redis if you want to scale across more than one
| host.
| BiteCode_dev wrote:
| Sensible decision, although hardly a surprise or a killer feature
| for something that is a major version bump.
|
| I assume the async story is going to be more fun.
|
| By the way, if you are looking for a good key/value store with
| expiration without the need to setup redis, have a look at the
| excellent diskcache:
|
| https://pypi.org/project/diskcache/
|
| It is based on sharded sqlite files, and it's surprisingly
| capable, even in multiprocessing based env like a django site.
|
| Sqlite is really showing up in all sort of use cases these days,
| like huey, a task queue that doesn't need any broker to be
| installed to be useful: https://pypi.org/project/diskcache/
|
| Now things can get fun...
|
| With supervisord (supervisord.org) you can manage multiple
| processes, and with fastapi + starlette, you can serve your WSGI
| site without a proxy pass, including static sites:
| https://fastapi.tiangolo.com/advanced/wsgi/
|
| Then with shiv you can bunddle all that in a pyz file:
| shiv.readthedocs.io/
|
| Meaning you can now just upload a pyz file to your server, run
| python on it, and that's it, you have a full stack deployed, with
| a web framework, db, caching, task queues, static file serving,
| which can handle around 100k visitor each day without a sweat.
|
| No more infra needed. Just Python code, all bundle in one file.
|
| That's really neat.
| lpasselin wrote:
| Last year I tried Huey for a project where usage was very
| lightweight. Performance was more than enough with SQLite.
| Unfortunately Huey with SQLite was not great at handling
| unexpected crashes like server and docker restarts.
|
| Switching to Redis solved all of these issues.
|
| Maybe it's better now.
| metalliqaz wrote:
| Interesting. I'll look into some of those projects.
|
| I think you put the wrong link for Huey, btw.
| BiteCode_dev wrote:
| Yep: https://huey.readthedocs.io/en/latest/
| midrus wrote:
| But then how do I do microservices in a monorepo based SPA
| server rendered running on kubernetes?
| zentrus wrote:
| Totally agree on hoping there is at least _some_ movement
| forward on async--particularly async ORM. I love Django and
| DRF. It is my go-to for a lot of things, but the lack of async
| support is increasingly a problem for more organizations.
| Honestly, it would not surprise me that if a solid, kitchen
| sink web framework emerges in golang, Django will be abandoned
| rather quickly.
|
| 2 cents.
| jgb1984 wrote:
| I've been writing web applications using python and django
| for close to 15 years now. Smooth, concise and stable. Doing
| the same using a verbose and boilerplate heavy language like
| golang doesn't sound attractive to me, at all.
|
| Not sure I understand the fuss about upgrades either, at my
| current job I've gone from python 2 to 3 and from Django 1.8
| all the way to 3.2 LTS without too many headaches. The
| release notes provide detailed instructions on what is
| deprecated and what you should do to fix it.
| arcturus17 wrote:
| I love Django but it seems like such a limitation for certain
| workloads. You can often work around some of these issues
| with Celery but still...
|
| .Net already offers a kitchensink with async, it's growing a
| lot and is much more attractive than a decade ago.
| midrus wrote:
| Yes, maybe in 10 years when this Go framework catches up with
| all the features and becomes as battle proven, well
| documented, maintained and funded as Django and somehow we
| know is not yet another thing that will be abandoned next
| year.
| AaronFriel wrote:
| Django is, of course, an open source project and the core team
| and contributors are largely free to work on whatever they
| prefer.
|
| However, as a user of Django I sincerely wish they would focus
| more on stability:
|
| Changing jsonb deserialization in a patch release, 3.1.1:
| https://code.djangoproject.com/ticket/31956#comment:16
|
| Queryset .get() can cause OOM, ticket has gone back and forth,
| problem existed as recently as Django 2.X:
| https://code.djangoproject.com/ticket/6785
|
| Django's cascade options differ from the database's, are manually
| implemented: https://code.djangoproject.com/ticket/21961
|
| Can't update keys via ORM, related to the above:
| https://code.djangoproject.com/ticket/21265
|
| Inconsistent behavior around constraints checking in unit tests,
| can lead to inexplicable behavior and difficulty testing when a
| constraint is violated:
| https://code.djangoproject.com/ticket/11665
|
| In general, the ORM functionality and the inconsistency, bugs
| being introduced in patch releases has me looking elsewhere for
| all new development and recommending to other engineers the same.
|
| We have a running joke in our engineering team that every bug we
| hit in Django will have an open or "closed wontfix" issue 5-10
| years old.
| WesleyJohnson wrote:
| I really wish they'd prioritize multi-column primary keys. I
| don't have a ticket handy, but it's been out there for many,
| many years as well. I feel like it would be huge for projects
| built on legacy database, where you have to rely on existing
| schema.
| ldiracdelta wrote:
| I love Django, but version upgrades are a gigantic hassle.
| hdjjhhvvhga wrote:
| What would you suggest as an alternative for a new project?
| rubyist5eva wrote:
| Are you married to Python? Ruby on Rails is mature, with a
| large set of useful libraries.
| ldiracdelta wrote:
| We're not married, but we're "living in sin"... Once
| python shows you that blocks are _always_ encoded with
| indentation for the human readers, then braces or beg
| /end are just static/visual interference on top of your
| code that makes it harder to read. Going from python back
| to braces or beg/end is like going from DVD back to VHS.
| You didn't notice how bad the static on VHS was until DVD
| showed you a better way. :-D
|
| <touches off small fire>
| rubyist5eva wrote:
| funny, I feel the same way about Ruby going to Python - i
| don't like list comprehensions, for example - feels so
| crusty and limited compared to yielding blocks in ruby
| lowercased wrote:
| I guess I've tended to be more focused on how much I can
| produce and the quality of the ecosystem vs that level of
| syntax. My limited experience is that other frameworks,
| languages and ecosystems allow for more productivity
| overall. The few places I know using Django are not as
| productive as the orgs I know using either Laravel or
| Rails. There may be some 'chicken/egg' stuff going on,
| but... if lack-of-braces means _that much_ to you, that
| you 'll cut off using other tech stacks altogether, so be
| it.
| ldiracdelta wrote:
| Heheheh. I'm a code craftsman and certain tools I enjoy
| more than others. It's OK to love the heft and feel of a
| model-77 wormdrive skilsaw and also to enjoy the
| cleanliness and readability of python code. Right tool
| for the job, but ceteris paribus, I prefer python. The
| tool feels good in your hands. I regularly code in C++,
| javascript, Rust, and python so I haven't joined a cult,
| but yes, I do enjoy python.
| ldiracdelta wrote:
| Django is still my go-to. Specifically [Django-REST-
| Framework](https://www.django-rest-framework.org/) with a
| front-end written with [react-
| static](https://github.com/react-static/react-static).
|
| Django's ORM is so nice and the ecosystem around it rocks.
|
| Its biggest downside is painful upgrades. They don't really
| follow [semantic versioning](https://semver.org/)
| matsemann wrote:
| To me however the ORM is one of the most annoying things.
| Not a fan of ORMs in general, but Django's implementation
| promotes a usage pattern where django-models are passed
| around everywhere, leading to hard to untangle
| dependencies between services once they do literally
| anything with the objects.
| brianwawok wrote:
| I really enjoy the Django ORM, and it is perhaps the main
| reason I still use it today. Right mix of simplicity and
| power.
| radus wrote:
| I'm relatively new to Django (longtime Flask user), and
| what struck me about the Django ORM so far is that it
| seems much easier to write extremely low performance
| queries. I didn't really feel this way when using
| SQLAlchemy or peewee, but perhaps that's just my
| inexperience.
| fb03 wrote:
| we are using FastAPI + SQLAlchemy here.
| Daishiman wrote:
| Is there a framework that does them better? The 1.6
| transition was the only one that took more than a day for me.
| Klonoar wrote:
| I'll second this - I maintain a few different Django
| backends and it's been smooth sailing upgrading most of
| them.
| ldiracdelta wrote:
| Yeah. Tons of other frameworks do it better. React for
| instance. Any framework that actually follows
| [semver](https://semver.org/) and really cares deeply about
| backwards compatibility. I've hit backwards incompatibility
| on almost every upgrade. Not just major 2->3, but each
| minor release.
|
| I still love love love Django, but I do wish upgrades were
| easier.
| wilsonfiifi wrote:
| I think Django needs to "fix" serving static assets in production
| without the need to muck about with witenoise or nginx settings.
| leetrout wrote:
| Interesting decision. The mailing list is usually full of "things
| are better served by third party packages" and divesting core of
| stuff like this.
|
| Ive not been following the mailing list, however, so I'll have to
| go back and see what the justification was...
| rafaelturk wrote:
| Redis everywhere!
___________________________________________________________________
(page generated 2021-09-17 23:02 UTC)