[HN Gopher] 12 requests per second: A realistic look at Python w...
___________________________________________________________________
12 requests per second: A realistic look at Python web frameworks
Author : gilad
Score : 462 points
Date : 2021-02-19 02:21 UTC (20 hours ago)
(HTM) web link (suade.org)
(TXT) w3m dump (suade.org)
| Laminary wrote:
| In my benchmark testing, SSL appears to be the bottleneck; e.g.,
| Apache vs. Nginx does not really matter. I assume the benchmarks
| above 10,000 RPS are not using SSL and regular HTTP? How are
| people doing benchmarks at 10k-100k RPS?
| ev1 wrote:
| HTTP/3 is 0-RTT. You only need to connect once, also UDP
| rovr138 wrote:
| Where is SSL the bottleneck? Wondering if terminating earlier
| and just relying on HTTP after would help.
| wahern wrote:
| Signing, verification, and key exchange are quite expensive.
| ECC doesn't help server-side; it mostly reduces client-side
| verification costs. Session caching can be an important
| optimization that can significantly reduce those costs, but
| scaling session caching has its own problems.
|
| But IME real-world bottlenecks have more to do with overall
| architecture. People tend to heavily focus on technical
| details, such as concurrency architecture--the _how_. But the
| biggest opportunities for improved performance usually
| involve functional aspects of an architecture--the _what_.
| (Note that these aren 't fixed categories; they're relative
| positions. A technical detail often becomes a functional
| model as development progresses.)
|
| 12 RPS is a long way from implicating SSL. If you get to the
| point where SSL is an identifiable bottleneck, you've either
| made a series of tremendously good decisions or exceptionally
| poor decisions.
| toast0 wrote:
| > Signing, verification, and key exchange are quite
| expensive. ECC doesn't help server-side; it mostly reduces
| client-side verification costs.
|
| I don't think that's right. Cloudflare's blog [1] says they
| can do about 9.5x the handshakes/sec with ECDSA at 256-bits
| vs RSA at 2048. Verification for ECDSA signatures are
| somewhat slower, but it's usually an acceptable tradeoff to
| make clients do a bit more work so that servers do a lot
| less.
|
| I agree though, at 12 RPS, TLS isn't the bottleneck.
|
| [1] https://blog.cloudflare.com/ecdsa-the-digital-
| signature-algo...
| wahern wrote:
| Ah, you're right. 1) I got it backwards, verification is
| much faster with RSA, so RSA is better for clients). 2) I
| was testing libressl (macOS, OpenBSD), where signing
| speeds between rsa2048 and ecdsap256 are nearly identical
| (Core i5, M1, AMD GX-412T), whereas with OpenSSL (AMD
| EPYC) ecdsap256 is faster (30x advantage to ecdsap256
| actually, as compared to 4x verification advantage to
| RSA). Though, the magnitudes here seem to be sensitive to
| optimization effort.
| kijin wrote:
| HTTP/1.1 Keep-Alive helps a lot. HTTP/2 is even better. You
| only do the SSL handshake once.
| notafraudster wrote:
| Flipping the HTTP/2 switch was amazing for me -- I have a
| page that for reasons needs to load ~500 small images.
| Initially I was worried about having to figure out a sprite-
| based method to compile the images (which is not ideal
| because there are many permutations of which 500 images), but
| when I turned on HTTP/2, the overhead just disappeared. The
| images load instantly. I'm nowhere near the multi-k RPS
| metrics as above, but it was night and day for me even for
| individual requests.
| Ueland wrote:
| Now web-developers just need to realize that much of the
| value in using CDN's is gone. Earlier years it made sense
| to spread content over multiple domains, like a CDN domain
| for static content. Now due to HTTP2, as much as possible
| should be served on the same domain to get the full HTTP2
| effect.
| gwittel wrote:
| SSL handshaking is definitely a bottleneck. I can't speak for
| the benchmarks but with SSL typically this scales via using
| persistent connections (keep alive), and SSL session resumption
| (much lower cost to setup vs a full handshake).
|
| In the end you want to measure perf with and without SSL so you
| can identify the real bottleneck. Otherwise you might just
| benchmark your SSL implementations handshake performance and
| not what you really want.
|
| As an example, I found that Kubernetes nginx-ingress can't
| cache SSL sessions on the upstream side (nginx to your app). So
| request bursts can really hurt your application unless your
| pool has enough open connections to handle the burst (keep
| alive and keep alive requests). Without benchmarking my app in
| different ways I wouldn't have figured this out as easily and
| might have just assumed my app was slow.
| crad wrote:
| I get much higher request throughput in my Tornado applications,
| with very low response latencies, strange.
| hgretg3443 wrote:
| C#/ASP.NET is the fastest web framework now:
|
| https://www.techempower.com/benchmarks/#section=test&runid=8...
|
| 7.000.000 requests per second
|
| Even GO can only achieve 4.500.000 million requests per secnod
| being a low-level language, in opposite to high-level C#.
| qw wrote:
| C# is only at number 3 in your list. Both Java and Rust are
| above it in the list.
|
| It's also a very "artificial" benchmark and real world code
| will give different results (if you have static content, just
| put it in a CDN and don't worry)
|
| Other benchmarks from the same site:
|
| - JSON Serialization: C# is number 34
|
| - Single query: C# is number 23
|
| - Fortunes: C# is number 7
| arcturus17 wrote:
| It ranks 8 in the composite (which adds a layer of human
| opinion, since they weigh each of the benchmarks...)
|
| Java and Rust are both above, but my takeaway is that it
| seems to be by far the best-performing batteries-included
| framework according to the benchmark. It leaves the likes of
| Rails, Laravel, Django, Phoenix, Spring or Nest.js in the
| dust.
|
| Is there something in the benchmark that favors .net core
| above all the others?
|
| I'm genuinely asking, I've never even tried .net but I like
| full-batteries frameworks and this catches my attention.
|
| [1] https://www.techempower.com/benchmarks/#section=data-r20&
| hw=...
| nodamage wrote:
| It needs to be emphasized how artificial these benchmarks
| really are. Here is the source for the Fortunes C# benchmark:
|
| https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast.
| ..
|
| There's no routing or templating, it just writes a bunch of
| strings. No one would build an actual web app this way.
|
| The only C# benchmarks that are remotely realistic are the
| mvc variants, starting with aspcore-mvc-ado-pg at number 79.
| infensus wrote:
| https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Pro
| j...
|
| Routing is required, but generally the rules allow things
| to be "reasonable" and "acceptable", which lets all these
| weird implementations through.
|
| Honestly, they should remove the "Implementation approach"
| column, because basically every implementation is marked as
| "realistic", making it meaningless.
| nodamage wrote:
| The "routing" implementation is more hard coded string
| matching:
|
| https://github.com/TechEmpower/FrameworkBenchmarks/blob/5
| b0e...
|
| I mean I guess technically that's routing but it's not
| remotely realistic. Only the mvc variants use the actual
| framework's routing system.
| EvilEy3 wrote:
| Go being low-level has nothing to do with performance. They
| deliberately keep feature set on this level, they're not making
| some kind of trade-off.
|
| Also, high performance C# is so low level that you might as
| well write C++. Or you think you will use EF, LINQ and have 7
| millions rps?
| arcturus17 wrote:
| > Or you think you will use EF, LINQ and have 7 millions rps?
|
| I'm imagining not, but it's still comparatively faster than
| most, if not all, "full-batteries frameworks", right?
|
| I'm assuming the use of ORMs and such is more or less uniform
| in the comparison (eg, if they don't use EF for .net, they
| don't use Django ORM either)
|
| The overall performance of .net across these benchmarks
| really catches my attention like it does GP...
|
| I'm looking for a full-batteries framework based on a
| strongly typed language and never in my life I thought I'd
| say this, but it might be time to give .net / C# a whirl?
|
| I've heard really good things about the dev experience from
| people here on HN, F# is a really cool bonus, and the fact
| that it looks at least _comparatively_ performant could be
| the icing on the cake.
| Ambix wrote:
| Why do enyone needs framework to print zillion Hello Worlds to
| the client?
|
| It's more interesting to see results of high-load DB tests, for
| example:
|
| https://www.techempower.com/benchmarks/#section=data-r20&hw=...
| nodamage wrote:
| That 7 million requests per second is achieved by writing a
| hard coded plain text HTTP response string directly to the
| client... it is so far disconnected from any real world use
| case that the number is basically meaningless.
| arcturus17 wrote:
| It's still 8th in the composite benchmark. And the criticism
| you're leveling would affect the entire benchmark design,
| rather than a particular framework score, no?
|
| [1] https://www.techempower.com/benchmarks/#section=data-r20&
| hw=...
| nodamage wrote:
| > And the criticism you're leveling would affect the entire
| benchmark design, rather than a particular framework score,
| no?
|
| Indeed. The problem is that many of the scores in the top
| 100 are really misleading because no one building a web app
| would implement things that way. There is still some value
| in the lower down benchmarks but you have to basically read
| the underlying source to determine if the implementation is
| remotely realistic or not. For starters I would ignore
| anything classified as "platform" which is described as:
| Platform, meaning a raw server (not actually a framework at
| all). Good luck! You're going to need it.
|
| For C# in particular I would only consider the mvc variants
| as realistic.
|
| Edit: I looked into the "asp.net core" composite score a
| bit more, it looks like those benchmarks are based on the
| aforementioned "platform" implementations for each test. I
| actually think this score is even more misleading than the
| individual benchmarks. At least the individual benchmarks
| show you the difference between "aspcore" (platform),
| "aspcore-mw" (middleware-only), and "aspcore-mvc" (full
| framework with routing).
|
| Here are recalculated composite scores based on the more
| realistic implementations (aspcore-mvc, aspcore-mvc-ado-pg,
| aspcore-mvc-dap-pg, aspcore-mvc-ef-pg):
| ASP.NET Core MVC with ADO.NET (raw SQL): 3029
| ASP.NET Core MVC with Dapper: 2591 ASP.NET Core MVC
| with Entity Framework: 2195
|
| Compared to Flask's 468 or Django's 280 it's still
| significantly faster, but not to the same extreme you might
| think at first glance at the chart.
| BiteCode_dev wrote:
| Yes but getting a good ranking at a useless benchmark if
| not very meaningful. It's cool, but I wouldn't use it to
| make any serious decision.
| Thaxll wrote:
| Hmm no they have the same speed with gnet ( Go ):
| https://www.techempower.com/benchmarks/#hw=ph&test=plaintext 7M
| req/sec both of them.
|
| ( all those benchmarks are sort of useless anyway )
| sgtnoodle wrote:
| The #1 project on that list appears to be implemented in rust?
| What does GO being "low-level" have to do with performance of
| serving requests? I'd imagine its bottlenecks would be due to
| something fairly arbitrary, like its garbage collection or how
| it represents strings or something.
| maxpert wrote:
| The fact that you are using offset of 50000 and complaining it
| slows everything down says a lot about the benchmarks. Top it all
| with ORM query with prefetch all, GIL, and shared CPU (I am
| guessing) that you used to run benchmark on. You see where this
| is headed?
| ximm wrote:
| Maybe I am missing something, but why wasn't sanic tested with
| pypy? I expect that this combination would outperform everything
| else.
| erdewit wrote:
| Was wondering the same thing. PyPy can do wonders for other
| asyncio-based frameworks.
| polyrand wrote:
| Related to ORMs/queries/performance, I have found the following
| combination really good:
|
| * aiosql[0] to write raw SQL queries and having them available as
| python functions (discussed in [1])
|
| * asyncpg[2] if you are using Postgres
|
| * Map asyncpg/aiosql results to Pydantic[3] models
|
| * FastAPI[4]
|
| Pydantic models become the "source of truth" inside the app, they
| are designed as a copy of the DB schema, then functions receive
| and return Pydantic models in most cases.
|
| This stack also makes me think better about my queries and the DB
| design. I try to make sure each endpoint makes only a couple of
| queries. Each query may have multiple CTEs, but it's still only a
| single round-trip. That also makes you think about what to
| prefetch or not, maybe I want to also get the data to return if
| the request is OK and avoid another query.
|
| [0] https://github.com/nackjicholson/aiosql [1]
| https://news.ycombinator.com/item?id=24130712 [2]
| https://github.com/MagicStack/asyncpg [3] https://pydantic-
| docs.helpmanual.io/ [4] https://fastapi.tiangolo.com/
| jordic wrote:
| We do something like this, also we took some inspiration from
| hashura. Asyncpg it's so fast an ergonomic.
| hangtwenty wrote:
| Thank you for sharing this stack! I'm a Pythonista at heart,
| recently was trying RxDB + TypeScript, and I was thinking hmm
| I'll bet I could do something with postgres and Pydantic.
| BiteCode_dev wrote:
| I always though it would be a nice alternative to an ORM:
| having a tool that take a marshmallow/pydantic/whatever model,
| optionally passing it additional db specific options, then it
| generates a bunch of sql files you can call with aiosql. The
| whole things would then let you optionally get the result
| wrapped in a model if you need to, with ORM like helpers for
| common CRUD things.
|
| That would have the benefit of the standardized api of an ORM
| and the flexibility of SQL, without the coupling.
| mixmastamyk wrote:
| Do you need to define your models more than once with these?
| I'm looking for a single source solution and haven't quite
| found it yet.
| jononor wrote:
| Is there any automatic glue between SQL and Pydantic, or is
| each mapping hand-written?
| pantsforbirds wrote:
| The asyncpg library is honestly incredible. I wrote a backfill
| script that would: 1. dump the rows of a postgres table
| matching a query (usually a range on the index with a filter or
| two on other columns) 2. Do some very basic transformation on
| the rows (few replaces with small regex) 3. Take each
| transformed row and dump into a rabbitmq queue.
|
| I was using aio-pika for the rabbit queue and asyncpg and was
| getting a consistent 25k messages/sec for like 200 lines of
| code.
| fokinsean wrote:
| This sounds like a really cool stack. I'm testing the waters
| with FastAPI and SQLAlchemy right now, but SQLAlchmey feels
| like it just gets in the way.
|
| Do you have an example project which uses all of these I could
| look at?
| mlthoughts2018 wrote:
| I think it's been bog standard practice to run flask via uwsgi or
| gunicorn with async workers and use multiple process based
| workers per deployed server unit (eg per pod in Kubernetes).
|
| What matters is that the cumulative latency & throughput solve
| your problem, not how fast you can make one singular async worker
| thread.
|
| I figure most people running complex web services in production
| would just do an eye roll at this post. Nobody's going to switch
| to PyPy for any of this.
|
| My team at work runs several complex ML workloads, and we use the
| exact same container pattern for every service running gunicorn
| to spawn X async workers per pod and then scale pods per service
| to meet throughput requirements. Sometimes we also just post
| complex image processing workloads to a queue and batch them to
| GPU processor workers. In all these use cases, super low effort
| "just toss it in gunicorn running flask" has worked without issue
| for services supporting up to peak load of thousands to hundreds
| of thousands of requests per second.
| 7kmph wrote:
| Could you share your company's website?
| mlthoughts2018 wrote:
| No, I can't speak on their behalf on Hacker News, so it is
| important to me to stay disconnected from my employer.
| ctvo wrote:
| It's a bit of a step back in time reading things like this.
|
| This is stateless HTTP requests hitting a relational database.
| How is this dead horse still being beaten? The patterns for load
| balancing, horizontal scalability, caching in this space well
| documented.
|
| What are we gaining still profiling Django, Flask and Ruby on
| Rails in 2021.
| fouc wrote:
| Does this mean you work with a stateful websockets setup? What
| stack?
| jtdev wrote:
| I suppose every app you work on runs the same query repeatedly?
| Yes, load balancing makes sense, but the author is specifically
| looking at requests per thread, i.e., wouldn't it be great (and
| more cost effective) to get as much throughput from a single
| thread as possible?
| forgotmypw17 wrote:
| I have to question the value of text written by someone who sets
| white-on-white text in their website...
| Jonnax wrote:
| The background is blue?
| forgotmypw17 wrote:
| The background image is blue, the background itself is white.
|
| Without loading the image, the text is the same color.
| throwawayffffas wrote:
| I question the value of the comment of someone who loads
| css but not background images.
| robertlagrant wrote:
| Note: SQLAlchemy 1.4 is async.
| https://docs.sqlalchemy.org/en/14/changelog/migration_14.htm...
| gchamonlive wrote:
| I have great experiences with Falcon for backend REST APIs, and
| it is supposed to be great in terms of requests per second.
|
| How does it compare to Sanic?
| divbzero wrote:
| > _I have great experiences with Falcon_
|
| Then keep using Falcon! OP's core thesis is that developer
| comfort matters more when choosing an API framework because the
| performance bottleneck is usually found elsewhere. The API
| itself should be fast enough with some combination of Gunicorn,
| gevent, PyPy, and horizontal scaling.
| gchamonlive wrote:
| I just wanted to know how it would compare with Sanic,
| because I never used it.
| divbzero wrote:
| The biggest difference is that Falcon is synchronous while
| Sanic is asynchronous. With Sanic, you would explicitly
| specify async/await for asynchronous operations and use
| asynchronous libraries for I/O. Switching to Sanic could
| also affect how you deploy to production. Both are plenty
| fast.
|
| FastAPI [1] is also worth considering if you're looking
| into asynchronous API frameworks. It comes with nice
| features for specifying API schemas.
|
| [1]: https://fastapi.tiangolo.com/
| gchamonlive wrote:
| I prefer async big time. We once had to implement async
| routines while using flask, where the server would return
| a 200 but keep processing the request, and the actual
| result would be sent by email. It was hellish and
| inefficient to make it. In hindsight would have been
| better to use a queue service and a consumer and decouple
| the whole process, even if it meant increased
| infrastructure and maintenance complexity.
| huffmsa wrote:
| I'm a big Falcon user as well. Got into it because it was
| easier to get up and running without bells and whistles vs the
| more "popular" frameworks.
| AtlasBarfed wrote:
| Where is japronto on the techempower? It's not even on there.
| p5v wrote:
| The TL;DR you should be looking for:
|
| > all of this emphasises the fact that unless you have some
| super-niche use-case in mind, it's actually a better idea to
| choose your framework based upon ergonomics and features, rather
| than speed.
| luord wrote:
| And here I was living under the assumption that psycopg2 was the
| only option, and probably the biggest reason I was not using
| pypy. Gotta take a look at pg8000.
|
| In general, I've always liked the idea of pypy, so I'll try to
| use it more, and not just for performance. Will also donate when
| I can.
| ramraj07 wrote:
| Don't forget that you're paying a huge price using the sqlalchemy
| orm - https://docs.sqlalchemy.org/en/13/faq/performance.html
|
| If I know an endpoint is going to be hit hard, I forgo trying to
| use the ORM (except to maybe get the table name from the model
| obj so some soul can trace it's usage here in the future) and
| directly do an engine.execute(<raw query>). Makes a huge
| difference. Next optimization I do is create stored procedures on
| the database. Only then I start thinking about changing the
| framework itself.
|
| For folks like me who want to get prototypes off the ground in
| hours, flask and fastapi are godsend, and if that means I have to
| worry about serving thousands of requests a second soon thats a
| happy problem for sure.
| nwsm wrote:
| For me, FastAPI is the sweet spot between performance and quick
| development.
| welder wrote:
| I'll happily forget that because it's such a small microscopic
| price that it's moot. You're way better off optimizing the
| actual query being made, which SQLAlchemy is great at because
| it doesn't hide the SQL from you. Don't use engine.execute(<raw
| query>), use SQLAlchemy Core if your endpoint is getting
| hammered.
| paulmd wrote:
| ORMs aren't inherently that heavyweight, as a Java developer I
| don't have performance concerns about hibernate.
|
| That sounds to me like a Python problem and a "this specific
| ORM isn't performant" problem, not ORMs being bad as a whole.
| Python has never been the fastest language (it's far slower
| than, say, Java) and the GIL really prevents applications from
| scaling well without multiple instances.
|
| And if you really want a "just load the data for me and do
| nothing else that incurs a performance hit" approach then you
| can use stateless objects and the ORM truly becomes just a
| wrapper around the DB to load and transform the data into an
| object for you and/or do a raw, whole-object update back to the
| DB.
| kaba0 wrote:
| Totally agree with you, only want to add that unfortunately
| the team doesn't really know the given ORM they use, and I
| have seen some utterly stupid uses of eg. Hibernate (like
| eager fetching basically everything there is for queries
| where it is not required, or simply not knowing anything
| about the boundaries where an object is "attached" or not).
| Which one might argue that it is a defect in the tool, but I
| doubt you would blame an airplane for crashing when the
| "pilot" is not trained to drive it.
| ramraj07 wrote:
| It's not just sqlalchemy but yes it is definitely a python
| problem. Problem or feature is up for discussion of course.
| The ability to mutate basically anything is powerful and you
| gain some tangible benifits from it.
| vosper wrote:
| You can also use SQLAlchemy Core, which is an intermediate
| between the full-blown ORM and running actual strings of SQL.
| I've had a great experience with Core - I can easily have it
| output essentially the exact SQL I'd write by hand, but I get
| many benefits (like the ability to compose queries) that are
| nicer than dealing with raw SQL.
| SatvikBeri wrote:
| Definitely agree, I'm happy to write raw SQL, but SQLAlchemy
| Core is even better than that because of composability.
| smoe wrote:
| This is what I like most about SQLAlchemy, I can use the ORM
| most of the time, but in cases where it needs optimizations
| or aggregation queries I can drop down or refactor to core,
| while still using the same model/table definitions, ability
| to combine or filter queries dynamically and leverage IDE
| features like refactoring.
| burnthrow wrote:
| To be clear, this is FUD. If you know how to make SQA emit the
| right SQL, the performance is basically the same as psycopg2 +
| your custom code, usually better. I've written many high volume
| SQA services and never once saw SQA per se as the bottleneck.
| KaiserPro wrote:
| We did an evaluation for our API. The API accepts an image
| upload, passes it onto the backend for processing and returns a
| ~2k json lump in return.
|
| Long story short, fastapi was much much faster than anything else
| for us. It also felt a bit like flask. The integration with
| pydantic for validating dataclasses on the fly was also great.
| sriku wrote:
| A humble request to folks making benchmark or other graphs -
| please understand that thin coloured lines are not easy to
| visually parse .. even for folks like me who aren't totally
| colour blind but have partial red-green colour blindness. At
| least, the lines can be made thicker so it is easier to make out
| the colours. Even better, label the lines with an arrow and what
| they represent.
| TickleSteve wrote:
| Absolutely! I have mild red/green issues and could barely tell
| those lines apart.
| lrossi wrote:
| This benchmark was run on a laptop, which has a very small number
| of cores compared to the servers that usually run such apps. The
| author doesn't mention any attempt to tweak the number of
| workers, which would make sense in this case. Given that they did
| notice at some point that CPU usage is lower than expected, I am
| surprised that they did not try it.
| sandGorgon wrote:
| TLDR - pypy is awesome. Dont use frameworks. Use pypy.
|
| Please donate. Pypy needs funds - https://opencollective.com/pypy
|
| Pypy doesnt get a fraction of the funding that python does.
| helsinkiandrew wrote:
| Since I've been a developer there have been two changes that I
| feel have given major performance improvements and made backend
| framework improvements much less significant (atleast in the apps
| I develop): CDNs and client side rendering (that means the more,
| smaller requests for data which are more suited to be served via
| a CDN)
|
| Using (for example) AWS Cloudfront was a gamechanger in how I
| design webapps and view performance. Being able to 'slice and
| dice' what requests get SSL terminated at the CDN, cached fairly
| locally, served from an Amazon managed webserver, or sent to our
| app server, increased our performance 10 fold.
|
| That approach isn't always practical, but I find that it's now
| much easier to choose the backend for developer performance and
| doubling the server CPU/memory is quicker and cheaper when
| needed.
| jaimex2 wrote:
| When you start hitting bottlenecks in your python web framework
| its probably time to switch to a faster language, not another
| framework in python.
|
| You're probably done rapid prototyping by this point anyway.
| antihero wrote:
| Well if you convert everything from ORM to raw SQL, it will
| then be easier to extract all of that SQL and use it in a
| different web framework once you've measured and confirmed that
| your bottleneck is servicing requests in Python.
| speedgoose wrote:
| I don't think there is much gains to rewrite everything in a
| faster language. Unless they are a very very successful company
| with billions of customers, it's often cheaper to scale
| horizontally.
| seanparsons wrote:
| Depends on if you identify it early enough really and even
| then halving your app tier costs could be sizable for any
| company. If you can get some productivity benefits at the
| same time from say static typing then there's even more
| reason to switch.
| square_usual wrote:
| There's still a good reason to pick fast frameworks in a slow
| language: you can delay the inevitable for a bit, probably
| enough time for you to work on a rewrite or whatever.
| cryptos wrote:
| Why Python at all? About 10 years ago I liked Python a lot (and
| still like it in principle) and felt very productive compared to,
| say, Java. Java was full of inconvenience, XML, bloated
| frameworks and all that. But today you can use Kotlin, that is in
| my opinion even nicer than Python, with performant frameworks (e.
| g. Quarkus or Ktor) on the super fast JVM.
|
| I don't want to start a language war, but maybe Python is not the
| first choice for their requirements.
| rkangel wrote:
| I agree - if you have any chance at the slightest volume (and
| based on this we're talking 100 concurrent users, not a
| million) then you are handicapping yourself if you're stuck on
| a single thread.
|
| Personally I would suggest Elixir (and the Phoenix web
| framework). It's fully 'parallel by default', will happily
| saturate all your cores and serve tens of thousands at the same
| time. The language is simple, the framework is good, although
| obviously not as many people know Elixir as Python.
| kavalg wrote:
| I can code comfortably in Python, Java, JavaScript and to some
| extent C/C++. In the last 4 years I have been using mostly
| Python for various reasons (Machine Learning, OS automation,
| web scraping ...). Compared to the other languages, Python
| feels lighter and faster to write to the extent that it rarely
| interrupts my flow of thoughts. Now and then I have to code in
| JavaScript (frontend), C/C++ (embedded / low level
| optimizations) and Java (maintenance) and my feelings are:
|
| - JavaScript still feels messy
|
| - C/C++ is complex, but it is often offset by the
| complexity/needs of the project (e.g. in embedded)
|
| - Java has become kind of bloated with all the new stuff
|
| So as others have already mentioned, development productivity
| is in many cases far more important than speed of code. In my
| ~20y long career I have rarely seen a project failure due to
| runtime performance. Most of them failed due to speed/agility
| of development iterations and also project/product management
| issues (bad fit, unrealistic project plan, lack of focus and
| customer feedback).
|
| That said, if I need to look for another language due to
| performance, that would probably be Rust.
| huffmsa wrote:
| I also notice I spend less time thinking "how do I do X in
| this language" with Python and just do it. Maybe not the best
| implementation of X, but that doesn't really matter because I
| implemented X, Y and Z while someone else got bogged down in
| their "better" language and barely got X across the finish
| line.
| arcturus17 wrote:
| I agree with your general take on developer productivity, but
| I don't feel that modern JS is significantly messier than
| Python, at least not to a level where it significantly
| impacts productivity (I'd rather avoid a debate on the
| abyssal depths of the language, eg, type coercion)
|
| I feel about the same amount of grievances with both. For
| instance I dislike Python's async and functional semantics (
| _list(map(lambda n..._ ). But it has a much better standard
| library overall, a lot of the scripting syntax semantics
| (file opening, requests, etc.) feel cleaner, etc.
|
| I'm more versed in JS but as the knowledge curves converge in
| months or years to come I don't think I'll be significantly
| more productive in Python.
|
| I know you were expressing an entirely subjective opinion,
| probably contingent on the amount of day-to-day practice you
| have with each language (you say you code JS in the front-end
| now and then... I write a lot of it), but I still wanted to
| offer my counterargument.
| kavalg wrote:
| Thanks for the JS perspctive. Since my frontend work mostly
| involves hacking together prototypes and trying to
| integrate some JS libs in as quickly as possible, I guess I
| am kind of biased.
| randomsearch wrote:
| JS feels very mess to me too. I think the main reason is
| it's so variable - things like modules - whereas Python
| everything is mostly just so consistently... pythonic.
| BiteCode_dev wrote:
| > list(map(lambda n...
|
| List comprehensions are much better for this. Functional
| doesn't mean you have to use a function call. If you can
| use the paradigm with literal syntax, just do so.
| arcturus17 wrote:
| I never said functional means you necessarily have to use
| a function call, and I also understand list
| comprehensions are semantically functional, but sometimes
| you do need to use a function call.
| pas wrote:
| They don't really compose nicely. (At least from my point
| of view, as I prefer fluent interfaces, eg. those you
| usually find in Rust/Scala/Java.)
| galacticdessert wrote:
| I feel like when you need to create value and make a product,
| better to do it in a language you know. Product value is not
| directly proportional to product performance or raw HTTP
| request serving time.
|
| I for instance know a bit of F#/C# and Java, but would probably
| pick Python to make a new product just to remove that mental
| barrier of not having my lack of language knowledge in the way
| of things
| tubbyjr wrote:
| My god, the CSS and styling on that page is absolutely abysmal
| dfgdghdf wrote:
| I would question choosing Python for large server projects
| because the performance ceiling is so low. At least with the
| "middle tier" performance languages such as Java / C# you are
| unlikely to require a complete language switch as the project
| scales.
| musingsole wrote:
| With cloud and infinite horizontal scaling...hitting that
| performance ceiling is an indication of a poor design.
| dfgdghdf wrote:
| This is false, since it is not always easy (or even possible)
| to horizontally scale a work-load.
| musingsole wrote:
| If you've made it impossible or just difficult to scale
| horizontally in the server world, yeah, you've got a bad
| design.
| dfgdghdf wrote:
| Sorry to keep coming back, I don't want to start a flame
| war here. Difficulty of horizontal scaling is a property
| of two things: 1. how your solution is
| designed (your point, and I agree this is often done
| poorly) 2. the problem / work-load you are trying
| to scale (my point).
|
| It might be that your problem does not "shard" very
| easily. You cannot fix this with solution architecture,
| at least not easily. Horizontal scaling of a relational
| database is very difficult, for example.
|
| Edit:
|
| Another example. Can you rewrite NGINX in a slower
| language and use horizontal scaling to fix it? Of course
| not, because that horizontal scaling would itself
| leverage NGINX (or something like it)!
| tnash wrote:
| Use of ORMs is often a performance choke point. Raw DB queries
| are often much, much faster. Almost always, the more you
| abstract, the worse you perform. It's great as a developer but
| not so great as a user.
| nmfisher wrote:
| I haven't touched an ORM in over 6 years, but unless they've
| improved since then, I honestly can't think of a single reason
| why anyone would choose to use one.
|
| They're clunky monstrosities that act only as guard-rails for
| inexperienced developers. Far better to invest a few days
| (which is realistically all you need) to improve their SQL
| skills and/or code-review practices.
| kaba0 wrote:
| Yes, things like setString(1, "asd") and conversely getInt(2)
| are so beautiful and will never introduce mistakes /s
|
| ORMs are there for mapping objects and well, relations. Most
| ORMs provide additional features, but at the core they can,
| and for more complicated queries they should be used with
| native SQL queries. They are made for OLTP not for OLAP
| exyi wrote:
| Like there wouldn't be anything in between /s
|
| There are "simple ORMs" that only map results of SQL
| queries to objects. They do not provide a magic query API -
| which is the source of most problems. I don't do Python,
| but for .NET there is Dapper
| https://github.com/StackExchange/Dapper, you can have a
| look what I mean. You write the SQL query, explicitly
| execute it, the library maps the results of that query into
| objects (it's C#, so you have to declare the class. In
| Python I'd imagine it would create the object for you)
| jaywalk wrote:
| +1 for Dapper. It does have some limited "magic" query
| building features, but only for straightforward CRUD
| operations that don't involve joins. And that's a Good
| Thing (TM).
| wenc wrote:
| > I honestly can't think of a single reason why anyone would
| choose to use one.
|
| In the case of SQLAlchemy (referenced in the article), many
| people use it as a database connection abstraction rather
| than an ORM. It's kinda the equivalent of "ODBC" for certain
| Python libraries like Pandas.
|
| For instance, in Pandas you can write your dataframe to the
| database by going `df.to_sql(tblname, sqlachemycon)` where
| sqlalchemycon is the connection instance to any database that
| SQLAlchemy supports.
|
| https://pandas.pydata.org/pandas-
| docs/version/0.23.4/generat...
|
| I use SQLAlchemy for this purpose alone and write straight up
| SQL. I've never used the ORM parts of SQLAlchemy.
| morpheuskafka wrote:
| I am by no means an experienced developer--but the issue I
| always seem to have without using an ORM is that there are
| string literals containing different bits of SQL scattered
| all throughout my code. In addition to being hard to refactor
| when the database model changes, I think there is a fairly
| high performance cost to doing so many string concatenations
| every time the code runs. Is there a better way to manage the
| SQL command strings when you are sending the queries by hand?
| branko_d wrote:
| A lot of times you don't need to change the SQL string at
| all. You just bind different parameter values before
| sending the same SQL string to the DB server over and over
| again.
|
| The exceptions are SQL elements that cannot be (easily, or
| at all) parametrized, such as the column lists in SELECT,
| ORDER BY, GROUP BY, or changing the WHERE "shape" and so
| on.
|
| In my experience, this tends to be a minority of queries,
| although an important minority.
|
| ----
|
| P.S. If string concatenations are your bottleneck, then
| your database is screaming fast! The real-life bottlenecks
| are usually in excessive database round-trips and
| unoptimized query plans, and are orders-of-magnitude
| larger.
| takeda wrote:
| That was my biggest issue with writing own SQL, until I
| started using PyCharm. A while ago they integrated DataGrip
| (I think it is only available in pro version) which makes
| the IDE also understand SQL code[1].
|
| If you connect the IDE to a database it starts to recognize
| the SQL to the point it behaves like rest of the code (you
| have autocomplete etc). I am starting to think that this is
| the correct approach and ORMs were just a hack trying to
| achieve that.
|
| [1] https://youtu.be/2bpmfjtoVVU?t=2831
| antihero wrote:
| I haven't done DB stuff in a while as I've mostly been
| frontend, but I reckon the way I'd lay it out is in the
| same way that I have a "clients" or "services" folder (or
| repo) which contains things that return Promise<Data> (and
| I don't have to care whether their source is HTTP,
| Firebase, or anything really). I would probably do the same
| with my back-end application (or lambda). Directories (or
| repo) full of services which are sets of high level calls
| (e.g. getPotatoes()) which are async functions that return
| data. Inside would be (probably) SQL.
|
| > I think there is a fairly high performance cost to doing
| so many string concatenations every time the code runs
|
| String concatenation is extremely cheap compared to any
| sort of IO or computation.
| 0xkd wrote:
| I disagree; not using ORMs isn't going to magically make
| developers write better queries, why not spend those few days
| training them to use the ORM better? Would you rather have
| raw SQL strewn about the codebase and have to worry about
| input validation and data (de)serialization every single
| place? Maybe it's ok for toy apps, but I wouldn't want
| developers bringing their own different styles of writing SQL
| all over a project. An ORM helps standardise this stuff
| pantsforbirds wrote:
| I honestly would rather just read a SQL query. Almost every
| developer is familiar with SQL so you can immediately know what
| is happening vs if you are looking at a code base with an ORM
| you're not familiar with.
| qeternity wrote:
| As a Django shop, we've always hoped PyPy would one day be
| suitable for our production deployments but in the end with
| various issues we were never able to make the switch.
|
| And then Pyston was re-released...and changed everything. It was
| drop in compatible for us and we saw a 50% drop in latencies.
|
| Source availability aside, I suggest anyone running CPython in
| prod take a look.
| twsted wrote:
| Can you tell us more about that?
|
| I remember pyston v1 from Dropbox. You are speaking about v2,
| which is a binary package (closed-source at the moment)?
| qeternity wrote:
| Yeah it's a closed source binary.
|
| We're very happy with it. Great compatibility, no horrendous
| warm up times, and very meaningful speedups.
|
| Not much more to say. It's the same, just a bit faster.
| hendry wrote:
| I always assumed Python could scale because of Reddit:
| https://github.com/reddit-archive/reddit
|
| Not quite sure if their current site's code is opensource...
| anyone know?
| andsko wrote:
| And because of Instagram: https://instagram-
| engineering.com/python-at-scale-strict-mod...
| segmondy wrote:
| Any language can scale. Pretty much most languages there can
| handle scale. It's usually bad algorithms or external services
| be it API & DB used poorly that impacts performance.
| Terretta wrote:
| Not that it matters any more, but a colleague mentions Flask was
| originally a joke of what not to do:
|
| https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/
|
| Flask author reflects on that here:
|
| http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf
|
| Quite relevant to the conclusion in the article.
| est wrote:
| Might as well refer to TechEmpower benchmarks.
|
| https://www.techempower.com/benchmarks/
| Jarred wrote:
| just.js (#9) looks interesting
|
| https://github.com/just-js/just
|
| It seems to be a much tinier JavaScript runtime than Node.js
| (still using v8), but linux only
|
| The benchmark is probably unrealistically optimized code but
| even so, it implies Node.js itself has a large performance
| overhead
| chmod775 wrote:
| If you're not paying attention it would look that way, yes.
|
| If you pay attention you'll notice that just.js is using
| postgres as their DB, while all of the node benchmarks are
| handicapped by either using mongoose/mongodb or MySQL.
|
| There is no node benchmark with postgres, but _all_ of the
| fastest benchmarks used it.
|
| "lithium" is a good example to show how much of an impact
| switching to postgres has. All of the 4 lithium benchmarks
| are identical except in what DB they use.
|
| The results are: lithium-postgres-batch (#2 - 659850),
| lithium-postgres-beta (#13 - 398773), lithium-postgres (#14 -
| 398258), lithium (#45 - 271989). The last result is MySQL.
| reasonabl_human wrote:
| What makes postgres special in these cases?
| chmod775 wrote:
| I would like to know as well. All I know is that for this
| specific workload there appears to be a strong
| correlation between the chosen DB and performance.
| Depending on the benchmark you're looking at on that
| site, the top 30-60 results are _exclusively_ postgres.
|
| Further, for every framework/language that was tested
| with both MySQL and postgres (there's quite a few of
| them), the postgres one always ranks higher.
| ahoka wrote:
| "In this test, the framework's ORM is used to fetch all
| rows from a database table containing an unknown number
| of Unix fortune cookie messages (the table has 12 rows,
| but the code cannot have foreknowledge of the table's
| size)."
|
| This is the "workload".
| est wrote:
| > the top 30-60 results are exclusively postgres.
|
| Perhaps it's because pg had better async drivers?
| edoceo wrote:
| Likely it's the better/faster support for multi-step
| transactions. (That's a big point for PG, since at least
| 2004)
| zerd wrote:
| Somehow just.js is faster than Rust and C++ in 20-queries
| benchmark, they're all using postgres. https://www.techempo
| wer.com/benchmarks/#section=data-r20&hw=...
| Spiritus wrote:
| It's likely because of this (custom Postgres client,
| request pipelining and not doing sync/commit after each
| query):
|
| > [Multiple Queries] This is the first test where
| Just(js) has quite a big lead. This is likely due to the
| fact it is using a custom postgres client written in
| Javascript and taking full advantage of pipelining of
| requests. It also avoids sending a Sync/Commit on every
| query. As far as I am aware this is within the rules but
| will be happy to make changes to sync on every query if
| it is not.
|
| https://just.billywhizz.io/blog/on-javascript-
| performance-01...
| Spiritus wrote:
| The author wrote a blog post about it:
| https://just.billywhizz.io/blog/on-javascript-
| performance-01...
| ancount wrote:
| I inherited a flask queue worker, and it suffers from some major
| problems (like 12 req/second when it's not discarding items from
| the queue). I am primarily a javascript programmer so I'm a
| little bit out of my element.
|
| I am tempted to refactor the worker to use async features, and
| that would require factoring out uWSGI, which is fine, I only
| added it last week. The article states that Vibora is a drop in
| replacement for flask, but I guess I'm a bit skeptical, as I
| can't find much information outside of Vibora having a similar
| api. For a web service with basically one endpoint, I could
| refactor to another implementation fairly easily, I'm just
| looking for the right direction.
|
| I thought maybe I should refactor the arch to either batch
| requests to the worker, or to use async. Anyone have a feeling
| where I should go? I am just getting started researching this,
| but any advice would be appreciated.
|
| Edit: at least quart has a migration page.. probably will just
| try it out, what can I lose?
| https://pgjones.gitlab.io/quart/how_to_guides/flask_migratio...
|
| Second edit: Also might try out polyrand's stack in the comments.
| imperio59 wrote:
| My experience doing perf optimizations in real world systems with
| many many people writing code to the same app is a lot of
| inefficiencies happen due to over fetching data, inefficiencies
| caused by naively using the ORM without understanding the
| underlying cost of the query, and lack of actual profiling to
| find where the actual bottlenecks are (usually people writing
| dumb code without realizing it's expensive).
|
| Sure, the framework matters at very large scale and the benefits
| from optimizing the framework become large when you're doing
| millions of requests a second over many thousands of servers
| because it can help reduce baseline cost of running the service.
|
| But I agree with the author's main point which seems to be that
| framework performance is pretty meaningless when comparing
| frameworks if you're just starting on a new project. Focus on
| making a product people wanna actually use first. If you're lucky
| enough to get to scale you can work about optimizing it then.
| vp8989 wrote:
| "Focus on making a product people wanna actually use first. If
| you're lucky enough to get to scale you can work about
| optimizing it then."
|
| This seems like a false dichotomy. Avoiding obvious performance
| mistakes such as the ones you mentioned does not require
| additional _focus_ that would detract from general building. It
| just requires that you know what you are doing.
|
| If you are the type of person who makes said mistakes, its
| unlikely you would ever go back and fix them by "focusing" on
| performance because the issue is simply that you don't know
| what you don't know. Likely someone else will come along in
| future and point out your mistakes to you.
|
| Optimization that actually hinders you from building and
| requires focus is at the very margins and almost no one is
| going to those levels in typical "application" code.
| devnonymous wrote:
| > My experience doing perf optimizations... without realizing
| it's expensive).
|
| Completely agree and this has been my experience ae well. To
| this I'll also add, inadequate thought put into data modelling.
| One would have to think lesser about query performance or cost
| of overfetching if data is modelled around the needs of the
| system it would serve instead of just modelling real life
| entries and their relationships as is, straight onto the
| database.
| stevenjohns wrote:
| There is lots of truth to this. Some ORMs like Django perform
| joins in very unsuspecting ways.
|
| A simple example is, say, foreign keys. Trying to access the
| foreign key of an object by doing `book.user.id` does an
| additional query for the user table to get the ID. It's less
| known that the id is immediately available by just doing
| `book.user_id` instead.
|
| I've spent time optimising things like text searches down from
| 2000+ queries to about ~4, and one of the more noticeable
| things to me isn't actually the number of joins, rather the
| SELECT's that take place. Many of these ORMs do a SELECT *
| unless you explicitly tell them to otherwise, and when dealing
| with large-ish datasets or on models that have large text
| fields this translates into significant time taken to
| _serialise_ these attributes. So you can optimise the query and
| still have it take a long time until you realise that limiting
| the initial `SELECT` parameter is probably more efficient than
| limiting the number of joins.
| paulmd wrote:
| > A simple example is, say, foreign keys. Trying to access
| the foreign key of an object by doing `book.user.id` does an
| additional query for the user table to get the ID. It's less
| known that the id is immediately available by just doing
| `book.user_id` instead.
|
| Hibernate (on Java) at least optimizes this specific use-
| case. At first, accessing a lazy-loaded property-object will
| give you a "proxy" and you can access the ID without
| incurring a database load (since it knows that anyway). And
| when doing a query, the object won't be joined when
| requesting book.user.id unless it needs to be (like you have
| some other WHERE clause that requires an actual join on that
| row).
| gregmac wrote:
| The most insidious part about misusing ORMs is it's often not
| visible for a while. Modern DBMSs on modern hardware are
| _crazy fast_ , so when you have only a few tens or hundreds
| of thousand rows in your table, those inefficient and
| pointless ORM queries are just not noticeable because you
| still get sub-second response times. As your database grows,
| the site begins to gets slower and slower, but it's hard to
| distinguish between the real problem and "I guess we're just
| handling more requests per second".
|
| I personally love tools like Miniprofiler [1] for this
| (though maybe there's something better today, it's been a
| while since I've worked on that type of thing). It's a
| constant and accessible way to keep an eye on what goes into
| each request, and I've caught many of those bad queries
| before they were problems by using it (eg: "WTF, why did it
| take 9 queries and 250ms to grab what looks to be a single
| row from a single table?!").
|
| [1] https://miniprofiler.com/
| fhood wrote:
| We have everything hooked into lightstep. Makes it
| extremely easy to track down problematic operations.
| zzzeek wrote:
| To be fair this is a problem inherent to databases in
| general. You can have hand written queries that perform
| badly due to structure or query frequency as well which are
| not apparent until the dataset grows. The ORM should make
| it easier to rectify such situations (eg drop in an eager
| loading directive) vs having to restructure hand-written
| routines for similar effects.
| magicalhippo wrote:
| Indeed, even with query analyzer you might see say table
| scans instead of index scan just because the DB realizes
| just scanning the 100 rows you got is faster than trying
| to use an index.
|
| So without a large number of rows it can be hard to know
| what it will actually do.
| bottled_poe wrote:
| > A simple example is, say, foreign keys. Trying to access
| the foreign key of an object by doing `book.user.id` does an
| additional query for the user table to get the ID. It's less
| known that the id is immediately available by just doing
| `book.user_id` instead.
|
| Hmm.. Sounds like a bug. Why is this not the same value for a
| foreign key?
| m12k wrote:
| Well, it is the same value, but the ORM doesn't handle
| book.user.id any differently than it does book.user.name
| where it isn't the same value, and thus the only option is
| to fetch the second table. So it's not a bug, it's really
| just the ORM being consistent in how it handles queries,
| thus missing out on a possible optimization in this special
| case, where a simpler query could have given the same
| result.
| lacksconfidence wrote:
| No, that is clearly a bug. The ORM already has the value
| of book.id, that's how it knows how to fetch the right
| book. Performing extra queries is just poor
| implementation.
| nerdponx wrote:
| But it doesn't have book.user itself. Unless you want
| Django to construct some empty proxy object representing
| book.user for this one particular optimization.
|
| If it's a bug, sounds like a "wontfix" to me.
| m12k wrote:
| The programmer also already had the value in book.user_id
| but still chose to ask the ORM to fetch all of .user so
| they could get .id from there instead. And they might
| then afterward call .name on it as well, and there would
| be no further queries, because the ORM has already been
| asked to fetch all fields of .user - so it might in fact
| have been sensible to fetch all of .user if so. The query
| builder cannot know whether all of it will be needed or
| not, because Python is not a compiled language, so
| there's no way to tell in advance when executing the
| book.user.id query that no further fields of .user will
| be needed, so it shouldn't actually do what it's been
| asked to do, to fetch the entire object, but rather only
| fetch .id which is available in a different way, so the
| whole query can be skipped. So yes, this is suboptimal
| usage, but only the programmer can know that, so it falls
| to them to optimize if they want to.
| lacksconfidence wrote:
| Perhaps i'm a bit odd, but when I'm going to lean on an
| ORM to do things I expect it to actually do them. I
| expect that foo.user_id does not exist, because that
| representation has been transformed into an object.
| foo.user.id should be the only viable reference to the
| id. foo.user.id should return the value it already knows,
| any other property access i would expect will do the
| equiv of `select * from ...` if the object has not
| previously been populated.
|
| Now perhaps some ORM's prefer to be thinner, to provide
| more footguns via a leaky abstraction that mixes
| implementation details with the object mapping. I don't
| think those are good implementaations.
| rglullis wrote:
| > I expect that foo.user_id does not exist, because that
| representation has been transformed into an object.
|
| Or you can just consider user_id to be a reference
| pointer that is part of foo, while user.id is an
| attribute of user. Totally different things and I am glad
| that the distinction is there.
| bottled_poe wrote:
| I'm sure there are valid engineering reasons to do it
| this way. One that comes to mind is memory footprint in
| allocating the objects associate with foreign key
| references.
| jtdev wrote:
| Wow! Who would have thought that using an ORM would
| create so much uncertainty in a codebase.
| RussianCow wrote:
| > Now perhaps some ORM's prefer to be thinner, to provide
| more footguns via a leaky abstraction that mixes
| implementation details with the object mapping. I don't
| think those are good implementaations.
|
| To me, it seems like the (hypothetical?) implementation
| you're talking about is much more leaky and footgun-y
| than the more straightforward ("thinner", in your words)
| version. In order for foo.user.id to not execute a new
| query, foo.user would have to return some sort of proxy
| object that only fetched the user row when you tried to
| access a field that hasn't been loaded. That's _way_ more
| magic than the more obvious solution--which is to load
| the row when you access the related object--and could
| easily cause more problems than it solves in the long run
| when you need to debug very specific queries.
|
| Furthermore, how is going out of your way to _hide_ a
| field that exists in the database (user_id) _not_ the
| definition of a leaky abstraction? What purpose does it
| serve to direct you through an unnecessary layer if all
| you need is the ID?
| zwily wrote:
| Fancier ORMs will return a proxy for foo.user that
| doesn't issue a query until the user asks for a property
| on it. And it can return user.id without querying.
| radicalism wrote:
| It would return the same value, but the approach to obtain
| it would be different.
|
| `user` would be a property defined as a User object on the
| Book model, so accessing `book.user` will cause the
| framework to fetch the entire user model (even if we then
| only fetch the id).
|
| On the other hand, `book.user_id` is the auto-generated
| database column, generated to make the above property
| definition possible. But since this `user_id` is directly
| defined on the book object, there is no need to query the
| user table.
| rglullis wrote:
| Because when you are acessing .user you are asking for all
| its properties, not just the id.
|
| Django does provide relatively easy ways to get over the
| N+1 issue, though. If you do
| Book.objects.select_related('user'), only one query is
| made.
| Hamuko wrote:
| `QuerySet.select_related()` and
| `QuerySet.prefetch_related()` are the bread and butter of
| Django query optimisation. I think most of the time that
| I've noticed a performance issue in our code, it's been
| easily fixed with one of those.
| rglullis wrote:
| Django's ORM gets a lot of flak, but I don't remember the
| last time I had complex queries that I could not do with
| it.
|
| You still need to understand a minimum of SQL and
| databases, and usually those that complain about the ORM
| are the ones that expect it to be a "sufficiently
| advanced compiler", but it has matured so much that
| nowadays the developers consider a *bug* every time the
| answer to _How do I do this query X?_ involves something
| along the lines of _use .extra or raw sql_.
| RussianCow wrote:
| This is true, though to be fair to the critics, the
| syntax through which you express these complex queries is
| often clunky and unintuitive. For example, I need to re-
| read the documentation every time I use the annotation
| API because it's generally not obvious how to use it, and
| I've run into a few edge cases where you need extra
| code/syntax just to deal with its nuances and
| ambiguities.
|
| Even though Django has come a long way, I greatly prefer
| ORMs like SQLAlchemy and Ecto that map more closely to
| the SQL query I'm trying to write.
| mrslave wrote:
| It's not a bug, it is the same _value_. Only instead of
| using `first_table.foreign_id` to fetch the entire record
| from `second_table` only to use `second_table.id`, if you
| only need the identifier itself you already have it in
| `first_table`.
|
| A similar concept called _covered queries_ exists whereby
| you index a table by foreign key, and a _few_ additional
| columns that you do not expect to use in join conditions,
| but you do expect to frequently retrieve. Depending on your
| database, requests for only columns in the index (some
| being the join condition, some subsequent columns being in
| the set of popular additional columns) means faster access
| to those popular columns only. In the context of ORMs, you
| would need to do something to avoid a default behavior of
| "select *" in order to exploit this index.
| Izkata wrote:
| > It's not a bug, it is the same value.
|
| Not necessarily, it can be overridden: "id" is only the
| default for models that haven't explicitly been given a
| field with the "primary_key" kwarg (common on legacy
| tables where the primary key column might for example be
| "user_id").
|
| The alias guaranteed to be the same value is ".pk", and
| I'm not sure what django does if you try to create a
| column named "pk" that isn't the primary key.
| ehutch79 wrote:
| If your tables don't use id, or are doing something
| strange, YOU wrote that code. You should know it's going
| to be a thing you need to deal with, because YOU did
| something unusual.
| grw_ wrote:
| I encountered this a few times and started adding tests that
| assert each handler only executes the expected number of
| queries (and no more). If the application code is modified
| such that this N+1 query pattern occurs the test will
| immediately fail and you go optimise the query, problem
| solved.
|
| https://docs.djangoproject.com/en/dev/topics/testing/tools/#.
| ..
| drunkpotato wrote:
| Or the person who changed the code then disables the test
| or sets N to 100,000,000 or something equally pleasurable
| to debug.
| robertlagrant wrote:
| I'm pretty sure Django has an active record ORM, which are
| generally a bit rubbish in terms of performance. A unit of
| work ORM such as SQLAlchemy seems to generate much better
| queries.
| bcrosby95 wrote:
| If you're selecting tons of data when you SELECT * you might
| also have a god object. I prefer to have my model be a bit
| more split up by use rather than being full of random stuff.
| E.g. a customer_address table rather than stuffing all that
| data into customer, even if they only have a single address.
| airstrike wrote:
| > ...in very unsuspecting ways
|
| > Trying to access the foreign key of an object by doing
| `book.user.id` does an additional query for the user table to
| get the ID. It's less known that the id is immediately
| available by just doing `book.user_id` instead.
|
| But that's not really unsuspecting. `book.user` is asking for
| the user table, `book.user_id` is not. Those two things are
| not identical even though they return the same value.
| cztomsik wrote:
| IIRC spark can optimize it away, and in OO there shouldn't
| be user_id
| airstrike wrote:
| > in OO there shouldn't be user_id
|
| I don't think that's right. There is a user_id column in
| the book table, so why shouldn't there be book.user_id?
| Izkata wrote:
| > (usually people writing dumb code without realizing it's
| expensive)
|
| Some years ago, one morning I gave a co-worker a recommendation
| on how to improve a loop that was unnecessarily hitting
| database through the Django ORM. He committed the fix that
| afternoon. Barely an hour later I accidentally reintroduced the
| exact same slowdown in the exact same loop when adding a
| different piece of data to it.
|
| Soooo yeah, ORMs can be so simplistic it's too easy to do by
| accident even if you know exactly what's going on under the
| hood.
| tasn wrote:
| That's a nice benefit of using async ORMs (not yet available
| in django), the db calls are explicit!
| 411111111111111 wrote:
| You can also commit to always use .values() in django. That
| makes it error out when you access non-prefetched entities
| Demiurge wrote:
| also, values is awesome for returning your models as
| dicts when that's all you need most of the time, avoiding
| all the overhead of objects
| solox3 wrote:
| Something about select_related? Please do share.
| DangitBobby wrote:
| My guess is accessing a related field within a loop causing
| a database request per iteration, e.g.
|
| ``` [book.author.name for book in Book.objects.all()] ```
| rglullis wrote:
| Maybe I spent too much time with Django already, but if I
| see anyone doing anything but
| Book.objects.values_list('author__name', flat=True) for
| this type of expression, I would mark it as a _3x WTF?_
| in the code review.
| acdha wrote:
| I'd also allow prefetch_related if you're using more than
| the most trivial data - no point in duplicating logic you
| have in your models if you have a method which generates
| something like a name, URL, etc. based on multiple
| fields.
| DangitBobby wrote:
| As written, it's obvious you should be doing something
| else like `values()` or `values_list()`. You're much more
| likely to fall victim to this anti-pattern if it's done
| within a standard for-loop that has a bunch of other
| stuff going on. I just wrote it as a list comprehension
| to avoid having to muck about with formatting on my
| phone.
| acdha wrote:
| That'd be my guess: prefetch_related is great but you
| need to guard it with something like an assertNumQueries
| test to avoid accidental regressions.
| ZephyrBlu wrote:
| Funnily enough, I recently optimized some code along
| these lines.
|
| The way I sped it up was to call `.values()` on the
| query, which serializes the data into a dict and
| prevented me from accidentally making subsueqent calls.
|
| PS: Indent by 4 spaces for code formatting.
| monkpit wrote:
| s/ident/indent/
| ZephyrBlu wrote:
| Thanks :).
| Izkata wrote:
| It wasn't anything that fancy, it's just the solution was
| something you usually try not to do so it just wasn't
| coming to mind for him. The data being looped over came
| from solr, and some of the fields were primary keys used in
| lookup tables in the database, for getting translated text.
| Instead of doing the lookup inline, load the entire table
| into a python dict before the loop (<100 rows for each of
| these tables) and do the lookup from the dict in the loop.
|
| And like I said above, usually you don't just select out
| the entire contents of a table and handle it in the
| application, so I reflexively did the wrong thing as well,
| because of how easy it was to do with Django's ORM.
| happyflower wrote:
| ORMs give an affordability to write code faster. That may save
| time, but instead of saving time, you can also reinvest in
| better quality and performance. That's up to you and your team.
|
| For instance, Django has prefetch_related and select_related.
| At almost every Django conference, there's a talk on this topic
| because it's so important and very underused/overlooked. But
| these are provided methods of the ORM.
|
| Aside from that, there are wonderful introspection tools such
| as django-debug-toolbar to view the raw SQL and its
| performance.
|
| It can be argued that if a solution written in Django hasn't
| had its database performance introspected with for instance
| django-debug-toolbar, then the solution isn't done. This is a
| small step with big rewards.
|
| This introspection can easily identify where raw SQL is useful.
| But apply it late in process: As a project matures, the costs
| of converting some queries into raw/hybrid SQL are lower, as
| the statement is less likely to change. But keep these SQL
| statements in the models and managers, don't let them spill
| into views, template tags etc.
| jerf wrote:
| I'm increasingly banging on the drum that web frameworks
| shouldn't be measured in requests per second but _seconds per
| request_. It sounds really impressive to go from 100,000 to
| 500,000 requests per second. It 's somewhat less impresssive if
| you consider that's going from 10 microseconds to 2
| microseconds... if you consider that your real, non-benchmark
| handler is likely in the dozens of _milliseconds_.
|
| I've got a couple of web handlers that after quite a bit of
| work I can legitimately claim will run on the microsecond
| timeframe... but they're the exception. Generally even a single
| DB hit across the network, even on the same system, is going to
| blow right past the web framework's time you're using.
|
| On that note, using this sort of metric, japronto's claimed
| results smell funny. Even with a 4GHz processor, getting
| 1,214,440 requests per second on a single core is ~3300 cycles
| per request. That's less than one cycle per byte in the HTTP
| request for a reasonable request (with no blocking on any sort
| of memory request), and that's not counting the TCP itself, any
| response, or the overhead of switching back and forth between C
| and Python. I can't see how this is possible without a huge
| degree of corner cutting; just validating that what you've
| received is a legal HTTP request, correctly encoded, decoding
| the fields, etc. is going to eat into that pretty fast, even
| with all the SSE instructions you may be able to throw at it.
| (And to emphasize, I'm not saying this is "impossible", just
| that it requires a lot of corner cutting. I've also got a "web
| server" out in the wild that handles "web requests" _blazingly
| fast_... because it basically ignores the entire web request
| and shovels out a hard-coded response. Very fast. Not a very
| good web server.)
| divbzero wrote:
| I increasingly lean towards plain SQL over ORMs. It requires
| greater familiarity with SQL but I prefer that over greater
| familiarity with ORM-specific syntax that doesn't translate
| across frameworks or languages. In addition, you can prototype
| new queries and profile existing queries in the database and
| copy-paste directly into your code.
| zem wrote:
| for me the value of an ORM is not so much query synthesis as
| serialisation of object fields to db columns and vice versa
| jtolmar wrote:
| Same here.
|
| I'd love to have a tool that just generates an object type
| for a given SQL query's result rows, and a function
| signature for its query parameters.
| reader_mode wrote:
| F# has this + static SQL query string checking at compile
| time using type providers
| PeterisP wrote:
| "just generates an object type for a given SQL query's
| result rows"
|
| For a similar purpose in Python I generally use psycopg
| with namedtuple resultsets; the namedtuple 'records' do
| what I need for the returned data and are reasonably
| efficient.
| branko_d wrote:
| This!
|
| We don't need ORM, we need OQM (object-query mapping)!
| grosskur wrote:
| sqlc can do this: https://sqlc.dev/
| jtdev wrote:
| That's a huge dependency to take on just for mapping and
| serializing, something that most languages provide in a
| standard library or base functionality of the language
| itself.
| jsmeaton wrote:
| https://github.com/charettes/django-seal Can help with these
| kinds of issues though I haven't yet used it myself (I
| should!)
| kevstev wrote:
| Yeah in my last job, I used my manager power to overrule on
| using an ORM. In every company I have ever been in, some
| major outage has occurred due to a seemingly innocuous change
| causing a DB operation to go from O(N) to O(N^2)+ and its
| really hard to track down because things seem to work fine in
| a dev environment, but then can't scale.
|
| And to me, SQL is the easiest language to learn and read of
| all, and while I understand doing basic marshalling of SQL
| records into objects is tedious, its really not hard at all,
| and it just saves so so much heartache down the road.
|
| The exception might be very basic CRUD apps that are meant to
| be used by third parties and want to support multiple
| backends like mysql/postgres/whatever. There might be other
| exceptions as well where you are just trying to
| prototype/find market fit, but for a typical project that you
| know is going to be used in a real way, the risks just don't
| outweigh the benefits IMHO.
|
| Some of the team griped a bit, but I absolutely think it was
| the right move in hindsight. At least once a year I would
| hear about an outage due to an ORM gone wild, and these
| outages were usually prolonged by the fact that things
| _function_ , and then there is finger pointing between the
| DB/DBAs and the app developers, etc.
| specialp wrote:
| I like ORMs for doing the things for me like making objects
| and converting dates and other data to DB ready form and vice
| versa. I do not like them querying. The premise with ORMs is
| that you should be able to query pretty much anything easily.
| The reality is that almost all of the time you are doing
| something you don't have many distinct queries. You have a
| few reads and writes that you have to do. Things that are
| easily done in SQL with full expressiveness and efficiency.
|
| So now my pattern is to use for example ActiveModel in Ruby
| for the models, but not ActiveRecord for the persistence
| part.
| fn1 wrote:
| In the java-world there are libraries like JDBI, which makes
| it possible to write interfaces with SQL-annotations and have
| serialization, connection setup etc. done for you:
| public interface UserDao { @SqlUpdate("CREATE
| TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
| void createTable(); @SqlUpdate("INSERT INTO
| user(id, name) VALUES (?, ?)") void
| insertPositional(int id, String name);
| @SqlQuery("SELECT * FROM user ORDER BY name")
| @RegisterBeanMapper(User.class) List<User>
| listUsers(); }
|
| This is great because it's explicit. No hidden queries.
| shock-value wrote:
| Another great option in Java is jOOQ, which lets you write
| type-safe and potentially composable queries such as:
| context .update(User.USER)
| .set(User.USER.NAME, userName)
| .where(User.USER.ID.eq(userId)) .execute()
| JulianWasTaken wrote:
| In Python this would look like sqlalchemy's query layer
| (which is great).
| alex-lx wrote:
| jOOQ and its DSL is good, however IMO it's more readable
| using raw SQL (by using `context.fetchInto` and its
| variants) than to using DSL when deal with complex query.
| lukaseder wrote:
| Why not just create views and query those with jOOQ,
| then?
| jwdunne wrote:
| I've noticed this in other contexts too.
|
| For small queries with straight forward joins, a query
| builder is nice and readable.
|
| But for larger, more complex queries, I found putting the
| query into its own file was best for readability.
| stiray wrote:
| I agree with you. ORMs are at the end "converting" complexity
| of SQL capabilities into objects. It is perfectly normal that
| in best situation they will get at least as complex as SQLs
| but with features hidden behind annotations or be less
| efficient.
|
| SQL is readable (at least far more than 20m lasigna of
| boilerplate objects decorated with tons of annotations
| googled from internet where no one really know what they do)
| and you KNOW that if you have optimized the database
| structure (and filesystem, and network,... :D) and SQL
| statement you will get peak performance while with ORMs you
| are on constant hunt what else can you turn on while they are
| far too huge to read their code.
|
| And they are becoming quite absurd after they "mature" and
| begin adding corner cases that no one has thought about when
| they were a starting project.
| [deleted]
| paulmd wrote:
| if that's your bag then you can still totally do that with an
| ORM, hibernate for example lets you just write a whole query
| in raw SQL while still getting all the benefits of eliding a
| bunch of boilerplate field copying, having "active" objects
| with ORM-level update/transaction management, etc.
|
| plus it means you don't have to write all the dumb "select *
| from books where bookName = :bookName" code that obviously
| can be handled trivially by the ORM. You just use SQL the
| places where it makes sense.
|
| the ORM hate always strikes me as a little misplaced because
| of this - you can always write SQL where it's appropriate, in
| any decent ORM. And you can write bad queries in SQL too.
| Obviously very complex queries are maybe better reserved for
| raw SQL but it seems like a lot of the hate comes from maybe
| less experienced engineers getting in over their head with
| complex work items because ORMs "make it easy", and that is
| going to happen with raw SQL too if you throw those same
| engineers at those work items.
|
| ORMs aren't inherently that heavyweight, I see people
| complaining here about SQLAlchemy and as a Java developer I
| don't have performance concerns about hibernate. That sounds
| to me like a Python problem and a "this specific ORM isn't
| performant" problem, not ORMs being bad as a whole. And if
| you really want a "just load the data for me and do nothing
| else that incurs a performance hit" approach then you can use
| stateless objects and it's just a wrapper around the DB to
| load and transform the data for you and/or do a raw, whole-
| object update back to the DB.
| arcbyte wrote:
| I have never seen anyone, even the ORM "experts" in my
| jobs, save time with an ORM over what a junior engineer
| couldn't do faster with plain SQL.
|
| The reason is that sure, for your first 10 basic select
| queries, the ORM saved you half an hour. Then you got to
| that complicated join and had to resort to looking up
| archane syntaxes and prototyping attribute quirks for an
| hour, when the junior guy got the whole thing written in 15
| minutes of trial/stackoverflow/error in a SQL prompt.
|
| Then, even when the "expert" did get it working, guess who
| is going to be the one looking at it again when trying to
| figure out production support issues? The junior guy, who
| is now clueless and has to spend two hours to figure out
| what this crazy ORM mess does here. The better alternative
| was just to have the SQL there ready to go so it is well
| understood and can be ran against the production database
| or a test database to reproduce the issue. No questions
| whatsoever.
|
| I have seen this over and over again, more than a
| statistically relevant number of times.
| deergomoo wrote:
| > The reason is that sure, for your first 10 basic select
| queries, the ORM saved you half an hour
|
| Maybe we work in very different fields, but "basic select
| queries" makes up 90% of what I need to fetch from the
| database.
|
| If I'm working on a forum and I want to load user 123,
| with all their posts, all the awards each post has, and
| the count of friends the user has, with Eloquent
| (Laravel's ORM), I could do:
|
| `User::with('posts.awards')->withCount('friends')->findOr
| Fail(123);`
|
| That would return me a User model, with a collection
| "posts" containing a list of Post models, each with a
| collection "awards" of Award models, and a field
| `friends_count` with the number of friends. It would run
| three queries: one to fetch the user, one to fetch the
| posts, and one to fetch the awards. Depending on how I
| have configured my models, I can have things like dates
| automatically hydrated to DateTime objects.
|
| Compare that to plain SQL queries; I would have to fetch
| the users, including manually writing the subquery for
| the friend count. Once I had those users I would then
| have to fetch the posts, and then again for the awards.
| If I want them in a hierarchy like the ORM example gives
| me, I then need to loop through each set of records and
| manually stitch them together. Not difficult, but super
| tedious.
|
| Sure a complicated join is better done with as little
| magic as possible, but Eloquent exposes functions for
| adding subselects, joins, etc. in a way that just reads
| like SQL (and maps 1:1 underneath).
| alexchamberlain wrote:
| We're gradually removing our ORM usage to avoid active
| objects. They look too much like VSTs, but subtly make
| foreign network calls when you least expect it, which makes
| it very hard to write effective, isolated tests or replace
| the database layer by something else, like a service call
| or a cache lookup.
| kaba0 wrote:
| I'm sorry if it sounds harsh, but that is the mistake of
| the developers than. I think you would not allow anyone
| to write production code without knowing the language; it
| should be the same way with most libraries, especially
| ones having a big reach, like ORMs. Nonetheless, I have
| seen it countless of times in the team I worked in -- so
| it is unfortunately really frequent.
| lordnacho wrote:
| IMO both are required. I'm lucky in that I did a lot of plain
| SQL early on, and found ORMs later, but I think ORMs do cut
| out a lot of time for quick-and-dirty queries that end up not
| being the bottleneck. The problem arises once you find a
| bottleneck, you won't know how to optimize it if you haven't
| done a bit of SQL mucking about earlier.
|
| Also, the big thing is you won't know how to translate to
| other ORMs if you don't know SQL. Eg once you know that you
| want an index, it's a matter of a web search to find out what
| the syntax is in your ORM. But if you just started with ORM,
| you might not realize that kind of thing is part of how it
| works.
| kaba0 wrote:
| Yeah, ORMs are specifically made for insert-heavy
| operations or very basic mapping of rows to objects.
| Analytical queries and the rest should be done with SQL.
| jtdev wrote:
| I keep seeing some variation of: "ORMs are really
| intended for x" or "ORMs work best when y is true". At
| some point we should take an honest look to determine if
| ORMs provide enough benefit to justify their existence.
| "I can talk to the database without knowing SQL" doesn't
| cut it.
| kaba0 wrote:
| Yes they do? They are basically available in every
| language, in some ecosystems there are multiple
| alternatives as well and there would not be that big of a
| market unless it is useful.
| acdha wrote:
| I favor a hybrid approach: use the Django ORM to define
| models, do migrations, auto generate the admin, etc. but
| don't be shy about using the extension points (extra, raw,
| cursors) to put in an optimized query for a hotspot. You can
| get pretty far using the ORM but it's really valuable to be
| able to be comfortable dropping down for things like reports
| or bulk processing.
| Jaxtek wrote:
| I use ORM for Django's auth system alone.I write all other
| REST API queries in plain SQL.
| girvo wrote:
| Plain SQL with a "hydration" system to objects is quite
| powerful. DataMapper, I believe is the term of art?
| jeromenerf wrote:
| > But I agree with the author's main point which seems to be
| that framework performance is pretty meaningless when comparing
| frameworks if you're just starting on a new project. Focus on
| making a product people wanna actually use first. If you're
| lucky enough to get to scale you can work about optimizing it
| then.
|
| It feels like a sensible advice but "optimization", if ever
| possible, can only get you so far until you need a costly
| refactoring or rewrite in my experience.
|
| As projects can be very different in context, it is all about
| what makes a minimal implementation "viable".
| closeparen wrote:
| With Symfony back in the day, you could turn on a little
| toolbar that would show in the top of your rendered HTML. You
| could see how many SQL queries were run, what the statements
| were, how long they took. I think there was some other tracing
| information as well. This strikes me as a basic and necessary
| kind of testing to do when developing with an ORM. Perhaps more
| systematically you could get ORM business into Zipkin or
| Jaeger, and then have some kind of staging vs. prod or canary
| vs. prod statistical comparison to see if you're about to
| release something dumb. Or maybe simpler to keep generated SQL
| in unit test assertions. You wouldn't have to write it, but you
| would have to read and update it on changes, so you could
| notice if you were winding up with N+1 queries or a ridiculous
| join.
| Ambix wrote:
| Why is it so? I've got 100K requests per sec with PHP easily [1]
|
| [1] https://github.com/gotzmann/comet
| hu3 wrote:
| "Blazing fast with 100K HTTP requests per second and ~0.1 ms
| latency on commodity cloud hardware"
|
| Ok what's the secret sauce and downsides? Because as far as I
| know there's no way to cheat with PHP like C# did here:
|
| https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
| Ambix wrote:
| The most important part is Workerman and its efficient
| network arhitecture based on libevent. The other part is
| efficient DB drivers. Thats why Comet ranks much higher than
| Go / NodeJS / Python frameworks in DB higloaded test:
|
| https://www.techempower.com/benchmarks/#section=data-r20&hw=.
| ..
| sdevonoes wrote:
| Interesting. I never heard of Japronto before. For the people
| working with Python: why Flask instead of Japronto?
| BiteCode_dev wrote:
| The ecosystem.
|
| You got loads of libraries built around flask, and a lot of doc
| and tutorials, not to mention how much battle tested it is.
|
| Those can matter much more than hypothetical perfs in a
| synthetic benchmark.
| throwdbaaway wrote:
| Good article, but I can't help but notice a gaping hole in the
| benchmark -- why was there no attempt to run gunicorn in multi-
| threaded mode?
|
| The article has a link to
| https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...,
| but failed to mention the key takeaway from the article:
|
| > threaded code got the job done much faster than asyncio in
| every case
| driverdan wrote:
| The article explicitly says they were testing single threaded
| use cases.
| throwdbaaway wrote:
| But why? The results would be a lot more useful if they
| include a single-process multi-threaded setup in the
| benchmark.
___________________________________________________________________
(page generated 2021-02-19 23:02 UTC)