[HN Gopher] Horrible Code, Clean Performance
___________________________________________________________________
Horrible Code, Clean Performance
Author : signa11
Score : 60 points
Date : 2023-04-17 02:02 UTC (1 days ago)
(HTM) web link (johnnysswlab.com)
(TXT) w3m dump (johnnysswlab.com)
| ch_123 wrote:
| It is absolutely true that some hot-path code needs to be mangled
| into an ugly mess to reach performance requirements. The problem
| is that I have encountered people who somehow take this as a
| blanket justification for writing unreadable code, and who
| operate on a false dichotomy about a choice between readable code
| and performant code. It is important to keep in mind that:
|
| 1) Most code, i.e. at least 80% of the code in a codebase, will
| never be a performance hotspot and does not need specific
| optimizations (i.e. as long as the code does not do stupidly
| inefficient things, it's probably good enough).
|
| 2) Even in performance hotspot codepath, you should not write
| unnecessarily hard to read code unless it is strictly necessary
| to achieve required performance.
|
| In both cases, the key point is to benchmark and profile to find
| the specific places where the ugly hacks need to be introduced,
| and do not introduce more than is strictly necessary to get the
| job done.
| louthy wrote:
| Agree on this. Although the 80% figure is probably more like
| 99% (I have no metrics, but 80% feels too low to make this
| point)
| WalterBright wrote:
| It's about 99% if you haven't run it through a profiler and
| optimized the hot spots. With such, you can get it down to
| 90% or so.
| pclmulqdq wrote:
| I have noticed that great code and terrible code often have
| the same characteristics on profiles: in both cases,
| slowness is spread around everywhere.
| kolbe wrote:
| In this video, David Gross says something along the lines
| of "if your data is laid out in a memory inefficient
| manner, it will be the bottleneck everywhere." If you're
| dependent on a database query for everything, that will
| be your bottleneck. If your in-memory structures are
| cache inefficient, that will be your bottleneck. But once
| you've fixed all of this (if you can), hot paths and
| whatnot matter
|
| https://m.youtube.com/watch?v=8uAW5FQtcvE
| jeffreyrogers wrote:
| The problem is that if you don't know what is required to write
| fast code you won't design your program to be performant from
| the start. Lots of applications can be sped up dramatically by
| rearchitecting them, but you're not going to hit on the right
| architecture without thinking really hard about memory layout,
| data access, network usage, etc.
|
| It's not really true that 80% of the code doesn't matter. 80%
| of the code might not matter the way it is currently designed,
| but that doesn't mean a different design isn't dramatically
| better. Benchmarking and profiling doesn't help with that. Real
| expertise is required and that is developed over time by people
| who care about improving performance and make it a priority.
| szundi wrote:
| 80% does not matter, period. I mean 20% is a lot. Hotspots
| are - spots. Like 0,5% of your code. Not UI for example.
|
| You know, when money is flowing in, you have to hire some
| good people, let them spend time on fixing hotspots and in
| some rare occasions, rearchitecture a part or maybe two.
| Probably won't pay off, but if you are growing fast enough,
| it will pay 100x.
|
| Also you're decreasing all future development speed with all
| this unreadable shit everywhere, so beware of optimizing
| unnecessary stuff.
| jeffreyrogers wrote:
| For seriously high performance systems like webservers or
| exchanges, you need to get the architecture right from the
| start. You can't rearchitect a part or two or solve design
| problems by making skilled hires later on. The 80% part is
| a distraction (it's also factually untrue that 80% of code
| doesn't matter. Some applications don't have hotspots, the
| performance impact is basically smeared all across the
| code. It really depends on the application. Sqlite found
| major speedups by implementing hundreds of little changes
| that were each almost undistinguishable from noise).
|
| High performance doesn't mean unreadable.
| Microoptimizations can make code less readable, true, but
| they also give you the least amount of speedup (in the best
| case you can get several times speedup if you found some
| hotspot that can be vectorized, but that's pretty rare).
| You can get huge performance improvements by optimizing
| networking and system calls and doing that doesn't make
| much of an impact on readability.
| citrin_ru wrote:
| I often see death by a thousand cuts - everything is slow
| and there is no single obvious hotspot to fix. We should
| not assume that performance matters only for a small
| fraction of the codebase. Sometimes it is the case,
| sometimes not.
|
| And UI is a bad example IMHO. UI with a high response time
| which is uncomfortable (or even frustrating) to use is
| probably the result of thinking that UI performance doesn't
| matter.
| HacklesRaised wrote:
| I think sometimes this exact point is lost in the context
| of a collection of programs that represent a system
| rather than simply in the context of a single program.
| seadan83 wrote:
| > Benchmarking and profiling doesn't help with that.
|
| I've learned that benchmarking and profiling is the _only_
| way to write performant code.
|
| I've seen in code review a number of examples of a very fancy
| algorithm being broken out, and asked, "You realize N is
| bounded to be at most 100 here?". Or, "you realize the thread
| overhead here for parallel processing is two magnitudes
| slower than serial data access on one thread?"
|
| Humans are _bad_ at intuitively understanding where the slow
| parts of code are. I 've seen processing be improved to the
| point of impossible to grok, shaving 10ms of a processing
| piece that is 50ms long, only to then spend time blocking
| waiting for network transfers that require 10s.
|
| I'm of the opinion the biggest performance improvements are
| usually in design and architecture. What if there were a
| design that avoided the need to do any network IO? In that
| case the 10s + 50ms would be a 50ms process, rather than a
| hard to grok "optimized" 10s + 40ms process. Simple code
| leads to simpler design, which is easier to reason about and
| spot the places where things like "this entire network round
| trip can be cut out", or "we are loading this data multiple
| times throughout this process, we can load it once", or "we
| are loading this data and then spending a lot of time
| querying "n+1", instead if we stored the data in this format
| with some pre-processing we'll avoid the "n+1" query."
|
| To further rant, the emphasis of algorithms in coding
| interviews, people enjoying algorithms more than cleaning up
| crufty architecture - _that_ is the root of a lot of bad
| software rather. In sum, it 's almost always the design that
| is slow, rarely it's the algorithm. The profiling is key as
| it let's you know where things are actually slow. (Recently a
| colleague was trying to optimize a tight loop that processed
| 1.5M rows. To "optimize" memory usage, they converted all
| variables to static to 'save' memory and avoid GC pauses.
| This in effect did _nothing_, the compiler instead was going
| to inline all the variables anyways and the resulting
| bytecode was not going to have any extra variables in at all.
| Converting local variables to static actually made the memory
| usage just slightly worse. So, this 'optimization' did
| nothing but make the code worse. A quick benchmark would have
| shown that optimization having no effect (to really optimize
| memory usage, we updated the design to stream results to a
| file rather than keep everything in memory for a final dump
| to file at the very end). Another example, I once helped a
| team do performance work for a DB that they spent a year
| tuning. They did not keep track of any performance
| benchmarks, what changes did what improvement; and after a
| year had nothing to show except for a DB that would crash
| after a few minutes. Taking that over, starting everything
| over from scratch, benchmarking everything, the project was
| done a month later and was stupid fast.)
| thethirdone wrote:
| I definitely agree on the false dichotomy between performance
| and readable code.
|
| > 1) Most code, i.e. at least 80% of the code in a codebase,
| will never be a performance hotspot and does not need specific
| optimizations (i.e. as long as the code does not do stupidly
| inefficient things, it's probably good enough).
|
| The hard part is knowing what "stupidly inefficient things"
| are. If you never do excessive optimization, its easy to drift
| towards less efficient over time because your baseline for what
| performance is possible slows down. Knowing how to do
| performance optimization that is not worth it and knowing what
| the best level of efficiency to shoot for is the mark of a good
| engineer.
| jackmott42 wrote:
| Yeah, absent an effort to speed up a code base, it will tend
| to get slower, unless the team has a culture of performance,
| everyone is looking for easy performance wins and taking
| them, and common performance mistakes and getting rid of
| them.
|
| To do that the team needs experience with performance, and
| most of internet programmer culture is just to lecture people
| about how programmers are cheaper than hardware if anyone
| asks or talks about performance. So many people don't get
| that experience.
| johnmaguire wrote:
| > programmers are cheaper than hardware
|
| Did you mean the reverse of this, I assume?
| 908B64B197 wrote:
| Depends on your install base/target.
|
| At Apple scale, paying a few performance engineers 1M/y
| is much cheaper than shipping bibber and more powerful
| CPU in each iPhone.
|
| Same thing for AWS or Azure.
| dahfizz wrote:
| This makes assumptions about what the performance requirements
| are.
|
| In latency sensitive code, _all_ code needs to be optimized. It
| doesn't matter if your slow function is only called once - it
| adds N microseconds of latency when it doesn't need to.
| zwieback wrote:
| I think that's where the craft of programming comes in. I feel
| like after 30 years in the field I have a reasonable idea how
| and when to optimize but it's not something I learned from
| classes or textbooks.
| xg15 wrote:
| > _1) Most code, i.e. at least 80% of the code in a codebase,
| will never be a performance hotspot and does not need specific
| optimizations (i.e. as long as the code does not do stupidly
| inefficient things, it 's probably good enough)._
|
| That's true, however, I've seen enough code where the author
| used this argument as a justification to _do_ stupidly
| inefficient things, such as building monstrous mountains of
| abstraction layers (or _re-scan the entire codebase at runtime_
| because why not? [1]) all covered by the "premature
| optimization is the root of all evil" mantra.
|
| You can absolutely write code that is both opaque _and_
| inefficient.
|
| [1] https://docs.spring.io/spring-
| framework/docs/3.0.0.M4/spring...
| flavius29663 wrote:
| 80% seems very low, in my experience it's 1% or less of the
| code that is a hotspot.
|
| On the other hand, we should talk about "doing stupid things",
| or having a bad design/architecture. For example, if you
| architect your application such that each web request is
| serviced by 20 micro-services, and those micro-services make 20
| requests of their own...no matter how fast your code is, the
| application will be slow.
| atq2119 wrote:
| I suspect the 1% number is typical only of programs that
| haven't been optimized. Once people start to care about
| performance and address these hotspots, which are often
| actually quite low hanging fruit, the profile starts to
| become flatter relatively quickly.
| tombert wrote:
| I had a manager who would write the most hideous, impossible-
| to-debug code imaginable and always have some sort of
| microbenchmark as some kind of justification. At the time I was
| young enough in my career to just think he was wiser than I
| was, but upon reflection (and remembering some of the stuff he
| did involving FFI-ing C into Haskell all the time), I realized
| that he just didn't he didn't like people criticizing his code,
| though the first hint should have been when he refused to
| answer my questions about actual performance profiling of our
| codebase.
| commandlinefan wrote:
| > blanket justification for writing unreadable code
|
| Well, ironically, the site itself is down (so the code must be
| beautiful?) but assuming I understand the content from the
| title, another thing that most of these "code quality doesn't
| matter" types overlook is that software changes over time,
| quite a bit. That's why it's called "soft"ware, it's supposed
| to be soft. Readable is changeable. If all the mattered were
| performance, we'd engrave the code path onto a circuit board.
| systematical wrote:
| Saved me a lot of typing.
| Shorel wrote:
| I agree so much with you, "in theory".
|
| In practice, most of the industry has been writing ridiculously
| inefficient code for decades, and I wish more people would pay
| attention to that 20%, or the 3% that Knuth described.
|
| Or better yet, I wish some groups don't irreversible sacrifice
| performance with frameworks and architectures that are
| impossible to optimize, no matter how the code is written.
| commandlinefan wrote:
| > I wish more people would pay attention to that 20%,
|
| I agree with both of you - there's no reason why software
| should be as inefficient as it is, and there's _also_ no
| reason why code should be as unreadable as it is, and those
| goals aren 't mutually exclusive.
| patrulek wrote:
| Its true for services/applications. In a case you write
| library, others will import into their projects, you should
| always treat performance seriously imo.
| taeric wrote:
| The problem is that so many practices we have gravitated to are
| actively harmful to performance. Worse, they are borderline
| harmful for maintenance. Specifically, some practices are
| better for larger teams than they are smaller teams.
|
| Difficulty in that last is that the best practice for how to
| maintain code changes as you get older on the project. This is
| easy to see in code that is a bit of a mungled mess. But it is
| also visible on code that it is a beautiful stalled out mess of
| needing too much to get small contributions in.
| wfurney wrote:
| https://archive.is/Q82MY
| jackmott42 wrote:
| The website seems to have gone down under load, if this is a
| snarky response to Casey's pro performance take previously, then
| the irony level is high!
|
| I really wish people would stop pushing back on people who are
| interested or curious about how to make code perform better. Of
| course you don't want people scattering weird SIMD intrinsics all
| over mundane parts of your web backend in an unhinged attempt to
| shave microseconds off a response. But in my career I've never
| really seen people do this anyway.
|
| What you _do_ want though is programmers who have spent some time
| understanding how to structure code to perform well on modern
| hardware. Because often times that structure is NOT a mess, it
| may even be easier to work with that the usual idioms in your
| industry or language. Grouping data into adjacent chunks with
| arrays or array backed data structures has more benefits than
| just leveraging the L1 cache. It is sometimes also convenient.
| secondcoming wrote:
| We put a 'does this person understand assembly output' into our
| interview process. They don't have to get it 100% right, but
| not knowing anything is a hard No.
| moosedev wrote:
| Interesting! What kind of company and technology domain is
| it? Assuming you don't want to get too specific.
|
| My first reaction is that I like this idea, but maybe I'm
| biased because I feel I'd "pass the test" myself (and thus
| have one additional small way to differentiate myself from
| pure Leetcoders :)
|
| In most of the environments I've worked (except maybe the
| games companies) I feel this would not have been widely
| considered an acceptable thing to ask in a generalist
| interview.
| secondcoming wrote:
| adtech exchange
| wk_end wrote:
| I'd say it's orthogonal to Casey's take, which was about
| trading abstraction for (arguable) simplicity + performance.
| This was an article about aggressively optimizing a simple
| loop, thus trading simplicity for performance.
|
| Given that it was from a consultancy advertising its
| optimization services, the irony of it going down is still
| pretty high, though.
| acl777 wrote:
| I remember this was a holy war amongst programmers (even myself!)
| - readability OR performance.
|
| With tools like ChatGPT - we can have both readability AND
| performance, right??
|
| Or am I missing something?
| circuit10 wrote:
| If you mean automatically optimising code, the reliability
| isn't quite there (it can't check its own code) and the limited
| context length means it might struggle to reason about the
| codebase as a whole
| pjc50 wrote:
| Correctness?
|
| I've not seen anyone seriously attempting to benchmark chatgpt
| output, without heavily cherry picking it first.
| jerf wrote:
| Dunno about "readibility" but I fully expect ChatGPT to produce
| vast swathes of code that nobody understands.
|
| While I'll cop to being more skeptical than the average HN
| denizen about ChatGPT, this isn't actually cynicism about
| ChatGPT, but about human cognition. If we do have something
| that produced large swathes of mostly correct code, it won't be
| long before the humans using it aren't even checking it
| anymore. Same reason that cars can do a little bit of
| assisting, and if they could do 100% of the driving that might
| be safe, but 98% of the driving is not a very good idea at all.
|
| In 20 years, someone will ask CodeGPT 8.3 to explain what some
| code does, and it'll give a perfectly understandable
| explanation back, except the human still won't understand it
| because the human doesn't actually understand how computers
| work anymore. They'll think they understand it, though.
| attractivechaos wrote:
| The blog post is effectively implementing memmem() or strstr()
| that searches a short string in a long string. If we are allowed
| to use GNU extensions, the cleanest solution is to call memmem().
| Without memmem(), I would implement Boyer-Moore or Knuth-Morris-
| Pratt, which will be more scalable than the O(MN) implementation
| in the blog post. Time complexity matters.
| wk_end wrote:
| Yes, this was my takeaway - it was a very poor choice of
| example, because a known faster algorithm will crush the sort
| of micro-optimizations done here _and_ be more readable.
|
| Or perhaps it was actually an incredibly good example, despite
| itself: a great illustration of precisely why jumping to micro-
| optimization isn't always the right solution.
| pclmulqdq wrote:
| This is also a problem where the algorithmically fastest
| solutions are also pretty much guaranteed to be faster, since
| they are equally cache-friendly and will almost always work
| out to fewer instructions.
|
| That is not always the case, but this is one of the times
| when it is.
| moomoo11 wrote:
| I mean it's possible to have both clean and performant.
|
| If you have a high performance requirement you should extract
| that code into its own "space" and not try to make the existing
| system shitty.
|
| Use API contracts between systems and don't muddy up existing
| systems.
| dack wrote:
| can't read the article because the site timed out.
|
| maybe the code behind his blog is too clean!
| Freedom2 wrote:
| A capital joke, kudos to you! Made me chuckle.
| bena wrote:
| Unfortunately, there are many reasons to write horrible code.
|
| I work in-house for a non-tech related company. I'm often the
| sole developer, sometimes I have help. I'm also the defacto DBA
| and systems administrator for the machines the software runs on.
| As the defacto DBA, I'm also roped into data analysis as well.
|
| There are also certain events that are non-compromisable. These
| events take place whether we are ready or not. Having nothing is
| not really an option. That means shelving the project until next
| year. Because the next event must be prepared for.
|
| So, at the end of the day, "done is best". If I have the time, I
| can go back and refactor everything into something better. But
| often, there's "the next thing".
| xg15 wrote:
| I would say this is not horrible code - or at least, it's
| "hollywood horrible". [1] Yes, it's not directly intuitive and
| probably would need some illustrating comments, but it's still
| reasonably close to the "naive" algorithm that you could quickly
| figure out how it works. More importantly, it's still localized,
| concrete and side-effects free: The code has a clear goal and
| accomplishes that goal completely inside that one function. The
| non-intuitive parts also have a clear reason.
|
| I think actual horrible code is more something that uses global
| mutable state, has logic spread over half a dozen units or _does_
| have weird roundabout implementations - however not because of a
| particular reason but because of unclear goals, big-ball-of-mud
| architecture, evolving codebase etc.
|
| [1] https://tvtropes.org/pmwiki/pmwiki.php/Main/HollywoodHomely
| KineticLensman wrote:
| Oh No! I've already been down one TvTropes rabbit hole today.
| fwlr wrote:
| I have long been in the habit of leaving in the old pre-optimized
| code, commented out, above the optimized code. I generally don't
| think that performant code ends up all that mangled, but it costs
| ~nothing to leave the old code in a comment.
|
| I picked this practice up from an older engineer who would set
| SLOW = false
|
| at the top of the file and then wrap old code in a
| if (SLOW) { ... }
|
| block for the compiler to do dead code elimination on. (I found
| linters and compilers complained less about commented-out code,
| but he preferred his way so it would always have syntax
| highlighting.)
| capableweb wrote:
| > but he preferred his way so it would always have syntax
| highlighting
|
| On that topic, probably one of the top-3 features from Clojure
| I miss in other languages, is having the option of making
| comments being a part of the language itself so everything that
| works on the code (linters, evaluation, syntax highlighting,
| etc) works in the comment as well. The `comment` macro really
| is a godsend in disguise as it's awfully simple implementation-
| wise. Its cousin `#_` is also a great tool. See
| https://clojuredocs.org/clojure.core/comment for more examples
| lozenge wrote:
| Another option is to keep both versions and write a test that
| they return the same value, e.g. using property-based testing.
___________________________________________________________________
(page generated 2023-04-18 23:02 UTC)