[HN Gopher] Smurf: Beyond the Test Pyramid
___________________________________________________________________
Smurf: Beyond the Test Pyramid
Author : BerislavLopac
Score : 25 points
Date : 2024-10-19 20:22 UTC (2 hours ago)
(HTM) web link (testing.googleblog.com)
(TXT) w3m dump (testing.googleblog.com)
| pydry wrote:
| They should at least admit they made a mistake with the "test
| pyramid".
| codeflo wrote:
| Not the least of which is that people who take stuff like that
| as gospel instead of as heuristics tend to endlessly argue what
| is and isn't a "unit".
| petesergeant wrote:
| > who take stuff like that as gospel
|
| In 2011 I wrote a blog-post[0] about the dangers of taking
| TDD too literally, and until I disabled comments on it ~5
| years later people were still angrily shaking their fists at
| each other online. 139 angry comments for a very short post.
|
| 0: https://writemoretests.com/2011/09/test-driven-
| development-g...
| candiddevmike wrote:
| Conjoined triangles of testing
| petesergeant wrote:
| I feel like "testing trophy" has been in vogue for a while now,
| and definitely feels more right as someone who's made a career
| of unfucking test suites, but there's almost no area of
| software engineering as involved in navel-gazing as testing.
| grahamj wrote:
| This is all pretty obvious
| sverhagen wrote:
| And yet, people focus endlessly on unit tests, pyramid in hand,
| saying things like: they're the best kind of test, or at least
| better than integration tests. It needs some maturity to
| articulate the nuances. And while I've tried, I think SMURF may
| be a good aid in that. While I've moved away from the religious
| focus on unit tests, long ago, I appreciate learning about
| SMURF today.
| usbsea wrote:
| This is obvious, as anoter commenter said, but this is
| nonetheless useful.
|
| You can use it to show graduates. Why have them waste time
| relearning the same mistakes. You probably need a longer blog
| post with examples.
|
| It is useful as a check list, so you can pause when working
| earlier in the lifecycle to consider these things.
|
| I think there is power in explaining out the obvious. Sometimes
| experienced people miss it!
|
| The diagram can be condensed by saying SMUR + F = 1. IN other
| words you can slide towards Fidelity, or towards "Nice
| Testibility" which covers the SMUR properties.
|
| However it is more complex!
|
| Let's say you have a unit test for a parser within your code. For
| a parser a unit test might have pretty much the same fidelity as
| an intergation test (running the parse from a unit test, rather
| than say doing a compilation from something like Replit online).
| But the unit test has all the other properties be the same in
| this instance.
|
| Another point is you are not testing anything if you have zero
| e2e tests. You get a lot (a 99-1 not 80-20) by having some e2e
| tests, then soon the other type of tests almost always make
| sense. In addition e2e tests if well written and considers can
| also be run in production as synthetics.
| candiddevmike wrote:
| What's useful here? There's nothing actionable, no way to
| quantify if you're doing "SMURF" correctly. All the article
| describes is semi-obvious desirable qualities of a test suite.
| viraptor wrote:
| You're not "doing SMURF". It's not an approach or a system.
| It's just a specific vocabulary to talk about testing
| approaches better. They almost spell it out: "The SMURF
| mnemonic is an easy way to remember the tradeoffs to consider
| when balancing your test suite".
|
| It's up to your team (and really always has been) to decide
| what works best for that project. You get to talk about
| tradeoffs and what's worth doing.
| stoperaticless wrote:
| > What's useful here?
|
| It is up to the reader to figure out this one.
| imiric wrote:
| This is interesting, but I see a few issues with it:
|
| - Maintainability is difficult to quantify, and often subjective.
| It's also easy to fall into a trap of overoptimizing or DRYing
| test code in the pursuit of improving maintainability, and
| actually end up doing the opposite. Striking a balance is
| important in this case, which takes many years of experience to
| get a feel for.
|
| - I interpret the chart to mean that unit tests have high
| maintainability, i.e. it's a good thing, when that is often not
| the case. If anything, unit tests are the most brittle and
| susceptible to low-level changes. This is good since they're your
| first safety net, but it also means that you spend a lot of time
| changing them. Considering you should have many unit tests, a lot
| of maintenance work is spent on them.
|
| I see the reverse for E2E tests as well. They're easier to
| maintain, since typically the high-level interfaces don't change
| as often, and you have fewer of them.
|
| But most importantly, I don't see how these definitions help me
| write better tests, or choose what to focus on. We all know that
| using fewer resources is better, but that will depend on what
| you're testing. Nobody likes flaky tests, but telling me that
| unit tests are more reliable than integration tests won't help me
| write better code.
|
| What I would like to see instead are concrete suggestions on how
| to improve each of these categories, regardless of the test type.
| For example, not relying on time or sleeping in tests is always
| good to minimize flakiness. Similarly for relying on system
| resources like the disk or network; that should be done almost
| exclusively by E2E and integration tests, and avoided (mocked) in
| unit tests. There should also be more discussion about what it
| takes to make code testable to begin with. TDD helps with this,
| but you don't need to practice it to the letter if you keep some
| design principles in mind while you're writing code that will
| make it easier to test later.
|
| I've seen many attempts at displacing the traditional test
| pyramid over the years, but so far it's been the most effective
| guiding tool in all projects I've worked on. The struggle that
| most projects experience with tests stems primarily from not
| following its basic principles.
| stoperaticless wrote:
| > I don't see how these definitions help me write better tests
|
| > What I would like to see instead ...
|
| If you hire me, I can address those needs.
| sverhagen wrote:
| This model ("mnemonic") feels like a good tool to reason about
| your testing strategy. I ran across the "testing trophy" in the
| past, which really changed my thinking already, having been
| indoctrinated with the testing pyramid for such a long time
| before that. I wanted to share my favorite links about the
| testing trophy, for those interested:
|
| https://tanzu.vmware.com/content/videos/tanzu-tv-springoneto...
|
| https://kentcdodds.com/blog/the-testing-trophy-and-testing-c...
| jbjbjbjb wrote:
| Not to be pedantic, but practically speaking it looks like there
| are two dimensions: fidelity and then the rest (the SMUR).
| eiathom wrote:
| It always amazes me how speed and testing are placed in the same
| bracket. I want solid verification, and a strong pattern to
| repeat verification no matter what. This then allows for fast
| implementation. So then something involving integrating a number
| of components as possible reliably makes the most sense
| (verification wise): I want to verify value early. It is eyebrow
| raising this pyrmaid nonsense has hung around.
| nemetroid wrote:
| End-to-end tests verify high-level expectations based on the
| specification of the system. These high-level expectations
| generally stay stable over time (at least compared to the
| implementation details verified by lower-level tests), and
| therefore end-to-end tests should have the best maintainability
| score.
___________________________________________________________________
(page generated 2024-10-19 23:00 UTC)