[HN Gopher] Improving end-to-end test reliability
       ___________________________________________________________________
        
       Improving end-to-end test reliability
        
       Author : dailymorn
       Score  : 48 points
       Date   : 2022-01-23 04:03 UTC (1 days ago)
        
 (HTM) web link (frantic.im)
 (TXT) w3m dump (frantic.im)
        
       | TotempaaltJ wrote:
       | I love learnings on automated testing. From the perspective of
       | someone who isn't used to TDD or even just building many tests,
       | maintaining E2E tests often seems extremely cumbersome. I wonder
       | if I'm just missing out on the best practices, or if the tooling
       | simply hasn't evolved enough yet.
        
         | martinald wrote:
         | The payoff is much higher imo though. Of all the tests we do,
         | e2e catches by far the most problems. Indeed the biggest
         | mistakes I've made often are me thinking tests are flakey
         | 'because e2e' when in reality they are showing a glaring
         | problem.
         | 
         | Especially in mobile/web applications where you are often
         | consuming loads of services/libraries/sdks, some in house, some
         | external, you are often running a tiny amount of your own code.
         | Adding tonnes of unit tests to that is sort of missing the big
         | picture - you need to test it all works together as a user
         | would.
        
           | eatonphil wrote:
           | Yep same. For libraries unit tests are great. For
           | applications though I feel the most value writing integration
           | tests and e2e tests. That's what helps capture the biggest
           | user-facing bugs.
        
           | sidlls wrote:
           | That's a rare attitude to have in the Bay Area anyway.
           | Unfortunately. Everyone wants lots of unit tests because they
           | run fast and give (roughly) instant feedback. Unit tests
           | paint an incomplete picture. Too little attention given to
           | integration tests and end-to-end tests leaves systems exposed
           | to critically bad edge case bugs.
        
             | mgkimsal wrote:
             | > Too little attention given to integration tests and end-
             | to-end tests leaves systems exposed to critically bad edge
             | case bugs.
             | 
             | From my POV and experience, the middle ground is often what
             | people refer to as 'integration' tests. Testing (without a
             | browser), hitting endpoints/urls with known payloads and
             | getting expected results catches errors with assumptions
             | made about the interaction between various individual
             | libraries.
             | 
             | At least in the web app world, my views are:
             | 
             | 1. Testing the individual libraries gets you one layer of
             | confidence. 2. Testing the interaction of those, usually
             | via URL endpoints as various identities, gets you another
             | layer of confidence. 3. Testing with E2E exposes primarily
             | UI/JS problems.
             | 
             | When the first 2 are strong/solid, you can focus
             | troubleshooting problems in #3 at the client/JS level
             | first. It's not always the case, but it can help reduce
             | concerns about "is this a back-end issue?".
             | 
             | I've been (slowly) trying to write more js component tests
             | (in one case, with jest and vue), as it makes it
             | easier/faster to test many permutations of
             | input/validation/etc all at once. It's yet another
             | 'confidence' area such that, when there are E2E tests, I
             | can narrow down focus even more.
             | 
             | On a couple projects I've been on the past few years, we've
             | found very few problems via E2E tests alone, mostly because
             | there are so many back-end unit and integration tests. The
             | E2E issues that are found are often UI-only (error state
             | changes not rendering, sometimes perf issues, etc).
        
             | bluGill wrote:
             | The problem I have with unit tests is they inhibit
             | refactoring off the API they test.
             | 
             | Sure if you write your own implementation of "string" or
             | "list" you will probably get the API right the first time -
             | those are commonly used and time tested so everyone knows
             | about what the API should be. However almost nobody is
             | writing them, they come with your language for everyone but
             | a few language implementers, or once in a while the company
             | library implementers.
             | 
             | For everyone else we are writing to a business requirement
             | that isn't well understood and may change. However the
             | purpose of a unit test is to assert that no matter what
             | this won't change. So every time you want to make a change
             | all those tests are in the way of the change and need to be
             | fixed.
             | 
             | Everyone writing tests needs to figure out their own middle
             | ground. Because end to end tests have their own problems.
        
               | Jtsummers wrote:
               | That stretches the idea of a refactor, though. If a
               | refactor changes the external behavior it's not really a
               | refactor, which is a structural change. Unit tests when
               | applied at an API level (which may be a very small unit
               | or up to the level of a library, but that also stretches
               | the definition of "unit test" depending on the size of
               | the library) are there to ensure that changes to the
               | _internals_ don 't impact the behavior.
               | 
               | As soon as you start changing the behavior, you have to
               | change the unit tests. If you're adding behavior, you
               | have to add tests. If you're removing behavior, you
               | remove tests. If you're changing the way a procedure
               | works, you change the related unit tests.
               | 
               | Really, any behavior change requires changes to the tests
               | (whatever level they may be, if you want a high degree of
               | test coverage).
        
               | mgkimsal wrote:
               | And without relatively comprehensive tests, you can't
               | ever tell if a 'refactor' (in the structural sense of the
               | word) actually worked. Did you change internal
               | implementation code without affecting the consumer
               | output? Without tests, you can't reliably tell with a
               | high degree of confidence.
        
               | bluGill wrote:
               | Units by definition are not external behavior. sometimes
               | they change external behavior, but there are a lot of
               | changes that make code cleaner without changing external
               | behavior. All too often I've discovered after a few years
               | that I really need to split some unit into two.
        
               | Jtsummers wrote:
               | To clarify, the unit's own behavior is indeed not
               | external to itself. But the way it behaves from the
               | perspective of a user of the unit is external behavior.
               | Changing the behavior, from that perspective, of a unit
               | will necessarily invalidate its tests (some at least) and
               | is _not_ a refactor. That was my point, if you are
               | changing an API you are not refactoring, you are changing
               | its external behavior.
               | 
               | But this isn't just a problem with unit tests, it's a
               | problem with all tests. And really, it's not a problem.
               | It's the cost of doing business. Anything in flux (where
               | that change changes its behavior) will force you to
               | change anything that depends on it, whether it's a test
               | or a user/client. If you aren't willing to discard the
               | tests then the tests "freeze" the system in place,
               | regardless of their level or kind (unit, integration, end
               | to end).
        
         | Afton wrote:
         | It's about tradeoffs. On one end you have precision, speed,
         | reliability, diagnosability. At the other end you have
         | "realness".
         | 
         | Unit tests fall on the far left, workload tests/E2E
         | tests/testing-in-production fall on the far right.
         | 
         | It turns out that there's no 'wrong' level, there's just
         | different tradeoffs. I've worked at a lot of companies that
         | embraced the realness of E2E tests, but then suffered from the
         | maintenance/performance/diagnosability/instability of those
         | tests. I have colleagues who worked at places that avoided E2E
         | at all costs, and suffered because they would have a green test
         | run, but user scenarios that a simple E2E test would have
         | caught, were completely broken.
         | 
         | IMO there is a lot that can be done to improve E2E testing at
         | most companies, but they definitely have the capacity to add
         | value to your release/testing pipeline.
        
       | mleonhard wrote:
       | I wish there were tools for small teams to achieve this level of
       | sophistication. It seems like only massive corporations can do
       | testing really well, because they can afford to assign multiple
       | engineers to build and maintain their bespoke test systems.
       | 
       | I'm a solopreneur building an app with Flutter. Flutter's testing
       | support is mostly broken and or unwritten. It's very frustrating.
        
       | vthommeret wrote:
       | I just want to plug Playwright by Microsoft as I've been using it
       | over the past month and have had a really great experience with
       | it: https://playwright.dev
       | 
       | It's built by the founders of Puppeteer which came out of the
       | Chrome team. Some things I like about it:
       | 
       | 1. It's reliable and implements auto-waiting as described in the
       | article. You can use modern async/await syntax and it ensures
       | elements are a) attached to the DOM, visible, stable (not
       | animating), can receive events, and are enabled:
       | https://playwright.dev/docs/actionability
       | 
       | 2. It's fast -- It creates multiple processes and runs tests in
       | parallel, unlike e.g. Cypress.
       | 
       | 3. It's cross-browser -- supports Chrome, Safari, and Firefox
       | out-of-the-box.
       | 
       | 4. The tracing tools are incredible, you can step through the
       | entire test execution and get a live DOM that you can inspect
       | with your browser's existing developer tools, see all
       | console.logs, etc...
       | 
       | 5. The developers and community are incredibly responsive. This
       | is one of the biggest ones -- issues are quickly responded to and
       | addressed often by the founders, pull requests are welcomed and
       | Slack is highly active and respectful.
       | 
       | My prior experience with end-to-end tests was that they were
       | highly buggy and unreliable and so Playwright was a welcome
       | surprise and inspired me to fully test all the variations of our
       | checkout flow.
        
         | wereHamster wrote:
         | Have you used Cypress before? If yes I'd be interested in a
         | comparison from your perspective.
        
           | vthommeret wrote:
           | I did, but only very briefly. I originally wasn't looking for
           | an E2E tool but was evaluating another tool for a different
           | problem (Nx) which included Cypress as part of its
           | opinionated defaults.
           | 
           | Cypress was a surprisingly nice experience as well and led me
           | to research other modern e2e tools. Most of the points above
           | can be compared against Cypress -- Playwright supports
           | parallel execution of tests within the same file on the same
           | machine, which Cypress doesn't, and so is much faster.
           | Cypress doesn't use modern async / await syntax. Due to its
           | architecture, Playwright can test across tabs, work with
           | iframes easily, which Cypress can't.
           | 
           | The UI for Cypress's developer tools is nice, but... as I
           | said, Playwright's tracing UI is really excellent and the
           | documentation is also really well done. This is also a
           | personal thing, but I trust tools that came out of browser
           | teams (Chrome) to emulate browsers in a more efficient way,
           | e.g. spinning up cheap, isolated browser contexts in Chrome,
           | the details of waiting for an element to be ready, etc...
           | 
           | Another post on this:
           | https://alisterbscott.com/2021/10/27/five-reasons-why-
           | playwr...
        
       | spuz wrote:
       | The title should mention this article is from 2019. I wonder if
       | Facebook testing practices have changed since then.
        
       | goodusername wrote:
       | A really hard problem that often arises when doing E2E tests, is
       | creating and managing test data.
       | 
       | If you have one or more integrations to external systems, where
       | you cannot control your test data, it becomes much harder to
       | write stable E2E tests.
       | 
       | Some don't have test environments, some have too few. Most don't
       | allow you to setup data easily either way.
       | 
       | You can, of course, mock the external systems, but if they play a
       | large enough part, your tests start looking more like integration
       | tests again, but with the added overhead of something like
       | browser automation.
       | 
       | It's a hard balance to strike.
        
       | sharmin123 wrote:
        
       | 8organicbits wrote:
       | > When an E2E test is failing consistently and nobody cares to
       | fix it, that means the test isn't useful. There's no point in
       | having it around.
       | 
       | I suspect this is a good idea, but it raises some red flags for
       | me. People may not want to fix tests if they don't feel like they
       | have time, or fixing tests will help their promotion (i.e.
       | culture). Of course if you have good engineering culture, this is
       | probably a useful signal for tests to remove.
        
         | domesticsimian wrote:
         | I think maybe the point of that quote is that failing tests are
         | just noise. Either fix them or remove them. If I try to run all
         | tests for some pull request and 20% fail, what does that tell
         | me? In the case where we regularly have some number of
         | continuously failing tests, it doesn't tell me much. Did my PR
         | make it worse? Did my PR make it better? Having continuously
         | failing tests definitely doesn't add value and explicitly makes
         | things harder to reason about when looking at test results.
        
         | avensec wrote:
         | One of the reasons we test is for confidence. If we can't trust
         | a test, it isn't providing value. It may give negative value
         | due to the time required to inspect the failure or general
         | erosion of trust in the test suite.
         | 
         | One pattern that we can apply to increase visibility or
         | ownership is stability metrics. If a test must/should be fixed
         | many times can be teased out once you can view these metrics.
         | On failure, display that this test has passed in this
         | configuration for the past x-amount of runs. - Pass the last
         | 100 runs? High likelihood the test is highlighting a bug and
         | must be engaged on. - 95% pass rate in the last 100 runs? It
         | may be time to quarantine this test and add it to the
         | remediation backlog. Your level of acceptable false-positive
         | rates may differ depending on team velocity and suite runtimes.
         | 
         | "How many tests are in quarantine, what is the average time-to-
         | fix, and what direction is this trending" are valuable metrics
         | that we can utilize to find ownership and highlight the
         | technical debt.
         | 
         | As you said, culture around such patterns isn't always there.
        
       | 0xbadcafebee wrote:
       | This article is spot on. For $LargeNetworkHardwareVendor we
       | maintained three different automation test frameworks for end-to-
       | end testing. Our tests were more abstract functions that were
       | given arguments for a particular test case. Those were then made
       | into collections of tests that could be re-used. A configuration
       | file allowed QE to build new test cases without programming
       | knowledge. QE would write configs and occasionally one or two of
       | them that could code would modify the test framework. All the
       | tests ran in a scheduler from clusters of test-running manager-
       | servers against globally distributed labs of hardware. While
       | teams did have unit tests and functional tests, the end-to-end
       | test was king (and necessary given the multiple levels of
       | interface for that gear)
       | 
       | A lot of reliability in that system came from being able to
       | quickly iterate on different levels of the system. The easier it
       | was to solve a failure where it's happening, the more likely your
       | bugs can be fixed quickly, so you have a healthy system (as
       | opposed to suffering from entropy and tech debt)
        
       | avensec wrote:
       | I appreciate the article for visibility as we could always use
       | more knowledge and partners in the Quality Engineering space.
       | Just recognize that these are relatively low-hanging fruit/early
       | maturity concepts in test engineering.
       | 
       | Are you are discovering these for the first time? Great, happy
       | that you are getting exposed! If you read these and think, "we
       | could utilize these concepts with our engineers(test or not)," I
       | would encourage you to look at it from an organizational
       | perspective. You may want to add someone to your team(s) with
       | these skillsets. Most automation testers understand these
       | concepts well and can help you on the next-level maturity items.
        
       ___________________________________________________________________
       (page generated 2022-01-24 23:19 UTC)