[HN Gopher] SafeTest: A novel approach to front end testing
___________________________________________________________________
SafeTest: A novel approach to front end testing
Author : kolodny
Score : 33 points
Date : 2024-02-13 16:20 UTC (6 hours ago)
(HTM) web link (netflixtechblog.com)
(TXT) w3m dump (netflixtechblog.com)
| greatjack613 wrote:
| Not sure why this is fundamentally different from existing e2e
| testers with bootstrapping
| kolodny wrote:
| Can you clarify what you mean? Usually, e2e testers don't have
| a bootstrapping stage for app-level changes, only for things
| that can be done via the browser automation APIs.
| foolswisdom wrote:
| My initial impression of this is that it enables controlling just
| how much of the app you want to render for your test, while more
| traditional solutions force you towards either rendering the
| whole app or testing just a component. Is that accurate?
| kolodny wrote:
| Yes, that's correct. You get all the benefits of react-testing-
| library mounting and the ability to test the entire app as a
| unit.
| mdaniel wrote:
| how many billions of dollars and they still host their blog on a
| nagware site SMH It's not like moving would be high drama since
| they already have a custom domain for it
|
| https://scribe.rip/introducing-safetest-a-novel-approach-to-...
| mdaniel wrote:
| it also has no license file, with
| https://github.com/kolodny/safetest/blob/main/package.json#L...
| saying "MIT" but as far as I know it says that a lot and I
| doubt gravely that any court would consider a json file a
| legitimate license declaration
| kolodny wrote:
| Sorry about that. I just added an official LICENSE file
|
| https://github.com/kolodny/safetest/blob/main/LICENSE
| joks wrote:
| Is hitting the (x) on the banner that difficult?
| mdaniel wrote:
| Is having nagware for another company a good look for an
| engineering blog?
|
| > How we GraphQL all the things, after this brief message
| from our ... sponsor?
| MajimasEyepatch wrote:
| From the Hacker News Guidelines: "Please don't complain about
| tangential annoyances--e.g. article or website formats, name
| collisions, or back-button breakage. They're too common to be
| interesting."
|
| https://news.ycombinator.com/newsguidelines.html
| zie wrote:
| I've always wanted to build a UX tester that uses accessibility
| tech. We get to test both at the same time, and it might make the
| accessibility tech saner and easier to use also.
| drewcoo wrote:
| > Conversely, using integration testing tools like Cypress or
| Playwright provides control over the page, but sacrifices the
| ability to instrument the bootstrapping code for the app. These
| tools operate by remotely controlling a browser to visit a URL
| and interact with the page.
|
| I don't think the author has used Cypress or Playwright. Their
| real value is that they do not drive browsers from the outside
| like slow, flakey WebDriver. They also allow injecting test
| doubles. And to override app functionality (not having to wait
| for a "normal" 60 second timeout like the author hypothetically
| suggests).
|
| This test framework seems to solve non-problems.
|
| Is Netflix not quite so FAANG these days?
| kolodny wrote:
| Author here. Can you show an example of how Playwright would
| progress a timer on a page? For example, how would you make
| this component pass faster than 60 seconds?
| export const Countdown = () => { const [time,
| setTime] = React.useState(60); React.useEffect(() =>
| { const interval = setInterval(() => {
| setTime(t => t - 1); if (t === 1)
| clearInterval(interval); }, 1000);
| return () => clearInterval(interval) }, []);
| return time > 0 ? <>Time left {time}</> : <>Done</> };
|
| Short of using Playwright Component Tests, this isn't possible.
|
| I also found an issue where the only solution was to inject
| sinon into the page which I wouldn't consider a great option:
| https://github.com/microsoft/playwright/issues/10230
| drewcoo wrote:
| If you want to do it with the whole page and talk only to the
| local code, then yes, I'd recommend Sinon. I think that's a
| much simpler solution than . . . creating an all new NIH
| framework!
|
| I'd also recommend refactoring to a more mock-friendly way to
| do that countdown if you don't want to cover up all the
| internal logic.
|
| If the timeout interval is loaded remotely from some API (and
| it probably is if you have reasonably configurable nag
| popups), then you can always mock that API call.
| kolodny wrote:
| The point is that you shouldn't need to rewrite your
| countdown component to allow testing. Can you provide a
| snippet of that change and what the test would look like?
|
| Not being toggle parts of the app is the root of the issue
| when creating e2e tests. For example overriding a feature
| flag, you could find the API call for the feature but what
| if it's a grpc call and part of your dev build pulls in any
| update, you can't easily change the binary response with
| confidence anymore.
|
| The current solutions are good enough to do smoke tests,
| but nitty-gritty e2e tests don't scale well.
|
| In the example above it's simple export
| const CountdownTime = createOverride(60); export
| const Countdown = () => { const initialTime =
| CountdownTime.useValue(); const [time, setTime] =
| React.useState(initialTime); React.useEffect(()
| => { const interval = setInterval(() => {
| setTime(t => t - 1); if (t === 1)
| clearInterval(interval); }, 1000);
| return () => clearInterval(interval) }, []);
| return time > 0 ? <>Time left {time}</> : <>Done</>
| }; it('tests faster', async () => {
| const { page } = await render( app =>
| <CountdownTime.Override with={() =>
| 5}>{app}</CountdownTime.Override> );
| await expect(page.getByText('Done')).not.toBeVisible();
| page.waitForTimeout(5000); await
| expect(page.getByText('Done')).toBeVisible(); });
|
| If I needed something like this, I'd probably also make
| setInterval an override as well, so I don't need to wait at
| all, but you get the idea.
| throwaway290 wrote:
| I like the library, but isn't createOverride / useValue
| basically modifying components to allow testing?
| kolodny wrote:
| Creating an override is basically just providing a
| placeholder for a value to be injected via React Context.
| I view this as a form of dependency injection. Contrast
| this with how this would be done in vanilla Playwright
| with reading it from a query param or exposing a global
| to call page.evaluate on which is more along the lines of
| forcing test code into a component.
|
| Note that if you needed a specific reference in an
| override there isn't a good way to get that via
| Playwright, consider this silly example:
| it('can test spies', async () => { const spy =
| browserMock.fn(); const { page } = await
| render( app =>
| <LogoutButtonAction.Override with={() => spy}>
| {app} <LogoutButtonAction.Override>
| ); await page.getByText('Log
| out').click(); expect(await
| spy).toHaveBeenCalled(); });
| bryanrasmussen wrote:
| Are there potential performance issues to consider with overrides
| - I'm supposing
|
| const FetchPerson = createOverride(fetchPerson);
|
| must be less performant by some amount than just using
| fetchPerson? How many over-rides do you find a reasonably complex
| component needs?
| kolodny wrote:
| React is pretty performant when context isn't changing. We
| haven't done any benchmarking but I doubt there's any real
| world perf hit. For large applications the number of overrides
| tend to be under 20.
|
| Overrides are opt-in so you can just expose any overridable
| value as a prop and run a isolated component test on it.
___________________________________________________________________
(page generated 2024-02-13 23:01 UTC)