[HN Gopher] Playwright Test Generator
___________________________________________________________________
Playwright Test Generator
Author : bovem
Score : 62 points
Date : 2024-03-14 11:06 UTC (11 hours ago)
(HTM) web link (playwright.dev)
(TXT) w3m dump (playwright.dev)
| __jonas wrote:
| It works quite decently. I've noticed a nice side-effect of
| setting up playwright tests is that it helps catch accessibility
| issues, like interactive elements not being properly labeled as
| such.
| baal80spam wrote:
| As a SDET, my question is - does anyone uses tools such as this
| in professional work? If yes, what is the use case?
|
| IME, creating and developing an automated test framework is 1%
| trying to find the best locator, and 99% maintenance of tests
| that use dynamic, nondeterministic flows, fighting with backend
| issues/flakiness etc. I find Codegen to be OK-ish for only the
| simplest/worthless test paths.
| baggachipz wrote:
| Yes, my company uses them to verify that nothing serious breaks
| as a result of deployment to staging. Of course we have unit
| and integration tests too, but this is used as a last line of
| defense against breaking things entirely.
|
| It cries wolf a lot, but it's also saved our ass several times.
| So we run 3 attempts per test and if all 3 fail, sound the
| alarm.
| throwaway74432 wrote:
| I think they meant the code generator tool.
| baggachipz wrote:
| oh. Carry on then.
| baal80spam wrote:
| Yes, that's true even though it might not be clear from my
| comment! I use Playwright on a daily basis, it's a great
| tool.
| throwaway74432 wrote:
| I tend to agree. An example of how it can be net-zero gain is
| in this screenshot[1]. To have something maintainable, you have
| to do your own refactoring pass on the generated code. At that
| point, it's not much effort to write the test yourself than it
| is to refactor all of the redundancies.
|
| 1.
| https://github.com/microsoft/playwright/assets/13063165/7794...
| space_ghost wrote:
| 100% agree, that screenshot is a _perfect_ example of why I
| don't use these tools.
| CharlieDigital wrote:
| Playwright has some pretty good strategies for selection using
| positional locators[0] which generally feel pretty resilient.
| Not quite as ergonomic as Taiko[1], but still really good.
|
| Both have good heuristics for auto-waiting that simplify
| interactions and make them "faster" because there's not as much
| wasted time doing manual waits.
|
| I particularly liked Taiko + Gauge[2], but Playwright can also
| be plugged into Gauge and the ergonomics are really nice
| because then you can write "plain English" test scripts that
| translate into system actions.
|
| I have two videos that go over both:
|
| - Playwright: https://www.youtube.com/watch?v=qYkphCJjD_k
|
| - Taiko: https://www.youtube.com/watch?v=i-sMXPV547
|
| [0]
| https://docs.gauge.org/?os=linux&language=javascript&ide=vsc...
|
| [1] https://docs.taiko.dev/api/reference/
|
| [2]
| https://docs.gauge.org/?os=linux&language=javascript&ide=vsc...
| martypitt wrote:
| The Taiko link you posted doesn't work - I think it's missing
| an s at the end - here's the working link:
|
| https://www.youtube.com/watch?v=i-sMXPV547s
| space_ghost wrote:
| SDET here, and no, I don't use these tools. Hammering out the
| actual test code is the easiest (and most fun!) part of the job
| for me, tools like this one actually make that experience
| painful and slower. :(
| cebert wrote:
| We use Playwright and the Page Object Model approach to test
| our applications because that approach is more maintainable for
| larger applications. Automated E2E testing is something all
| developers are expected to do, not just an "SDET" engineer or
| QA engineer role. While we take the POM approach, I frequently
| use Playwright's recorder to help me get selectors and
| interactions with the DOM needed to verify a captured feature.
| Next, I take the code generated and incorporate it into object
| models. Adding an automation attribute to all of your elements
| that are both unique and stable helps go a long way to make E2E
| tests enjoyable.
| baal80spam wrote:
| > Adding an automation attribute to all of your elements that
| are both unique and stable helps go a long way to make E2E
| tests enjoyable.
|
| First of all maybe it wasn't clear from my comment but I use
| (and like it a lot) Playwright on a daily basis.
|
| Second - did you mean using a testid for example? Because we
| specifically avoid using special attributes for this purpose
| and instead follow best practices as detailed here:
| https://testing-library.com/docs/queries/about#priority (main
| reason being: killing two birds with one stone by making sure
| your page is accessible).
| ddoice wrote:
| Imho IDs are more explicit, and a bless when working with
| large codebase apps, when trying to finding the related
| code to an element just grab the id and search for it.
| willio58 wrote:
| > creating and developing an automated test framework is 1%
| trying to find the best locator, and 99% maintenance of tests
| that use dynamic.
|
| This is completely true. Quality e2e tests that have good setup
| and cleanup and aren't flaky take focused dev work. I find
| myself regularly using the locator finder tool in playwright
| UI/debugger but I've never found any real value in using the
| test generator.
|
| I could see it being valuable in a really small subset of
| exceedingly simple UI, probably personal-project level but even
| those tend to grow too complex for this.
| fwlr wrote:
| I do use Playwright's codegen a lot and I have found it most
| valuable when I mentally predict what it should generate and
| then look to see how it differs. This often highlights
| incorrect assumptions. I would say approximately 1% of the
| generated code ends up in the tests, but about 30% of the
| generated code informs how the tests are written.
| liquidpele wrote:
| We tried to get a QA team to use it once... it was apparently
| too technical for anyone but the devs, so it never really got
| used much.
| px1999 wrote:
| My org uses codegen as a starting point for one of our test
| layers.
|
| It works for us probably because we sidestep the pain points
| you list - the environments we run in are pristine complete
| copies of known datasets, we remove as many sources of
| randomness as possible, and our environment flakiness level is
| very low.
|
| They still break but usually because the locators in use have
| been chosen poorly (or we've made planned changes to a
| page/component)
|
| We're a web based b2b saas that runs an instance of the entire
| environment for each of our customers. Our non prod setup
| consists of a bajillion static test environments but more
| importantly we use testcontainers to spin up the transient test
| environments from database snapshots. Using the recorder on the
| static environments (before the transient ones existed) _was_ a
| pain
| mherrmann wrote:
| Unreasonably shameless plug for my open source Python library
| Helium [1], so you don't need a test generator.
|
| 1: https://github.com/mherrmann/helium
| latchkey wrote:
| Wow, that looks really nice! Thanks for the plug.
| cute_boi wrote:
| I think this doesn't work with tsx tool, which is kinda annoying
| :(
| rybosworld wrote:
| Probably worth pointing out, this isn't new to playwright, and
| it's not a new idea in general (Selenium IDE).
___________________________________________________________________
(page generated 2024-03-14 23:01 UTC)