https://hacks.mozilla.org/2024/08/puppeteer-support-for-firefox/

[mdn-logo-m] Mozilla Hacks

[                    ]
Hacks on YouTube @mozhacks on Twitter Hacks RSS Feed Download Firefox

Announcing Official Puppeteer Support for Firefox

Avatar photo By James Graham, Henrik Skupin, Julian Descottes, 
Alexandra Borovova

Posted on August 7, 2024 in Featured Article, Firefox, and Web
Developers

We're pleased to announce that, as of version 23, the Puppeteer
browser automation library now has first-class support for Firefox.
This means that it's now easy to write automation and perform
end-to-end testing using Puppeteer, and run against both Chrome and
Firefox.

How to Use Puppeteer With Firefox

To get started, simply set the product to "firefox" when starting
Puppeteer:

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  browser: "firefox"
});

const page = await browser.newPage();
// ...
await browser.close();

As with Chrome, Puppeteer is able to download and launch the latest
stable version of Firefox, so running against either browser should
offer the same developer experience that Puppeteer users have come to
expect.

Whilst the features offered by Puppeteer won't be a surprise,
bringing support to multiple browsers has been a significant
undertaking. The Firefox support is not based on a Firefox-specific
automation protocol, but on WebDriver BiDi, a cross browser protocol
that's undergoing standardization at the W3C, and currently has
implementation in both Gecko and Chromium. This use of a
cross-browser protocol should make it much easier to support many
different browsers going forward.

Later in this post we'll dive into some of the more technical
background behind WebDriver BiDi. But first we'd like to call out how
today's announcement is a great demonstration of how productive
collaboration can advance the state of the art on the web. Developing
a new browser automation protocol is a lot of work, and great thanks
goes to the Puppeteer team and the other members of the W3C Browser
Testing and Tools Working Group, for all their efforts in getting us
to this point.

You can also check out the Puppeteer team's post about making
WebDriver BiDi production ready.

Key Features

For long-time Puppeteer users, the features available are familiar.
However for people in other automation and testing ecosystems --
particularly those that until recently relied entirely on HTTP-based
WebDriver -- this section outlines some of the new functionality that
WebDriver BiDi makes possible to implement in a cross-browser manner.

Capturing of Log Messages

A common requirement when testing web apps is to ensure that there
are no unexpected errors reported to the console. This is also a case
where an event-based protocol shines, since it avoids the need to
poll the browser for new log messages.

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  browser: "firefox"
});

const page = await browser.newPage();
page.on('console', msg => {
  console.log(`[console] ${msg.type()}: ${msg.text()}`);
});

await page.evaluate(() => console.debug('Some Info'));
await browser.close();

Output:

[console] debug: Some Info

Device Emulation

Often when testing a reactive layout it's useful to be able to ensure
that the layout works well at multiple screen dimensions, and device
pixel ratios. This can be done by using a real mobile browser, either
on a device, or on an emulator. However for simplicity it can be
useful to perform the testing on a desktop set up to mimic the
viewport of a mobile device. The example below shows loading a page
with Firefox configured to emulate the viewport size and device pixel
ratio of a Pixel 5 phone.

import puppeteer from "puppeteer";

const device = puppeteer.KnownDevices["Pixel 5"];

const browser = await puppeteer.launch({
  browser: "firefox"
});

const page = await browser.newPage();
await page.emulate(device);

const viewport = page.viewport();

console.log(
  `[emulate] Pixel 5: ${viewport.width}x${viewport.height}` +
  ` (dpr=${viewport.deviceScaleFactor}, mobile=${viewport.isMobile})`
);

await page.goto("https://www.mozilla.org");
await browser.close();

Output:

[emulate] Pixel 5: 393x851 (dpr=3, mobile=true)

Network Interception

A common requirement for testing is to be able to track and intercept
network requests. Interception is especially useful for avoiding
requests to third party services during tests, and providing mock
response data. It can also be used to handle HTTP authentication
dialogs, and override parts of the request and response, for example
adding or removing headers. In the example below we use network
request interception to block all requests to web fonts on a page,
which might be useful to ensure that these fonts failing to load
doesn't break the site layout.

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  browser: 'firefox'
});

const page = await browser.newPage();
await page.setRequestInterception(true);

page.on("request", request => {
  if (request.url().includes(".woff2")) {
    // Block requests to custom user fonts.
    console.log(`[intercept] Request aborted: ${request.url()}`);
    request.abort();
  } else {
    request.continue();
  }
});

const response = await page.goto("https://support.mozilla.org");
console.log(
  `[navigate] status=${response.status()} url=${response.url()}`
);
await browser.close();

Output:

[intercept] Request aborted: https://assets-prod.sumo.prod.webservices.mozgcp.net/static/Inter-Bold.3717db0be15085ac.woff2
[navigate] status=200 url=https://support.mozilla.org/en-US/

Preload Scripts

Often automation tooling wants to provide custom functionality that
can be implemented in JavaScript. Whilst WebDriver has always allowed
injecting scripts, it wasn't possible to ensure that an injected
script was always run before the page started loading, making it
impossible to avoid races between the page scripts and the injected
script.

WebDriver BiDi provides "preload" scripts which can be run before a
page is loaded. It also provides a means to emit custom events from
scripts. This can be used, for example, to avoid polling for expected
elements, but instead using a mutation observer that fires as soon as
the element is available. In the example below we wait for the
<title> element to appear on the page, and log its contents.

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  browser: 'firefox',
});

const page = await browser.newPage();

const gotMessage = new Promise(resolve =>
  page.exposeFunction("sendMessage", async message => {
    console.log(`[script] Message from pre-load script: ${message}`);
    resolve();
  })
);

await page.evaluateOnNewDocument(() => {
  const observer = new MutationObserver(mutationList => {
    for (const mutation of mutationList) {
      if (mutation.type === "childList") {
        for (const node of mutation.addedNodes) {
          if (node.tagName === "TITLE") {
            sendMessage(node.textContent);
          }
        }
      }
    };
  });

  observer.observe(document.documentElement, {
    subtree: true,
    childList: true,
  });
});

await page.goto("https://support.mozilla.org");
await gotMessage;
await browser.close();

Output:

[script] Message from pre-load script: Mozilla Support

Technical Background

Until recently people wishing to automate browsers had two main
choices:

  * Use the W3C WebDriver API, which was based on earlier work by the
    Selenium project.
  * Use a browser-specific API for talking to each supported browser
    such as Chrome DevTools Protocol (CDP) for Chromium-based
    browsers, or Firefox's Remote Debugging Protocol (RDP) for
    Gecko-based browsers.

Unfortunately both of those options come with significant tradeoffs.
The "classic" WebDriver API is HTTP-based, and its model involves
automation sending a command to the browser and waiting for a
response. That works well for automation scenarios where you load a
page and then verify, for example, that some element is displayed,
but the inability to get events -- e.g. console logs -- back from the
browser, or run multiple commands concurrently, makes the API a poor
fit for more advanced use cases.

By contrast, browser-specific APIs have generally been designed
around supporting the complex use cases of in-browser devtools. This
has given them a feature set far in advance of what's possible using
WebDriver, as they need to support use cases such as recording
console logs, or network requests.

Therefore, browser automation clients have been forced to make the
choice between supporting many browsers using a single protocol and
providing a limited feature set, or providing a richer feature set
but having to implement multiple protocols to provide functionality
separately for each supported browser. This obviously increased the
cost and complexity of creating great cross-browser automation, which
isn't a good situation, especially when developers commonly cite
cross-browser testing as one the main pain points in developing for
the web.

Long time developers might notice the analogy here to the situation
with editors before the development of Language Server Protocol
(LSP). At that time each text editor or IDE had to implement bespoke
support for each different programming language. That made it hard to
get support for a new language into all the tools that developers
were using. The advent of LSP changed that by providing a common
protocol that could be supported by any combination of editor and
programming language. For a new programming language like TypeScript
to be supported across all editors it no longer needs to get them to
add support one-by-one; it only needs to provide an LSP server and it
will automatically be supported across any LSP-supporting editor. The
advent of this common protocol has also enabled things that were hard
to imagine before. For example specific libraries like Tailwind
getting their own LSP implementation to enable bespoke editor
functionality.

So to improve cross-browser automation we've taken a similar
approach: developing WebDriver BiDi, which brings the automation
featureset previously limited to browser-specific protocols to a
standardized protocol that can be implemented by any browser and used
by any automation tooling in any programming language.

At Mozilla we see this strategy of standardizing protocols in order
to remove barriers to entry, allow a diverse ecosystem of
interoperable implementations to flourish, and enable users to choose
those best suited to their needs as a key part of our manifesto and
web vision.

For more details about the design of WebDriver BiDi and how it
relates to classic WebDriver, please see our earlier posts.

Removing experimental CDP support in Firefox

As part of our early work on improving cross-browser testing, we
shipped a partial implementation of CDP, limited to a few commands
and events needed to support testing use cases. This was previously
the basis of experimental support for Firefox in Puppeteer. However,
once it became clear that this was not the way forward for
cross-browser automation, effort on this was stopped. As a result it
is unmaintained and doesn't work with modern Firefox features such as
site isolation. Therefore support is scheduled to be removed at the
end of 2024.

If you are currently using CDP with Firefox, and don't know how to
transition to WebDriver BiDi, please reach out using one of the
channels listed at the bottom of this post, and we will discuss your
requirements.

What's Next?

Although Firefox is now officially supported in Puppeteer, and has
enough functionality to cover many automation and testing scenarios,
there are still some APIs that remain unsupported. These broadly fall
into three categories (consult the Puppeteer documentation for a full
list):

  * Highly CDP-specific APIs, notably those in the CDPSession module.
    These are unlikely to be supported directly, but specific use
    cases that currently require these APIs could be candidates for
    standardization.
  * APIs which require further standards work. For example
    page.accessibility.snapshot returns a dump of the Chromium
    accessibility tree. However because there's currently no
    standardized description of what that tree should look like this
    is hard to make work in a cross-browser way. There are also cases
    which are much more straightforward, as they only require work on
    the WebDriver BiDi spec itself; for example page.setGeolocation.
  * APIs which have a standard but are not yet implemented, for
    example the ability to execute scripts in workers required for
    commands like WebWorker.evaluate.

We expect to fill these gaps going forward. To help prioritize, we're
interested in your feedback: Please try running your Puppeteer tests
in Firefox! If you're unable to get them in Firefox because of a bug
or missing feature, please let us know using one of the methods below
so that we can take it into account when planning our future
standards and implementation work:

  * For Firefox implementation bugs, please file a bug on Bugzilla
  * If you're confident that the issue is in Puppeteer, please file a
    bug in their issue tracker.
  * For features missing from the WebDriver BiDi specification,
    please file an issue on GitHub
  * If you want to talk to us about use cases or requirements, please
    use the #webdriver channel on Mozilla's Matrix instance or email
    dev-webdriver@mozilla.org.

About James Graham

Software engineer focused on maintaining a healthy open web.
Web-platform-tests core team member.

  * https://hoppipolla.co.uk

More articles by James Graham...

About Henrik Skupin

  * https://www.hskupin.info/
  * @whimboo

More articles by Henrik Skupin...

About Julian Descottes

  * @juliandescottes

More articles by Julian Descottes...

About Alexandra Borovova

  * @lutien02

More articles by Alexandra Borovova...

Discover great resources for web development

Sign up for the Mozilla Developer Newsletter:

E-mail [                              ]
[ ] I'm okay with Mozilla handling my info as explained in this
Privacy Policy.
Sign up now
Thanks! Please check your inbox to confirm your subscription.

If you haven't previously confirmed a subscription to a
Mozilla-related newsletter you may have to do so. Please check your
inbox or your spam filter for an email from us.

Except where otherwise noted, content on this site is licensed under
the Creative Commons Attribution Share-Alike License v3.0 or any
later version.

the Mozilla dino logo