Playwright Browser and Test Automation

by · Published · Updated

Print PDF eBook

If you’ve been following me for a while, you know that I have a thing for browser and test automation.  Previously, I’ve spent quite a bit of time with both LeapWork and Taiko+Gauge from Thoughtworks.

(One of my favorite pieces of production code that I’ve written was a framework for massive scale desktop automation which was used at Pfizer to automate opening and closing Microsoft Project back in the days when Project Server required opening each project to update dependent timelines.  The tool built on top of the framework was used to automate publishing of 1200+ Project plans daily!)

Today, we’re looking at an open source library and toolset from Microsoft called Playwright.

In the JavaScript world, it is still relatively unknown.  The 2020 State of JavaScript survey reveals that it is still quite nascent in its adoption (at least among the JavaScript crowd) with Cypress being far and away the more widely adopted “default” solution for front end testing.

Playwright is, in fact, a spiritual successor of Puppeteer as the two main contributors to the Playwright project were hired from the team that built Puppeteer.

So should you switch from Cypress?  Is it better than Taiko?  Let’s find out.

Playwright Compared

The Gauge team at ThoughtWorks actually has a great blog post comparing the different automation tools.  This handy chart actually sums it up quite well:

Taiko Infographic

Puppeteer, the predecessor of Playwright in some ways, definitely stands out for speed and framework integration but scores quite poorly on several other fronts.

The good news is that the Playwright team has addressed many of these gaps since ThoughtWorks published this blog post.

Cross-browser support?  Easily 4 ⭐⭐⭐⭐ in Playwright as it can automate Chromium, Firefox, and WebKit as well as experimental support for Android and Electron.

Reliability of wait mechanisms? An emphatic 4 ⭐⭐⭐⭐ (watch the video).  In fact, the beauty of Playwright and Taiko is that you almost never have to explicitly wait.  What’s even better with Playwright is that when you do want to wait, it provides mechanisms for waiting on specific network responses.  This solves for one of the biggest challenges with respect to using such end-to-end test automation frameworks which is handling of variation in response times in different environments and under different conditions.  With page.waitForResponse, it’s possible for your code to wait for a specific REST or GraphQL response instead of just blindly waiting for 5000ms.

Ease of test failure analysis? 4 ⭐⭐⭐⭐.  The Playwright trace viewer is fantastic and provides a recording of not only the UI, but also the network and console during the test run.  Each “frame” shows the before and after state of the UI for each step.  It makes is incredibly easy to walk through failed test runs.  Additionally, there are a variety of options for debugging of tests.

Number of languages to author tests? 4 ⭐⭐⭐⭐.  Playwright supports authoring in JS/TS, Python, Java, and .NET!  Because Playwright itself is fundamentally an automation library, it can be used with any test runner including Mocha, Jest, NUnit, and more!

Benjamin Gruenbaum’s writeup and handy comparison tool is a more recent evaluation of the tooling and includes Playwright in his comparisons.

Other Key Features

Both blog posts above highlight some of the reasons why Playwright should be under serious consideration by any team doing front-end automation, but there are many, many more including:

  • Visual comparisons which compare the visual state of the user interface.  This is handy for working with rendered images, charts, graphs, or diagrams or when you need to test for very specific visual layouts.
  • Integrated API testing which allows you to use one toolset for your end-to-end UI tests as well as API tests.
  • Support for Chrome extensions which can be loaded into the Chromium instance.
  • Multi-window, multi-tab support which allows for testing of complex interactions and scenarios including real-time, mutli-user use cases (e.g. chat).

Because core Playwright is simply an automation library, it can even be used with Gauge to provide a declarative BDD style approach to building a test suite when you want documentation/specifications as well as automation.

Closing Thoughts

Where I think Playwright comes up a bit short — at least compared to Taiko — is that it relies heavily on strings in working with the selectors.  This has the downside of being a bit more error prone in the authoring phase compared to Taiko’s approach of using functions.  The other aspect that I miss from Taiko is the REPL mode which is really handy, in my opinion.  But in all other respects, I am a convert; Playwright’s all-around capabilities easily trump these minor gaps.

As mentioned, Playwright itself is not strictly a test framework, it includes tooling for test automation.  It is at its core, a multi-platform, cross-browser browser automation tool.  As such, there are a variety of ways that it can be leveraged to do interesting things such as monitoring, screen capturing, generalized UI automation, and so on.

For any team considering end-to-end test automation, do not miss out on evaluating Playwright.

You may also like...