How to Run Regression Tests with Playwright: Setup, Parallelism, and CI

Set Up the Project for Regression Testing

Install Playwright and its test runner in your project. The npm init playwright command scaffolds the project with a configuration file, an example test, and the browser binaries. For regression suites, the configuration file is where most of the important decisions live.

In playwright.config.ts, define projects for the browsers you want to cover. A typical regression configuration runs tests on Chromium, Firefox, and WebKit, giving you three-engine coverage from one test file. Each project can have its own settings, so you can run Chromium tests on every PR and the full cross-browser suite nightly by creating separate project groups.

Set the retries option to 1 or 2 for CI runs. In regression testing, a test that fails once but passes on retry is a flaky test, and the retry result is the one that matters for the merge decision. Playwright reports the retry status so you can identify and track flaky tests separately from hard failures. Set retries to 0 for local development so flakiness is visible immediately rather than hidden by retries.

Configure the reporter to produce both a human-readable terminal output and a machine-readable format like JUnit XML or JSON for CI integration. The JUnit reporter integrates with GitHub Actions, GitLab CI, and most CI platforms to display test results directly in the PR interface.

Write Regression Tests That Stay Reliable

The number one reason regression suites become a burden is test brittleness: tests that break when the UI changes even though the behavior is unchanged. Playwright provides tools to prevent this, but they require conscious use.

Use role-based and text-based locators as your default strategy. page.getByRole('button', { name: 'Submit Order' }) finds the submit button regardless of its CSS class, ID, or position in the DOM. This locator survives redesigns, component library migrations, and markup refactors because it describes what the element is rather than how it is implemented. Reserve CSS and XPath selectors for elements that cannot be identified by role or text, and use data-testid attributes as a fallback when neither works.

Build page objects to absorb selector changes. A LoginPage class with a login(username, password) method contains the selectors for the username field, password field, and submit button in one place. When a redesign changes the form, one file updates. Without page objects, every test that logs in updates individually, and missing one is a maintenance regression that produces a false failure.

Isolate test state so tests can run in any order. Each test should create its own data, execute its scenario, and leave no state behind. Playwright's browser context isolation helps: each test gets a fresh browser context with no cookies, local storage, or cache from previous tests. For server state, use API calls in beforeEach hooks to set up the necessary data rather than depending on data created by other tests.

Use Playwright's web-first assertions, like expect(locator).toBeVisible() and expect(locator).toHaveText(), which automatically wait and retry until the condition is met or the timeout expires. These eliminate the race conditions that cause most timing-related flakiness in browser tests. Never use page.waitForTimeout() as a stability mechanism, because a fixed delay is either too short (and flaky) or too long (and slow).

Enable Parallel Execution and Sharding

Playwright runs tests in parallel by default, distributing test files across worker processes up to the number of CPU cores. For a regression suite of 100 test files, a machine with 4 cores runs 4 tests simultaneously, cutting wall time to roughly one quarter of sequential execution.

For CI, sharding distributes tests across multiple machines. The command npx playwright test --shard=1/4 runs the first quarter of the test files on one machine, --shard=2/4 runs the second quarter on another, and so on. Four CI runners executing in parallel produce results in about one quarter of the single-runner time. The cost is compute hours (the same total work happens), but developer wait time drops dramatically, which is almost always the higher-value metric.

Configure sharding in your CI workflow by creating a matrix of shard indices. In GitHub Actions, a matrix with shard: [1, 2, 3, 4] and total-shards: 4 creates four parallel jobs. Each job runs npx playwright test --shard=$shard/4 and uploads its test results as an artifact. A final job merges the shard results into a single report using npx playwright merge-reports. This pattern scales linearly: doubling the shard count halves the wall time.

For suites that mix fast and slow tests, consider running all tests in parallel at the file level (the default) rather than using fullyParallel mode, which parallelizes individual tests within a file. File-level parallelism provides good distribution without the complexity of ensuring every test within a file is truly independent. If specific test files are much slower than others, split them into smaller files for better load balancing across workers.

Configure Trace and Artifact Collection

When a regression test fails, the trace is the evidence that makes investigation fast. Playwright's trace files capture a complete step-by-step replay of the test execution: screenshots at every action, DOM snapshots, network requests and responses, console logs, and the source code location of each step.

Set trace: 'on-first-retry' in playwright.config.ts. This records a trace only when a test fails on the first attempt and retries. The trace for the retry captures the failure conditions without the storage overhead of tracing every passing test. For debugging specific tests locally, run with --trace on to capture traces for all tests.

Configure screenshot: 'only-on-failure' and video: 'retain-on-failure' for additional failure evidence. Screenshots provide a quick visual check, and videos show the full interaction sequence. All three artifacts, traces, screenshots, and videos, should be uploaded to CI artifacts so developers can download and review them without reproducing the failure locally.

Open traces with npx playwright show-trace trace.zip. The Trace Viewer is a browser application that plays back the test step by step, showing the page state, network activity, and console output at each point. For regression investigation, it answers "what did the page look like when the assertion failed" without any local reproduction, which turns a potentially hour-long investigation into a 5-minute review.

Integrate with CI/CD

The regression suite delivers value only when it runs automatically on every change. Wire it into your CI pipeline with these components.

In GitHub Actions, install Playwright browsers using the official action (microsoft/playwright-github-action or npx playwright install --with-deps in the pipeline). Cache the browser binaries to avoid re-downloading them on every run, which saves 1 to 2 minutes per job.

Create separate workflow triggers for different regression depths. On pull requests, run the Chromium-only project with a subset of shards for fast feedback. On merges to main, run the full cross-browser suite with maximum sharding. On a nightly schedule, run the complete suite including slow tests that are tagged out of the PR run.

Use Playwright's tag system to control which tests run at each trigger. Tag tests with @critical, @slow, @visual, or custom labels, then filter with --grep or --grep-invert on the command line. A PR workflow that runs only @critical tests gives fast feedback on the most important paths, while the nightly workflow runs everything including @slow and edge-case tests.

Upload test results and artifacts as CI artifacts for every run. Configure the JUnit reporter to produce a results file that your CI platform can parse and display inline. GitHub Actions' test results visualization, GitLab's test report widget, and similar features make regression failures visible directly in the PR interface, which speeds up the feedback loop from "test failed" to "developer knows about it."

See the CI/CD regression guide for detailed pipeline examples and the CI/CD test automation overview for how regression fits alongside other test types in the pipeline.