Config, projects, parallelism, CI

Configuration and projects

The main file is playwright.config.ts with defineConfig. At the top level you set testDir, timeout, retries, workers, fullyParallel, reporter and webServer for automatically starting the application before tests. The use section sets settings for all tests: baseURL, trace, screenshot, video, viewport, locale.

Projects are sets of tests with different settings: different browsers, mobile devices through devices['iPhone 13'], different roles or environments. Each project can override use, and have its own testMatch and dependencies. A specific project is run with the flag --project=chromium.

Typical values for CI: retries: process.env.CI ? 2 : 0, forbidOnly: !!process.env.CI, so that a forgotten test.only doesn't get into the pipeline. Also trace: 'on-first-retry' and screenshots only on failure. Environment variables are loaded through dotenv, and secrets come from CI.

Parallelism, workers and sharding

Playwright runs tests in parallel in workers, which are separate processes. By default files run in parallel and tests inside a file sequentially; fullyParallel: true also parallelises tests within a file. The number of workers is set through workers, and on CI it is often reduced because of limited resources.

test.describe.configure({ mode: 'serial' }) runs a block's tests sequentially, and if one fails, the rest are skipped: this is for dependent steps, but it is better to make tests independent. The parallel mode, conversely, parallelises tests in a particular file. For parallelism each test must work with its own data.

Sharding splits tests between several machines: npx playwright test --shard=1/4, in GitHub Actions through a matrix. Each shard produces its own report, so the blob reporter is used, the artifacts of all jobs are collected and merged with npx playwright merge-reports. The distribution goes by files, so files of uneven size give a skew between shards.

Timeouts and the priority of settings

There are several timeouts, and they are often confused:

"No limit" means an action is in effect limited only by the test timeout. In practice I set actionTimeout to about 10-15 seconds: then a specific click will fail with a clear error, and not the whole test by timeout without details.

test.slow() triples a test's timeout, and test.setTimeout(60_000) sets a specific value. For a single assertion: toBeVisible({ timeout: 15_000 }).

Settings have a hierarchy, from general to specific:

1. The config's top level and use.
2. The use of a specific project.
3. test.use in a file or describe.
4. The options of a specific action or assertion.

A lower level overrides a higher one. test.use can't be called inside a test body, only at file or describe level, because it may require a new context.

globalTimeout is useful on CI: if something hangs, the run will stop instead of eating hours of the runner. maxFailures plays a similar role: stop the run after N failures when the environment is obviously down.

webServer and environment variables

The webServer option starts the application before the tests and waits until it responds:

export default defineConfig({
  webServer: [
    {
      command: 'npm run start:api',
      url: 'http://localhost:4000/health',
      reuseExistingServer: !process.env.CI,
      timeout: 120_000,
    },
    {
      command: 'npm run start:web',
      url: 'http://localhost:3000',
      reuseExistingServer: !process.env.CI,
    },
  ],
  use: { baseURL: process.env.BASE_URL ?? 'http://localhost:3000' },
});

reuseExistingServer lets you use an already running server locally, while on CI it forces a fresh one to start. You can pass an array if you need to bring up a backend and a front end. After the run Playwright stops the processes itself.

For several environments (dev, stage) I keep .env.dev and .env.stage, load the needed one through dotenv at the start of the config by a variable, for example TEST_ENV, and take baseURL from it. Secrets don't sit in the repository: locally they are in .env, which is in .gitignore, and on CI in the pipeline's secrets.

baseURL lets you write page.goto('/login') and request.get('/api/users') with relative paths, so the same set of tests runs on any environment by changing one variable.

CI: sharding, Docker and fast reruns

A typical pipeline in GitHub Actions with sharding:

jobs:
  e2e:
    strategy:
      fail-fast: false
      matrix:
        shard: [1/4, 2/4, 3/4, 4/4]
    runs-on: ubuntu-latest
    container: mcr.microsoft.com/playwright:v1.55.0-noble
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright test --shard=${{ matrix.shard }} --reporter=blob
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-${{ strategy.job-index }}
          path: blob-report

A separate job after the shards downloads all the artifacts and runs npx playwright merge-reports --reporter=html ./all-blob-reports.

Details worth knowing:

• fail-fast: false, so that a failure of one shard doesn't cancel the others.
• The Docker image version must match the @playwright/test version, otherwise the browsers won't be found. The image provides identical fonts, which is critical for screenshot tests.
• With fullyParallel: true sharding distributes individual tests, not files, so shards are balanced more evenly.
• workers on CI are set from the runner's resources, for example '50%' or 1-2 on weak machines.

For faster work there are flags: --last-failed reruns only the tests that failed last time, and --only-changed runs tests from files changed relative to HEAD or a given branch (--only-changed=main). The first is handy locally after a failure, the second for quick feedback in a PR before the full run.