QA Interview Hub · Knowledge base · Quizzes

Logs and observability

Logs in automation

Browser console and page errors in Playwright

An E2E test can pass green while unhandled exceptions fly in the browser console or background requests return 500. Playwright lets you listen to these events:

  • page.on('console', msg => …) — console.* calls; msg.type() returns 'error', 'warning', 'log' and so on, msg.text() returns the text;
  • page.on('pageerror', err => …) — unhandled exceptions on the page;
  • page.on('requestfailed', req => …) — network failures; page.on('response', …) — responses (you can filter status() >= 500).

The most convenient way is to wrap this in an auto fixture so the rule applies to every test:

import { test as base, expect } from '@playwright/test';

export const test = base.extend<{ browserErrors: string[] }>({
  browserErrors: [
    async ({ page }, use) => {
      const errors: string[] = [];
      page.on('pageerror', (e) => errors.push(e.message));
      page.on('console', (m) => {
        if (m.type() === 'error') errors.push(m.text());
      });
      await use(errors);
      expect(errors, 'unexpected browser errors').toEqual([]);
    },
    { auto: true },
  ],
});

Nuances:

  • subscribe before navigation, otherwise early errors are lost;
  • you need an allowlist of exceptions for known third-party noise (analytics, widgets), otherwise tests turn red over things the team does not control; but the allowlist must be narrow and commented;
  • if a test deliberately triggers an error (a negative scenario), it expects it explicitly rather than switching the check off;
  • errors of a particular resource (for example a 404 on the favicon) can be noise; filter by URL instead of turning everything off.

This way frontend regressions are caught immediately, without a separate "no errors in the console" test.

Server logs in CI, asserting on logs in tests, and logging in the test itself

Server logs as a CI artifact. When e2e fails in CI, a screenshot is not enough: you also need the application logs for the same time. The approach: run services so that their output is saved (docker compose logs --no-color --timestamps > server.log or a file the logger is configured to write) and at the end of the job upload it as an artifact, with a condition that also runs after a failure:

- name: Collect service logs
  if: ${{ !cancelled() }}
  run: docker compose logs --no-color --timestamps > server.log
- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}
  with:
    name: server-logs
    path: server.log
    retention-days: 7

To match a test with entries, every test request carries its own traceparent / X-Request-Id, and the value is added to the report (testInfo.annotations.push or testInfo.attach('server-log', { body, contentType: 'text/plain' })).

Asserting on logs in a test. Sometimes you need to be sure an event was logged (audit, security, "a secret did not reach the log"). This is appropriate when:

  • the log is part of a requirement (action audit, compliance), not incidental text;
  • you search by structured fields and a unique id, not by a substring of the message;
  • the wait uses expect.poll/toPass, because logs reach the store with a delay.

It is brittle when a test checks exact message wording, line order or DEBUG logs: any refactoring breaks the test although the behaviour did not change. What matters to the business is better exposed as an event or a metric with a contract.

Logging in the test itself. Make steps with test.step('…', async () => …) — they land in the report and the trace. Attach diagnostic data (an API response, a relevant log excerpt) with testInfo.attach. Do not leave console.log all over the code: it clutters the output, and in parallel workers lines get mixed.

Secrets and PII in logs, noise and retention

Logs are read by many people and live long, so whatever lands there leaks. Never log: passwords, tokens (access/refresh, API keys), full card numbers, CVV, full authentication request bodies, and personal data (email, phone, address, documents) — it falls under GDPR and other regulations.

What teams do:

  • Masking/redaction in the logger or the OTel Collector: Authorization: Bearer ***, a card becomes **** 4242, an email becomes a hash or a partial form;
  • log a user identifier (an internal id), not an email or a name;
  • an allowlist of fields for requests/responses instead of "log everything, then remove";
  • DEBUG off in production and access control on the logs.

The QA's role: check (manually or with an automated test) that after login, payment and password reset the sensitive values do not appear in logs and error bodies; that secrets are masked in CI logs (GitHub Actions masks secrets values but not values derived from them, such as base64-encoded ones); that test data holds no real personal data.

Noise. If there are hundreds of ERRORs a day and nobody looks, a real problem goes unnoticed. Signs: expected exceptions logged as ERROR (for example "user not found" on a wrong login); the same entry thousands of times per second; a health check spamming INFO. Solutions: correct levels, deduplication, sampling, moving errors to a tracker with grouping.

Retention. Logs are kept for a limited time (days, weeks) because of cost and privacy requirements; audit logs may be kept longer. For a QA this means: read or save logs and artifacts of a failed run before they disappear; attach an excerpt to a bug report rather than a link to an entry that will be deleted.