Types of testing

smoke, regression, exploratory and more

Functional and non-functional testing: what is the difference?

Functional is what the system does: logic, business rules, conformance to requirements.

Non-functional is how it does it: performance (load, stress), security, usability, accessibility, compatibility, reliability, localisation.

In an interview: Functional testing checks what the system does: does a transfer work, is the fee calculated correctly. Non-functional checks how it does it: speed, security, usability, browser compatibility.

How do smoke, sanity and regression testing differ?

Kind	What it checks	When
Smoke	The critical functions work, the build can be tested at all	On every new build
Sanity	A specific change or fix works, narrow and deep	After a small fix
Regression	Changes didn't break what worked before	Before a release, after changes

In an interview: Smoke is a broad shallow check that the build is alive and the main functions work. Sanity is a narrow check of a specific change or fix. Regression is a check that new changes didn't break existing functionality, and it is the widest of the three.

How does a retest differ from regression testing?

A retest is checking that a specific bug is fixed: we go through the same steps as in the bug report.

Regression is checking that a fix or a new feature didn't break anything else. A retest checks one place, regression checks everything around it.

In an interview: A retest is checking that a specific bug is fixed, by the steps from the report. Regression is checking that this fix or new feature broke nothing nearby. After a fix I do both: a retest of the bug itself and regression of the related functionality.

What are black box, white box and grey box?

Black box: we test against requirements without knowing the code. The main approach of manual QA.
White box: we test knowing the internal structure: branch and condition coverage. Usually developers.
Grey box: we partly know the internal structure: the database schema, the API, logs. For example, after an action in the UI we check the record in the DB or the request in DevTools.

In an interview: Black box is testing against requirements without knowing the code. White box is with full knowledge of the internal logic. Grey box is in between: I don't read the code, but I check the database, API requests and logs. That is how I work most often: an action in the UI, then checking the request in DevTools and the record in the database.

What is exploratory testing and how do you do it in a structured way?

Exploratory testing is studying the product and designing and executing tests at the same time, without ready cases. It is strong where requirements are incomplete or you need to find the most important bugs quickly.

So that it isn't chaotic "clicking around", sessions with a charter are used: a goal ("explore refunds with partially paid orders"), a fixed time (60–90 min), notes, and a result of bugs, questions and ideas for new cases.

In an interview: Exploratory is when I study the product and design and execute tests at the same time. To keep it controlled, I work in sessions: I set a charter with a specific goal, limit the time and keep notes, and at the end I have bugs, questions for the team and ideas for new test cases.

What are alpha, beta and UAT?

Alpha: internal testing by the team or employees before release outside.
Beta: a limited group of real users in a real environment.
UAT: acceptance testing by the customer or the business: does the product meet business needs. QA often prepares the scenarios and supports the process.

In an interview: Alpha is internal testing before release outside. Beta is real users in a limited group. UAT is acceptance of the product by the business or the customer, and my role there is to prepare scenarios, the environment and data and to help sort out what they find.

How does integration testing differ from end-to-end?

Integration is checking the interaction of two or three components: front end and API, a service and a database, our backend and a payment provider. It can use mocks for the other parts.

End-to-end is the full user path through the whole system: registration → top-up → transfer → statement, with real integrations.

In an interview: Integration checks the joint of several components, for example how our backend works with a payment provider. End-to-end is the whole user path through the system from start to finish. There are more integration tests and they point more precisely at the place of a problem, while e2e checks the most important scenarios as a whole.

Exploratory testing

Exploratory testing is studying the product and designing and executing tests at the same time, where the next step is determined by the result of the previous one. It is not chaotic "clicking around": it has a goal, boundaries and recording of results. It is especially useful for new features, incomplete requirements and finding non-obvious bugs.

Session-based testing structures this process: a session has limited time, usually 60–120 minutes, and a specific charter. A charter is the mission of a session, for example "Explore order checkout with different payment methods to find validation problems". During the session notes are kept: what was checked, what bugs and questions were found.

After a session a debrief is held with the team lead or the team. Heuristics are used for ideas: boundary values, interrupting flows, changing the network, parallel actions, "tours" of the product.

Functional, integration, system testing

Functional testing checks that the system does what the requirements describe, regardless of the internal implementation. For example, a product filter by price returns only products in the given range. It is a type of testing, not a level, and it is applied at all levels.

Integration testing checks the interaction between modules or services. For example, after an order is placed the orders service correctly passes data to the payment service and the notification service. Here we look for problems at the joints: data formats, API contracts, timeouts.

System testing checks the whole system as a whole in an environment as close to production as possible. For example, the full user path from registration to receiving an order confirmation email. This includes both functional and non-functional checks.

Regression vs retest and the scope of regression

A retest is a repeated check of a specific fixed bug by the same steps. Regression is checking that changes haven't broken already working functionality. A retest is always tied to a bug, regression to changes in the code.

I determine the regression scope under limited time through risk analysis. First the areas touched by the changes and the adjacent modules they are integrated with. Then the critical business flows: login, payment, order checkout. Then the areas where bugs have historically been frequent.

A conversation with the developer about what exactly changed and an analysis of the git diff help. Everything stable and repeatable should preferably be covered by automated tests, and manual time spent on new and risky areas.

UAT

User Acceptance Testing is the final level of testing, where the customer or end users check that the system meets their business needs. The question here is not "are there no bugs" but "does the product solve our problem". It is held after system testing in an environment close to production.

It is performed by business users, the PO or customer representatives, not by QA. QA's role is to prepare the environment and test data, help compose UAT scenarios and explain how to use the system. QA also records and traces the problems found and separates defects from change requests.

Varieties: alpha and beta testing, operational acceptance, contract acceptance. The result of UAT is a decision to accept the product, or a sign-off.

Kinds of load testing and JMeter

Load testing checks the system's behaviour under the expected load, for example 1000 simultaneous users at peak hours. The goal is to make sure that response time and the number of errors are within the SLA. Stress testing loads the system beyond normal to find the breaking point and check how it recovers.

Spike testing is a sudden short-term jump in load, as during a sale or a mailing. There are also soak or endurance testing, a long load over hours to look for memory leaks, and scalability testing. Key metrics: response time (average and the percentiles p95, p99), throughput, error rate, CPU and memory usage.

In JMeter the basis is a Test Plan, in which a Thread Group sets the number of virtual users, the ramp-up and the number of iterations. A Sampler is the request itself, for example an HTTP Request. Assertions check the response: the status code, text or time. Listeners collect and show results, for example the Summary Report. Config Elements (CSV Data Set Config, HTTP Header Manager), Timers for pauses and running in non-GUI mode for real tests are also important.