Tools and queries

ELK / OpenSearch and Kibana: KQL

ELK (Elastic Stack) is Elasticsearch (storage and search), Logstash or the lighter Beats/Elastic Agent (collection and processing) and Kibana (the UI). OpenSearch is a fork with an equivalent, Dashboards. Logs are indexed by time (indices or data streams), so in Kibana you always pick a time range and a data view.

The main search language in Kibana Discover is KQL (Kibana Query Language):

log.level: "error" and service.name: "checkout"
log.level: "error" and not message: "healthcheck"
http.response.status_code >= 500
trace.id: "4bf92f3577b34da6a3ce929d0e0e4736"
user.email: *@example.com and event.action: "login"

Rules: field: value, operators and, or, not, parentheses for grouping, comparisons >, >= for numbers and dates, the wildcard *, values with spaces in quotes. Field names depend on the schema (often Elastic Common Schema: log.level, service.name, trace.id).

A typical QA workflow in Kibana:

1. pick a time window around the reproduction of the bug;
2. narrow by service/environment, then by level ERROR;
3. or search directly by the trace.id/request id from the response;
4. add columns (fields) to the table, sort by time, open a document and read the stack trace;
5. save the search or build a visualisation (errors over time) for a report.

Besides KQL there are Lucene syntax and ES|QL (Elastic's piped query language); for an interview it is enough to understand filtering by fields, time and trace id.

Grafana Loki, LogQL, Datadog and Sentry

Grafana Loki is a log system that indexes only the stream's labels (service, environment, pod), not the full text; the rest is filtered at query time. Queries are written in LogQL in Grafana (Explore). A query starts with a stream selector in curly braces, followed by filters:

{app="checkout", env="staging"} |= "timeout"
{app="checkout"} |= "error" != "healthcheck"
{app="checkout"} |~ "order (8841|8842)"
{app="checkout"} | json | level="error" | durationMs > 1000
sum by (app) (count_over_time({env="staging"} |= "error" [5m]))

• |= contains a string, != does not contain, |~ matches a regex, !~ does not match;
• | json (or | logfmt) parses the line and creates labels from fields, which are then filtered (level="error");
• metric queries (count_over_time, rate) turn logs into a series for a graph or an alert; the range goes in square brackets.

Datadog is a SaaS for logs, metrics and APM traces in one place; when configured, logs are linked to traces (the fields dd.trace_id, dd.span_id), so from a trace you jump to its logs in one click.

Sentry is an error tracker: it collects exceptions (frontend and backend), groups them into issues by stack trace, shows breadcrumbs (what happened before the error), the environment and the release version. Useful for a QA: to see whether a new error appeared in this very release, how many users are affected, and whether it regresses after a "fix". For readable frontend stack traces, source maps are uploaded.

The choice of tool depends on the company; what matters is being able to state the general sequence: time → service/environment → level → request identifier → root cause.

The command line: docker logs, kubectl logs, grep, jq

Even when Kibana or Grafana is available, logs of a local environment and of CI are read in a terminal.

Docker:

docker logs -f --tail 100 api            # follow, last 100 lines
docker logs --since 10m api              # for the last 10 minutes
docker compose logs -f --tail=50 api db  # several compose services
docker compose logs --no-color --timestamps > compose.log

Kubernetes:

kubectl logs deploy/checkout --since=15m
kubectl logs -f pod/checkout-7d9f -c app      # a specific container
kubectl logs pod/checkout-7d9f --previous     # the container that crashed and restarted
kubectl logs -l app=checkout --tail=100 --prefix   # by label, with pod names

--previous is a favourite interview question: if a pod is in CrashLoopBackOff, the current container is still empty, and the crash reason is in the logs of the previous one.

Files and text:

tail -f app.log | grep --line-buffered ERROR   # a live stream of errors
grep -n -C 3 "orderId=8841" app.log            # 3 lines of context around a match
grep -c "ERROR" app.log                        # count
zgrep "timeout" app.log.2026-10-11.gz          # in a compressed archive

JSON logs with jq:

jq -c 'select(.level=="error")' app.log
jq -r 'select(.traceId=="4bf92f35...") | [.ts,.service,.msg] | @tsv' app.log
jq -s 'group_by(.error) | map({error: .[0].error, n: length})' app.log

In PowerShell the analogue of grep is Select-String and of tail -f is Get-Content -Wait -Tail 50.