Question 1

Which test metrics do you use and why?

Accepted Answer

Answer by tying every metric to a decision — far stronger than a plain list. "At release time: coverage of critical flows and open defects broken down by severity — these feed the GO/NO-GO decision directly. For the process: escaped defects (how many reached production) — the most honest indicator of the test process; defect density per module — this decides where I put extra regression next release; reopen rate — a signal about bug-report and fix quality." Then state a position: "I avoid metrics like test cases written or bugs found — they reward volume and serve no decision." That last sentence is one very few candidates say and it lands well.

Question 2

A manager wants to measure your work by "bugs found". How do you respond?

Accepted Answer

Don't refuse — offer a replacement. First acknowledge the concern: the manager wants QA's value to be visible, which is legitimate. Then show the problem concretely: "This metric looks worst exactly when I do my best work. If I ask a question in refinement and the defect never gets created, the statistic shows zero — yet that's the most valuable outcome for the team. It also incentivises me to file cosmetic bugs." Then propose alternatives: escaped defects (how many reached production and at what severity), critical-flow coverage, reopen rate. Those reflect QA's value more honestly. Finish with a compromise: "We can keep the bug statistic too — as a trend, not as an individual performance measure." This shows conviction and diplomacy together.

Question 3

What goes into your release report?

Accepted Answer

Give the structure and one line on each section's purpose: (1) the bottom line — GO or NO-GO recommendation, right at the top, because that's the reader's first question; (2) coverage — were critical flows fully executed, what percentage of planned cases; (3) defects — found/fixed/open, split by severity, with workaround status for each open one; (4) what was NOT tested and its risk — the most forgotten and most important section; (5) trend — escaped defects and reopen rate across the last few releases; (6) the decision requested — exactly what is expected from the PO. Key sentence: "A report isn't a list of numbers — it's a story leading to a decision. '142 tests executed' says nothing; 'critical flows fully covered, 1 open High with a workaround' lets someone decide."

Question 4

You found three times fewer bugs this sprint than last. How do you explain that?

Accepted Answer

This is a trap question — going defensive is the mistake. Investigate the causes first, then conclude. Possible explanations: the sprint was smaller; it was a refactoring sprint with little user-facing change; shift-left worked — questions I raised in refinement prevented several defects from existing; or I genuinely tested less (a blocker, an environment problem, time). Then point to the deciding number: escaped defects. "If production defects also fell, quality improved. If they rose, that's a real problem and we need to find out why." Close with your measurement stance: "Bugs found is never a quality indicator on its own — it depends on code quality, sprint content and prevention." This answer demonstrates analytical thinking.

Vanity metric (misleading)	Why it's bad	Replace it with
Number of test cases written	500 weak cases are worse than 50 good ones; volume doesn't measure coverage and creates an incentive to write filler	Requirement/AC coverage: what share of critical flows is covered
Bugs found (per tester)	Cosmetic bugs start getting filed; it discourages early prevention (shift-left) and creates internal competition	Escaped defects: how many defects reached production and at what severity
100% pass rate	Weak tests also produce 100%; "everything is green" carries no information	Pass rate plus which cases ran plus the escaped-defect trend, together
Number of automated tests	What matters is which risk is covered, not the count; flaky tests raise the number and lower trust	Automated coverage of critical flows + suite stability percentage
Hours spent executing tests	It rewards inefficiency — working slowly looks good	Coverage of critical flows and the release's quality outcome

Test metrics & reporting