# QA Review Template ## Diff Review - Check whether the diff matches the requested scope. - Check for unrelated source, dependency, schema, generated-code, or UI changes. - Check GraphQL schema and codegen output consistency. - Check Prisma model and database-touching changes carefully. ## Build And Test Review Run: ```sh node ai/runtime/dist/cli.js validation auto-profile --json node ai/runtime/dist/cli.js validation execution-plan --profile runtime-intrinsic --tier chunk --changed-only --json node ai/runtime/dist/cli.js validate --tier full ``` Record pass/fail status for each command. If a command needs local server/database permission, state that explicitly. ## Runtime Smoke Review Decide whether runtime smoke is applicable: - Applicable when the chunk changes behavior, UI, auth, configuration, database access, integration paths, or dev-server behavior. - Not applicable for documentation-only, metadata-only, or other changes that cannot affect runtime behavior. When applicable, run the default runtime smoke command unless the chunk specifies a more precise one: ```sh yarn smoke:runtime ``` Record: - Commands run. - Whether local server/database permission was needed. - Manual or browser checks performed, if any. - Cleanup verification for smoke users, test records, temporary files, and started servers. ## Human-Verifiable Delivery Review Required when a chunk changes product behavior, UI, backend/API behavior, auth, database access, integration paths, Telegram behavior, workflow commands, setup, environment variables, operator-facing docs, or developer commands. Use `ai/standards/human-verifiable-delivery.md`. Record: - `Human-Verifiable Delivery: PASS | BLOCKED | Not applicable` - What changed that a human must observe, configure, access, or verify. - Exact manual/operator verification path, command, URL, role, fixture, seed, reset, or smoke path. - Whether verification is `runtime smoke`, `manual operator path`, `scenario test`, `not applicable`, or `blocked`. - Whether README/setup docs explain the path. - Whether hidden credentials, hidden local state, missing roles, or missing reset/setup steps could prevent verification. - BLOCK if a human cannot reasonably verify the delivered change. ## Environment Configuration Review Required when a chunk introduces, changes, or depends on environment variables, tokens, credentials, bootstrap/reset flows, smoke config, Telegram config, or workflow helper config. Record: - `Environment Configuration: PASS | BLOCKED | Not applicable` - `.env.example` files checked. - Required variables documented with comments and safe placeholders. - Optional variables marked optional. - Setup docs updated. - Local `.env` presence checked only for matching `.env.example`; do not print or copy values. - Confirmation that `.env`, `.tmp`, secrets, local DB files, and local runtime state are not staged. - BLOCK if required configuration cannot be reproduced safely by a human. ## Risk Review - Backend behavior and e2e coverage. - Frontend behavior and generated Apollo service usage. - Frontend architecture boundary when Angular/UI data flow changed: `ai/standards/angular-ui-data-flow.md` was applied; services/facades own data acquisition, services/facades/signals own state coordination, components render typed view models, templates do not orchestrate raw backend/socket or Runtime data, and Socket.IO remains invalidation-only. - Prisma/database assumptions. - Generated file drift. - Archived experiment isolation. ## Acceptance Criteria Verification Review - For stateful/live UI/service/work-registration chunks, record whether QA used the baseline proof strategy: individual failing cases may be cleaned up while diagnosing, but QA PASS requires one uninterrupted final clean run across the required sequence after the fixes. If a different strategy was used, record the research/design checkpoint proving equivalent coverage. - Record root-cause analysis for failures/regressions before accepting symptom patches, retries, fallback polling, or resilience layers. QA should confirm the root cause was identified, improved where feasible, and the fallback has bounded side effects. - Confirm the active chunk has `## Acceptance Criteria Verification`. - Confirm every acceptance criterion is represented. - Confirm every item is marked `Verified`, `Blocked`, or `Not Applicable`. - BLOCK if the section is missing, stale, unmarked, or contains unresolved `Blocked` items. - Record the criterion-by-criterion assessment in the QA Review or summarize that every item was verified/not applicable. ## Test Impact Review Required when a chunk changes behavior, UI, auth, backend/API behavior, database access, integrations, Telegram behavior, workflow tooling, or developer/operator commands. Use `ai/standards/test-strategy.md`. - Confirm the active chunk has `## Test Impact` when applicable. - Confirm test impact describes behavior changed, existing tests affected, new tests required, regression risks, runtime smoke, frontend/browser coverage, backend/API coverage, scenario/workflow coverage, and not-applicable rationale. - BLOCK if behavior changed and test impact is missing, weak, stale, or not verified. - Distinguish missing required tests, accepted follow-up tests, and genuinely not-applicable tests. - Check `ai/commands/workflow-state.sh` output for Test Impact readiness blockers when the chunk changes behavior or workflow tooling. - Distinguish missing tests, accepted follow-up tests, and tests that are genuinely not applicable. - For backend/API chunks, record whether unit tests, e2e/API tests, backend scenario checks, runtime smoke, fixture prefixes, and cleanup were run, not applicable, or deferred with an accepted follow-up. - For backend/Runtime architecture-sensitive chunks, run or review `ai/commands/backend-architecture-report.sh --json` and record advisory findings, accepted exceptions, fixed violations, or carry-forward items. - For visible frontend/UI chunks, apply `ai/standards/ui-review.md` and record UI-review status, browser smoke, screenshots, or accepted gaps. - For Angular/Runtime Action Center UI data-flow chunks, run or review `ai/commands/frontend-architecture-report.sh --json` and record advisory findings, accepted exceptions, or fixed violations. ## Operator Sanity Review Required for workflow/tooling/prompt/Telegram chunks. Use `ai/standards/workflow-output-quality.md`. Record: - `Operator Sanity: PASS | BLOCKED` - Exact output checked. - Issues found. - Whether suggested commands are copy-pasteable. - Whether exact commands are real commands, not prose. - Whether next actions are in the expected final section. - Whether commit messages are concise and sentence-case. - Whether blocked states explain why and what to do next. ## Adversarial False-PASS Review Required for workflow/tooling chunks, requirements/chunk-planning chunks, report-only workflow claims, and high-risk product chunks. Record: - `Adversarial False-PASS: PASS | BLOCKED | Not applicable` - Strongest false PASS risk. - Evidence type for the central claim: `machine-verified`, `simulation-verified`, `runtime-verified`, `manual-review`, or `prose-only`. - Attempted falsification: the counterexample, malformed fixture, stale state, missing section, weak Test Impact, untracked-only change, or other failure mode checked. - Remaining unproven claims. - Whether any unproven claim should block QA. ## Adversarial Sanity Review Required during Chunk Autopilot QA and high-risk workflow/auth/data/integration/operator chunks. Record: - `Adversarial Sanity Review: PASS | BLOCKED | Not applicable` - Practical/operator/product risks considered. - Implementation-path assumptions checked. - User/operator friction checked. - Hidden failure modes checked. - Sanity Finding Classifications: - `blocker` - `retry-safe Developer fix` - `requirements/product decision needed` - `scope-change required` - `follow-up recommendation` - `not applicable / accepted risk` - Evidence type for material findings: `machine-verified`, `simulation-verified`, `runtime-verified`, `manual-review`, or `prose-only`. - Next action for every finding. Do not leave sanity findings as vague prose. - Whether Orchestrator should continue, run a focused Developer retry, or stop for human intervention. ## QA Blocker Classification Required when QA verdict is `BLOCKED`. Record: - `Blocker Classification: fixable | requires_decision | scope_change | retry_limit_reached` - `Retry Safety: retry-safe | unsafe | needs human/requirements clarification` - Evidence type: `machine-verified failure`, `simulation-verified failure`, `runtime-verified failure`, `manual-review concern`, `prose-only uncertainty`, `requirements ambiguity`, or `scope-change request`. - Whether Orchestrator may use `ai/commands/prompt-synthesize.sh dev-fix`. - Stop condition, if retry is unsafe. ## Follow-Up Recommendations List concrete follow-ups only. Separate required fixes from optional improvements. ## Verdict Report one final verdict: - `PASS`: all applicable Definition of Done items and QA gates pass. - `BLOCKED`: one or more required Definition of Done items or QA gates fail. ## Handoff Include the standard handoff block from `ai/standards/workflow-handoff.md`. That standard is canonical for field meanings and for distinguishing gate, review, prompt, transition, and post-approval commands. Keep this example as an output shape, not a separate policy source: ```md ## Handoff - Canonical State: ready_to_complete | qa_blocked | manual_intervention_required - Gate Checked: ai/commands/workflow-state.sh --ready-to-complete - Result: passed | blocked - Blockers: None | - Recommended Next Action: - Immediate Next Step: - Human Review Command: ai/commands/workflow-summary.sh | not_applicable - Prompt Handoff Command: ai/commands/prompt-synthesize.sh dev-fix | not_applicable - Transition Command: ai/tools/operator-daemon/request-action.sh --action complete_chunk --target | not_applicable - Post-Approval Command: ai/tools/operator-daemon/request-action.sh --action complete_chunk --target - Trusted Daemon Git Commands: ai/tools/operator-daemon/request-action.sh --action git_add_approved --files ""; ai/tools/operator-daemon/wait-result.sh ; ai/tools/operator-daemon/request-action.sh --action git_commit --message ""; ai/tools/operator-daemon/wait-result.sh | not_applicable - Optional Prompt Review Command: ai/commands/prompt-synthesize.sh review dev-fix | not_applicable - Human Approval Needed: yes | no ``` ## Chunk QA Review Section When reviewing an active chunk file, append or update: ```md ## QA Review Role: QA qa_evidence_type: qa_independent_review | qa_self_check | qa_operator_override | qa_not_required | qa_blocked qa_executor: runtime-qa-executor | qa_scope: qa_independence: independent | self_check | operator_override | policy_not_required | blocked qa_result: PASS | BLOCKED | PENDING qa_categories: qa_required_evidence: qa_evidence_provided: qa_missing_evidence: qa_conditional_missing_evidence: qa_required_evidence_override: none | requested | accepted qa_override_risk: none | qa_final_clean_run: not required by resolver plan | conditional; provide when applicable | required final clean pass evidence present | missing required final clean pass evidence qa_reviewed_artifacts: qa_validation_evidence: qa_limitations: - QA Required Evidence: `node ai/runtime/dist/cli.js qa required-evidence --chunk --json` - QA Evidence Check: `node ai/runtime/dist/cli.js qa evidence-check --chunk --json` - Verdict: PASS | BLOCKED - Blockers: None | - Acceptance Criteria: - Test Impact: PASS | BLOCKED | Not applicable, with missing tests or accepted follow-ups - Adversarial False-PASS: PASS | BLOCKED | Not applicable, with strongest false PASS risk, evidence type, attempted falsification, and remaining unproven claims - Adversarial Sanity Review: PASS | BLOCKED | Not applicable, with sanity finding classifications and next actions - Blocker Classification: fixable | requires_decision | scope_change | retry_limit_reached | Not applicable - Retry Safety: retry-safe | unsafe | needs human/requirements clarification | Not applicable - Operator Sanity: PASS | BLOCKED | Not applicable, with exact output checked when applicable - Carry Forward: PASS | BLOCKED | Not applicable, with blocking/non-blocking classification and registry ids or rationale - Human-Verifiable Delivery: PASS | BLOCKED | Not applicable, with manual/operator path or rationale - Environment Configuration: PASS | BLOCKED | Not applicable, with `.env.example` and setup-doc assessment - Runtime Smoke: Not applicable | - Validation: - Cleanup: - Recommended Next Action: ``` ## Pass History Entry Also append or update the current QA pass under `## Pass History`: ```md ### QA Pass N - Role: QA - Date: YYYY-MM-DDTHH:mm:ss.sssZ - Goal: - Verdict: PASS | BLOCKED - Blockers: None | - Acceptance Criteria: - Test Impact: PASS | BLOCKED | Not applicable - Adversarial False-PASS: PASS | BLOCKED | Not applicable - Adversarial Sanity Review: PASS | BLOCKED | Not applicable - Sanity Finding Classifications: - Blocker Classification: fixable | requires_decision | scope_change | retry_limit_reached | Not applicable - Retry Safety: retry-safe | unsafe | needs human/requirements clarification | Not applicable - Operator Sanity: PASS | BLOCKED | Not applicable - Human-Verifiable Delivery: PASS | BLOCKED | Not applicable - Environment Configuration: PASS | BLOCKED | Not applicable - Validation: - Cleanup: - Recommended Next Action: ``` Do not overwrite `### Developer Pass N` entries. Preserve pass history as chronological audit history.