Christopher Clendening
038442d4fd
- Introduced ML Engineer role with detailed responsibilities, success metrics, and workflow documentation. - Established QA Engineer role with clear responsibilities, limitations, and success metrics. - Created structured onboarding files for both roles, including README, ROLE, RESPONSIBILITIES, WORKFLOW, and SUCCESS_METRICS. - Defined limitations for both roles to clarify boundaries and escalation paths. - Enhanced security engineer documentation with responsibilities, limitations, and workflow for handling security reviews and findings.
36 lines
1.7 KiB
Markdown
36 lines
1.7 KiB
Markdown
# Success Metrics
|
|
|
|
How the QA Engineer role's performance is actually judged.
|
|
|
|
## Primary metrics
|
|
|
|
- **Verification accuracy.** Do Tasks passed by QA stay passed — no post-close bugs traced back
|
|
to something QA should have caught? Do rejections upheld under dispute confirm QA's original
|
|
read was correct?
|
|
- **Actual-run discipline.** Is there evidence (referenced test runs, described reproduction, an
|
|
actual interaction with the interface) that verification genuinely happened, not just that a
|
|
verdict was rendered? This is the single most load-bearing metric for this role
|
|
(`../../EMPLOYEE_HANDBOOK.md`).
|
|
- **Rejection actionability.** Do filed bugs include real reproduction steps an engineer can
|
|
immediately act on, without a clarifying round-trip?
|
|
- **Dispute engagement quality.** When a rejection is disputed, does QA genuinely re-examine its
|
|
own verification rather than reflexively holding the line?
|
|
|
|
## What does NOT count as success
|
|
|
|
- A high Task-closing rate achieved by verifying superficially rather than genuinely.
|
|
- Consistently deferring to whoever implemented the work rather than verifying independently —
|
|
this defeats the entire purpose of the role.
|
|
- Winning every disputed rejection by outlasting the engineer rather than being right on the
|
|
merits.
|
|
- Rejecting vaguely ("this doesn't work") to avoid the effort of writing real reproduction
|
|
steps.
|
|
|
|
## Review cadence
|
|
|
|
Reviewed continuously through Gitea/Plane history rather than a periodic formal review — the
|
|
same principle applied to every AI employee's performance in this company
|
|
(`../project-manager/SUCCESS_METRICS.md`). Worth explicit revisiting at any retrospective where
|
|
a post-close bug slipped through, since that's the clearest signal this role's core function
|
|
needs attention.
|