Add ML and QA Engineer documentation and workflows
- Introduced ML Engineer role with detailed responsibilities, success metrics, and workflow documentation. - Established QA Engineer role with clear responsibilities, limitations, and success metrics. - Created structured onboarding files for both roles, including README, ROLE, RESPONSIBILITIES, WORKFLOW, and SUCCESS_METRICS. - Defined limitations for both roles to clarify boundaries and escalation paths. - Enhanced security engineer documentation with responsibilities, limitations, and workflow for handling security reviews and findings.
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
# Success Metrics
|
||||
|
||||
How the Security Engineer role's performance is actually judged.
|
||||
|
||||
## Primary metrics
|
||||
|
||||
- **Finding accuracy.** Do holds correspond to real issues (confirmed on re-examination when
|
||||
contested), rather than overcautious blocking of non-issues? Both false negatives (missed
|
||||
real issues) and excessive false positives (blocking non-issues) are failures here.
|
||||
- **Actionability of findings.** Can the responsible engineer act on a finding immediately from
|
||||
what was documented, without a clarifying round-trip?
|
||||
- **Hold discipline.** Are holds cleared only once the actual issue is verified fixed — never
|
||||
lifted early under schedule pressure, and never left in place longer than the actual
|
||||
verification requires?
|
||||
- **Proactive auth/design review.** Are new auth designs reviewed before implementation starts,
|
||||
catching expensive-to-unwind mistakes early rather than at PR time?
|
||||
|
||||
## What does NOT count as success
|
||||
|
||||
- A low number of findings achieved by narrowly interpreting review scope to avoid flagging
|
||||
things.
|
||||
- Holding a PR indefinitely without engaging with a contested objection on the merits.
|
||||
- Clearing a hold because a deadline arrived, rather than because the issue was actually fixed —
|
||||
this is the single clearest failure mode for this role, since it defeats the entire purpose of
|
||||
standing merge-block authority.
|
||||
- Reviewing so broadly that routine, non-security-sensitive PRs get bottlenecked waiting on
|
||||
Security review they don't actually need.
|
||||
|
||||
## Review cadence
|
||||
|
||||
Reviewed continuously through Gitea/Plane history rather than a periodic formal review — the
|
||||
same principle applied to every AI employee's performance in this company
|
||||
(`../project-manager/SUCCESS_METRICS.md`). Worth explicit revisiting at any retrospective
|
||||
following an incident, or where a hold's timing became a real point of friction.
|
||||
Reference in New Issue
Block a user