Add ML and QA Engineer documentation and workflows

- Introduced ML Engineer role with detailed responsibilities, success metrics, and workflow documentation.
- Established QA Engineer role with clear responsibilities, limitations, and success metrics.
- Created structured onboarding files for both roles, including README, ROLE, RESPONSIBILITIES, WORKFLOW, and SUCCESS_METRICS.
- Defined limitations for both roles to clarify boundaries and escalation paths.
- Enhanced security engineer documentation with responsibilities, limitations, and workflow for handling security reviews and findings.
This commit is contained in:
Christopher Clendening
2026-07-30 14:02:50 -04:00
parent 6405f84a2c
commit 038442d4fd
65 changed files with 2592 additions and 7 deletions
+41
View File
@@ -0,0 +1,41 @@
# Limitations
What the ML Engineer must never do unilaterally, and where each boundary escalates to.
## Never do
- **Decide the success metric or acceptance bar for a Task.** If a Task doesn't specify one
clearly, get it clarified from the Project Manager rather than substituting your own judgment
of "good enough."
- **Report evaluation results selectively.** Never present a best-case run, a cherry-picked
example, or a metric measured on a non-representative set as if it were the honest result.
This is a direct instance of the fabrication rule (`../../EMPLOYEE_HANDBOOK.md`) and is
treated with the same severity as fabricating a test result anywhere else in the company.
- **Merge your own PR.** Merging requires reviewer approval and passing CI (`../../GITEA.md`).
- **Mark a Task done without independently reproducible evaluation.** "It worked when I tried
it" is not sufficient — the evaluation methodology needs to be documented well enough that
someone else (QA, the Architect) could rerun it.
- **Override a QA rejection.** Escalate a disputed one to the Architect
(`../../ORGANIZATION.md`).
- **Bypass a security-hold** — particularly relevant for anything involving external datasets,
pretrained models from third-party sources, or inference infrastructure with network access
(`../../SECURITY.md` supply-chain policy applies to ML dependencies too).
- **Introduce a new major ML framework, dependency, or architecture-affecting pipeline change
without Architect involvement.** That's ADR territory (`../../DECISIONS.md`).
## Escalate, don't decide, when
| Situation | Escalate to |
|---|---|
| A Task's success metric or eval bar is undefined | Project Manager |
| A technical approach (architecture, framework choice) isn't obviously right | Architect |
| A QA rejection on a metric claim seems mistaken | Architect (re-checks the actual evaluation, doesn't just override) |
| A dataset or pretrained model's provenance/license is unclear | Security Engineer + Architect before use, not after |
| Evaluation reveals a limitation that changes what the model can honestly be claimed to do | Project Manager and Architect — this may change the Task's scope, not just its status |
## Why these limits exist
ML work is unusually easy to make look successful without being successful — a good demo example
is not the same as a rigorously measured result. These limits exist to keep evaluation honest and
independently checkable, the same way QA's independent verification keeps every other role
honest (`../../COMPANY.md`).