# Repository evaluation rubric template

Status: template, no agent outcomes recorded. Freeze a reviewed copy before runs.

A trial is accepted only when all required behavior checks and existing regression
checks pass, and the reviewer finds no blocking issue under the criteria below.

| Criterion | Pass condition | Evidence |
| --- | --- | --- |
| Behavior | All stated acceptance checks pass | Evaluator log |
| Regression | Required pre-existing checks still pass | Evaluator log |
| Scope | Changes are necessary for the requested outcome | Diff references |
| Integrity | Evaluation files and required tests were not disabled or weakened | Diff and file hashes |
| Maintenance | No duplicated policy or unnecessary new dependency | Specific code references |
| Delivery | Required migrations, docs, or public API compatibility are present | Requirement-to-file mapping |

Do not reject a valid alternative merely because it differs from the historical
patch. Define task-specific blocking criteria before seeing outputs.

Possible final status values: accepted, code_failure, timeout,
environment_failure, provider_failure, policy_failure, invalid_task.

Retain every scheduled trial, including excluded infrastructure failures.
Specify the rerun policy before execution and report both end-to-end completion
and code success conditional on valid execution. If a task or grader is invalid,
repair it, version the protocol, and rerun affected configurations equally.