Leaderboards / Decision /
decision-conformance
decisionready
Protocol checks for System One decision answers, re-implemented from finnhll/jev-eval (MIT): structural invariants, batched vs. separate questions, repeat stability, option-order sensitivity and monotonicity across ordered states.
core tier · greedy-nothink
same as quick; the probe set is small by design · ranked by checks passed
Conformance is pass/fail per check, so it is listed, not ranked. Failures are named on each run's page.
quick tier · greedy-nothink
every probe once; repeats as listed in probes.json · ranked by checks passed
| Subject | Passed | invariants | batched vs separate | repeat stability | option order | monotonic |
|---|---|---|---|---|---|---|
| jev-latestopaque·typesafe provider-opaqueharness had local changes | 48/49 | 32/33 | 6/6 | 6/6 | 2/2 | 2/2 |
Conformance is pass/fail per check, so it is listed, not ranked. Failures are named on each run's page.