EnigmaForge
▶ Try one instance yourself — the question is hidden in the story
The benchmark where a model must find the problem before it can solve it.
2026-09-24T03:37:15+00:00 · 600 instances · 120 world families · methodology under “Benchmark details”
The leaderboard at a glance
gpt-6-astra leads: task success 80.3/100 (482/600 items), fact F1 90.6/100, 550/600 items answered. Solid baselines: constraint-solver and story-copy bound the scale.
Task success (/100)
Fact retention (/100): balance of right claims made vs hidden facts found
Trust (/100): of every asserted fact, the share that was right
Intuition (/100): task success when handed only the story, no stated question
Earned decisions (/100): correct actions reached on an exactly-right world
All scores are 0–100; higher is better. Deterministic baselines appear in the tables only: a perfect-information solver marks the ceiling, and a text-copier marks the floor at zero.
Open the full performance table (per metric: all-item and scored-only, with confidence intervals)
How to read this benchmark
What a model must do
Every item is a short story that hides a set of definite facts and a rule for what should happen next. The model reads the story and must report the facts it is sure of, plus the action the rule calls for. Everything is exact: a fact is right or wrong, an action is right or wrong.
Finding the problem is scored
The same world is presented three ways: as a plain list of rules, as a story with the question stated, and as a story with no question at all. Discovery retention shows how much performance survives when the model is not told what it is looking for — the benchmark's signature measure.
Three honesty checks
Reasoning discipline flags models that burn their entire thinking budget without producing an answer. Earned decisions counts actions that were correct with a fully correct world, separating insight from lucky guessing. A story-copying control is graded alongside every model and must score zero — if it doesn't, the leaderboard is void.
| Score (0–100) | What it means |
|---|---|
| Task success | Share of items where the model reported every fact exactly right and took the action the rule requires. The headline measure. |
| Fact F1 | Balance of how many stated facts were right vs how many of the hidden facts were found. Forgives a few misses; punishes wrong claims. |
| Discovery retention | Performance without a stated question, relative to with one. 100 means finding the problem cost nothing. |
| Reasoning discipline | Share of items answered without running out of thinking budget first. |
| Earned decisions | Correct actions that came with a fully correct world — the no-luckiness measure. |
Why it is different
| Typical benchmarks | EnigmaForge |
|---|---|
| The question is stated | The question is hidden in the story |
| Hand-written items | Every world is generated and machine-verified |
| One checked answer | Answer plus the rule that justifies it |
| A right-sounding guess scores | Guesses are measured and shown separately |
| Scores can be gamed by copying | Copying the source text scores zero |
Every EnigmaForge item is a short mystery with a machine-checked answer. Here is how one is made, in plain terms.
1 · Write a hidden rulebook
Each world starts as a list of secret facts — which record says what — and clues that connect them. No story exists yet.
2 · Prove there is exactly one answer
A checker proves two things before anything is written down: the clues have exactly one possible set of facts, and every clue is needed — remove any one and other answers become possible. No broken puzzles.
3 · Hide it in a story
Each clue becomes one sentence, woven into a story with extra scenes that mean nothing. A second, differently written story hides the same puzzle, so a model can't memorise the wording.
4 · Ask, three ways
The same puzzle goes out three ways: as a bare list of rules, as a story with the question stated, and as a story with no question at all. The last version is the real test — the model must notice there is something to solve.
5 · Grade exactly, not generously
The model must state each fact as 'this record equals this value' and name the action the rule requires. Paraphrase and partial credit are not awarded: values are checked letter-for-letter against the secret rulebook.
6 · Score it four ways
Solving (did it get everything right?), discovery (what did it keep when the question was hidden?), discipline (did it answer instead of running out of thinking room?), and honesty (were its right answers earned, or lucky?).
A note on honesty: the story-copying control is graded alongside every model. It submits the raw text as its answer and must score zero — a standing proof that the leaderboard cannot be gamed by copying.
How scores work
All scores are 0-100; lower is worse. Headline measure: fact F1. Ranking: all-item task success, then all-item fact F1, then provider name. No weighted composite. Conditional means use scored-item denominators; all-item means include missing, filtered, invalid, and failed attempts as zero achieved success. Unadjudicated outcomes remain unavailable. Decisions and task success use policy-bearing items only. Derived diagnostics: discovery cost is the conditional-F1 drop from a stated task (explicit) to an unstated one (implicit) — the price of finding the problem; reasoning overflow is the share of items where the model exhausted max_tokens without visible output; educated guesses are decisions that were correct while the submitted world was not exactly right. Answered includes invalid content; valid means a schema-compliant scored answer. Confidence intervals resample world families, not realizations.
Recorded solver attempt costs only; not full experimental spend.
claude-sonnet-5: 155/600 attempts were blocked by the provider's content filter before any answer. These count as missing mass in all-item scores; treat this row as a lower bound, not a capability estimate.
claude-fable-5: 206/600 attempts were blocked by the provider's content filter before any answer. These count as missing mass in all-item scores; treat this row as a lower bound, not a capability estimate.
claude-opus-5: 176/600 attempts were blocked by the provider's content filter before any answer. These count as missing mass in all-item scores; treat this row as a lower bound, not a capability estimate.
kimi-k3: 157/600 attempts failed in transport (network/provider errors, no response). These count as missing mass in all-item scores; treat this row as a lower bound, not a capability estimate.
claude-fable-5.1: 585/600 attempts were blocked by the provider's content filter before any answer. These count as missing mass in all-item scores; treat this row as a lower bound, not a capability estimate.
| Provider | Expected | Recorded | Answered | Valid | Missing | Filtered | Transport error | Invalid response | Judge error | Adjudication required | Recorded attempt cost | Known-cost records |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | — | 0 |
| baseline:forward-chain | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | — | 0 |
| gpt-6-astra | 600 | 600 | 550 | 550 | 0 | 0 | 50 | 0 | 0 | 0 | $41.1670 | 550 |
| gpt-6-sol | 600 | 600 | 600 | 598 | 0 | 0 | 0 | 2 | 0 | 0 | $11.6764 | 600 |
| gemini-3.7-flash | 600 | 600 | 600 | 598 | 0 | 0 | 0 | 2 | 0 | 0 | $4.8211 | 600 |
| gpt-5.6-sol | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | $13.2369 | 600 |
| gemini-3.8-flash | 600 | 600 | 598 | 597 | 0 | 0 | 2 | 1 | 0 | 0 | $5.7595 | 600 |
| glm-5.3 | 600 | 600 | 563 | 557 | 0 | 0 | 36 | 7 | 0 | 0 | $60.8358 | 600 |
| gpt-5.6-terra | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | $13.2981 | 600 |
| deepseek-4.1-flash | 600 | 600 | 595 | 595 | 0 | 0 | 5 | 0 | 0 | 0 | $4.2486 | 600 |
| gemini-3.6-flash | 600 | 600 | 600 | 596 | 0 | 0 | 0 | 4 | 0 | 0 | $7.5967 | 600 |
| deepseek-v4-pro | 600 | 600 | 598 | 592 | 0 | 0 | 0 | 8 | 0 | 0 | $17.2358 | 600 |
| gpt-6-luna | 600 | 600 | 600 | 599 | 0 | 0 | 0 | 1 | 0 | 0 | $0.8073 | 600 |
| claude-sonnet-5 | 600 | 600 | 445 | 429 | 0 | 155 | 0 | 16 | 0 | 0 | $19.1364 | 450 |
| gpt-5.6-luna | 600 | 600 | 600 | 599 | 0 | 0 | 0 | 1 | 0 | 0 | $1.6427 | 600 |
| claude-opus-5.5 | 600 | 600 | 591 | 591 | 0 | 9 | 0 | 0 | 0 | 0 | $25.7479 | 600 |
| claude-fable-5 | 600 | 600 | 393 | 392 | 0 | 206 | 1 | 1 | 0 | 0 | $61.8898 | 449 |
| mimo-v2.6-flash | 600 | 600 | 594 | 583 | 0 | 0 | 6 | 11 | 0 | 0 | $0.7466 | 595 |
| mimo-v2.6-pro | 600 | 600 | 593 | 573 | 0 | 0 | 0 | 27 | 0 | 0 | $2.5694 | 600 |
| claude-opus-5 | 600 | 600 | 424 | 389 | 0 | 176 | 0 | 35 | 0 | 0 | $27.3236 | 429 |
| grok-4.6 | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | $20.2564 | 600 |
| kimi-k2.6 | 600 | 600 | 599 | 599 | 0 | 0 | 1 | 0 | 0 | 0 | $26.2999 | 600 |
| minimax-m3 | 600 | 600 | 597 | 585 | 0 | 0 | 0 | 15 | 0 | 0 | $3.3520 | 600 |
| qwen3.8-27b | 600 | 600 | 600 | 585 | 0 | 0 | 0 | 15 | 0 | 0 | $2.6885 | 600 |
| kimi-k3 | 600 | 600 | 443 | 443 | 0 | 0 | 157 | 0 | 0 | 0 | $12.1732 | 443 |
| glm-5.3-flash | 600 | 600 | 596 | 593 | 0 | 0 | 4 | 3 | 0 | 0 | $3.2404 | 600 |
| claude-haiku-4.5 | 600 | 600 | 600 | 591 | 0 | 0 | 0 | 9 | 0 | 0 | $9.0504 | 600 |
| glm-4.7-flash | 600 | 600 | 565 | 543 | 0 | 0 | 34 | 23 | 0 | 0 | $9.8742 | 600 |
| claude-fable-5.1 | 600 | 600 | 14 | 14 | 0 | 585 | 1 | 0 | 0 | 0 | $2.4004 | 14 |
| baseline:direct-facts | 600 | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 0 | 0 | — | 0 |
| baseline:story-copy | 600 | 600 | 600 | 0 | 0 | 0 | 0 | 600 | 0 | 0 | — | 0 |
Levels are configured strata, not calibrated capability ceilings. Charts show scored-only fact F1 per provider; tables add all-item values and intervals (family clusters).
level · fact_f1 (/100)
scenario · fact_f1 (/100)
condition · fact_f1 (/100)
realization · fact_f1 (/100)
genre · fact_f1 (/100)
renderer · fact_f1 (/100)
| Provider | level | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | 0 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:constraint-solver | 1 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:constraint-solver | 2 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:constraint-solver | 3 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:constraint-solver | 4 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:constraint-solver | 5 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:forward-chain | 0 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:forward-chain | 1 | 100 | 100 | 99.3% (n=100) | 99.3% (n=100) | 95.0% (n=100) | 95.0% (n=100) | 95.0% (n=100) | 95.0% (n=100) |
| baseline:forward-chain | 2 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:forward-chain | 3 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| baseline:forward-chain | 4 | 100 | 100 | 99.8% (n=100) | 99.8% (n=100) | 95.0% (n=100) | 95.0% (n=100) | 95.0% (n=100) | 95.0% (n=100) |
| baseline:forward-chain | 5 | 100 | 100 | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) | 100.0% (n=100) |
| gpt-6-astra | 0 | 100 | 100 | 96.0% (n=100) | 96.0% (n=100) | 65.0% (n=100) | 65.0% (n=100) | 65.0% (n=100) | 65.0% (n=100) |
| gpt-6-astra | 1 | 100 | 97 | 96.0% (n=100) | 99.0% (n=97) | 82.0% (n=100) | 84.5% (n=97) | 82.0% (n=100) | 84.5% (n=97) |
| gpt-6-astra | 2 | 100 | 100 | 99.0% (n=100) | 99.0% (n=100) | 93.0% (n=100) | 93.0% (n=100) | 93.0% (n=100) | 93.0% (n=100) |
| gpt-6-astra | 3 | 100 | 98 | 97.7% (n=100) | 99.7% (n=98) | 93.0% (n=100) | 94.9% (n=98) | 93.0% (n=100) | 94.9% (n=98) |
| gpt-6-astra | 4 | 100 | 97 | 96.8% (n=100) | 99.7% (n=97) | 92.0% (n=100) | 94.8% (n=97) | 92.0% (n=100) | 94.8% (n=97) |
| gpt-6-astra | 5 | 100 | 58 | 57.7% (n=100) | 99.5% (n=58) | 57.0% (n=100) | 98.3% (n=58) | 57.0% (n=100) | 98.3% (n=58) |
| gpt-6-sol | 0 | 100 | 100 | 93.5% (n=100) | 93.5% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) |
| gpt-6-sol | 1 | 100 | 100 | 96.8% (n=100) | 96.8% (n=100) | 72.0% (n=100) | 72.0% (n=100) | 72.0% (n=100) | 72.0% (n=100) |
| gpt-6-sol | 2 | 100 | 100 | 96.4% (n=100) | 96.4% (n=100) | 68.0% (n=100) | 68.0% (n=100) | 68.0% (n=100) | 68.0% (n=100) |
| gpt-6-sol | 3 | 100 | 100 | 98.0% (n=100) | 98.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 80.0% (n=100) | 80.0% (n=100) |
| gpt-6-sol | 4 | 100 | 100 | 98.0% (n=100) | 98.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) |
| gpt-6-sol | 5 | 100 | 100 | 97.2% (n=100) | 97.2% (n=100) | 84.0% (n=100) | 84.0% (n=100) | 84.0% (n=100) | 84.0% (n=100) |
| gemini-3.7-flash | 0 | 100 | 100 | 98.2% (n=100) | 98.2% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) |
| gemini-3.7-flash | 1 | 100 | 100 | 97.1% (n=100) | 97.1% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) |
| gemini-3.7-flash | 2 | 100 | 100 | 95.8% (n=100) | 95.8% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) | 81.0% (n=100) |
| gemini-3.7-flash | 3 | 100 | 100 | 90.2% (n=100) | 90.2% (n=100) | 67.0% (n=100) | 67.0% (n=100) | 67.0% (n=100) | 67.0% (n=100) |
| gemini-3.7-flash | 4 | 100 | 100 | 83.3% (n=100) | 83.3% (n=100) | 53.0% (n=100) | 53.0% (n=100) | 53.0% (n=100) | 53.0% (n=100) |
| gemini-3.7-flash | 5 | 100 | 100 | 83.3% (n=100) | 83.3% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) |
| gpt-5.6-sol | 0 | 100 | 100 | 94.6% (n=100) | 94.6% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) |
| gpt-5.6-sol | 1 | 100 | 100 | 96.9% (n=100) | 96.9% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) |
| gpt-5.6-sol | 2 | 100 | 100 | 91.9% (n=100) | 91.9% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) |
| gpt-5.6-sol | 3 | 100 | 100 | 94.8% (n=100) | 94.8% (n=100) | 78.0% (n=100) | 78.0% (n=100) | 78.0% (n=100) | 78.0% (n=100) |
| gpt-5.6-sol | 4 | 100 | 100 | 85.4% (n=100) | 85.4% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) |
| gpt-5.6-sol | 5 | 100 | 100 | 82.5% (n=100) | 82.5% (n=100) | 67.0% (n=100) | 67.0% (n=100) | 67.0% (n=100) | 67.0% (n=100) |
| gemini-3.8-flash | 0 | 100 | 100 | 96.7% (n=100) | 96.7% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) | 71.0% (n=100) |
| gemini-3.8-flash | 1 | 100 | 100 | 96.4% (n=100) | 96.4% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) | 70.0% (n=100) |
| gemini-3.8-flash | 2 | 100 | 99 | 90.2% (n=100) | 91.1% (n=99) | 63.0% (n=100) | 63.6% (n=99) | 63.0% (n=100) | 63.6% (n=99) |
| gemini-3.8-flash | 3 | 100 | 100 | 84.3% (n=100) | 84.3% (n=100) | 59.0% (n=100) | 59.0% (n=100) | 59.0% (n=100) | 59.0% (n=100) |
| gemini-3.8-flash | 4 | 100 | 99 | 76.6% (n=100) | 77.4% (n=99) | 50.0% (n=100) | 50.5% (n=99) | 50.0% (n=100) | 50.5% (n=99) |
| gemini-3.8-flash | 5 | 100 | 100 | 73.2% (n=100) | 73.2% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) |
| glm-5.3 | 0 | 100 | 98 | 89.6% (n=100) | 91.4% (n=98) | 57.0% (n=100) | 58.2% (n=98) | 57.0% (n=100) | 58.2% (n=98) |
| glm-5.3 | 1 | 100 | 95 | 90.5% (n=100) | 95.3% (n=95) | 66.0% (n=100) | 69.5% (n=95) | 66.0% (n=100) | 69.5% (n=95) |
| glm-5.3 | 2 | 100 | 93 | 90.1% (n=100) | 96.8% (n=93) | 76.0% (n=100) | 81.7% (n=93) | 76.0% (n=100) | 81.7% (n=93) |
| glm-5.3 | 3 | 100 | 94 | 77.4% (n=100) | 82.3% (n=94) | 47.0% (n=100) | 50.0% (n=94) | 46.0% (n=100) | 48.9% (n=94) |
| glm-5.3 | 4 | 100 | 93 | 69.4% (n=100) | 73.9% (n=94) | 38.0% (n=100) | 40.4% (n=94) | 38.0% (n=100) | 40.4% (n=94) |
| glm-5.3 | 5 | 100 | 90 | 63.6% (n=100) | 70.6% (n=90) | 36.0% (n=100) | 40.0% (n=90) | 36.0% (n=100) | 40.0% (n=90) |
| gpt-5.6-terra | 0 | 100 | 100 | 90.0% (n=100) | 90.0% (n=100) | 51.0% (n=100) | 51.0% (n=100) | 51.0% (n=100) | 51.0% (n=100) |
| gpt-5.6-terra | 1 | 100 | 100 | 86.1% (n=100) | 86.1% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) |
| gpt-5.6-terra | 2 | 100 | 100 | 84.7% (n=100) | 84.7% (n=100) | 59.0% (n=100) | 59.0% (n=100) | 59.0% (n=100) | 59.0% (n=100) |
| gpt-5.6-terra | 3 | 100 | 100 | 82.1% (n=100) | 82.1% (n=100) | 49.0% (n=100) | 49.0% (n=100) | 49.0% (n=100) | 49.0% (n=100) |
| gpt-5.6-terra | 4 | 100 | 100 | 76.5% (n=100) | 76.5% (n=100) | 47.0% (n=100) | 47.0% (n=100) | 47.0% (n=100) | 47.0% (n=100) |
| gpt-5.6-terra | 5 | 100 | 100 | 71.1% (n=100) | 71.1% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) |
| deepseek-4.1-flash | 0 | 100 | 100 | 91.8% (n=100) | 91.8% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) | 60.0% (n=100) |
| deepseek-4.1-flash | 1 | 100 | 100 | 87.3% (n=100) | 87.3% (n=100) | 57.0% (n=100) | 57.0% (n=100) | 57.0% (n=100) | 57.0% (n=100) |
| deepseek-4.1-flash | 2 | 100 | 100 | 83.3% (n=100) | 83.3% (n=100) | 55.0% (n=100) | 55.0% (n=100) | 55.0% (n=100) | 55.0% (n=100) |
| deepseek-4.1-flash | 3 | 100 | 100 | 77.1% (n=100) | 77.1% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) |
| deepseek-4.1-flash | 4 | 100 | 96 | 66.1% (n=100) | 68.9% (n=96) | 39.0% (n=100) | 40.6% (n=96) | 39.0% (n=100) | 40.6% (n=96) |
| deepseek-4.1-flash | 5 | 100 | 99 | 66.6% (n=100) | 67.2% (n=99) | 35.0% (n=100) | 35.4% (n=99) | 35.0% (n=100) | 35.4% (n=99) |
| gemini-3.6-flash | 0 | 100 | 100 | 95.4% (n=100) | 95.4% (n=100) | 68.0% (n=100) | 68.0% (n=100) | 68.0% (n=100) | 68.0% (n=100) |
| gemini-3.6-flash | 1 | 100 | 100 | 90.3% (n=100) | 90.3% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) | 54.0% (n=100) |
| gemini-3.6-flash | 2 | 100 | 100 | 86.4% (n=100) | 86.4% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) |
| gemini-3.6-flash | 3 | 100 | 100 | 75.4% (n=100) | 75.4% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) |
| gemini-3.6-flash | 4 | 100 | 100 | 68.9% (n=100) | 68.9% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) |
| gemini-3.6-flash | 5 | 100 | 100 | 60.6% (n=100) | 60.6% (n=100) | 26.0% (n=100) | 26.0% (n=100) | 26.0% (n=100) | 26.0% (n=100) |
| deepseek-v4-pro | 0 | 100 | 100 | 91.6% (n=100) | 91.6% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| deepseek-v4-pro | 1 | 100 | 99 | 92.8% (n=100) | 92.8% (n=100) | 52.0% (n=100) | 52.0% (n=100) | 52.0% (n=100) | 52.0% (n=100) |
| deepseek-v4-pro | 2 | 100 | 100 | 93.2% (n=100) | 93.2% (n=100) | 47.0% (n=100) | 47.0% (n=100) | 47.0% (n=100) | 47.0% (n=100) |
| deepseek-v4-pro | 3 | 100 | 100 | 90.8% (n=100) | 90.8% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) |
| deepseek-v4-pro | 4 | 100 | 100 | 92.6% (n=100) | 92.6% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| deepseek-v4-pro | 5 | 100 | 99 | 88.0% (n=100) | 88.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) |
| gpt-6-luna | 0 | 100 | 100 | 89.8% (n=100) | 89.8% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) |
| gpt-6-luna | 1 | 100 | 100 | 86.7% (n=100) | 86.7% (n=100) | 50.0% (n=100) | 50.0% (n=100) | 50.0% (n=100) | 50.0% (n=100) |
| gpt-6-luna | 2 | 100 | 100 | 80.5% (n=100) | 80.5% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) |
| gpt-6-luna | 3 | 100 | 100 | 85.2% (n=100) | 85.2% (n=100) | 43.0% (n=100) | 43.0% (n=100) | 43.0% (n=100) | 43.0% (n=100) |
| gpt-6-luna | 4 | 100 | 100 | 73.9% (n=100) | 73.9% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| gpt-6-luna | 5 | 100 | 100 | 72.6% (n=100) | 72.6% (n=100) | 37.0% (n=100) | 37.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| claude-sonnet-5 | 0 | 100 | 88 | 81.2% (n=100) | 92.2% (n=88) | 55.0% (n=100) | 62.5% (n=88) | 55.0% (n=100) | 62.5% (n=88) |
| claude-sonnet-5 | 1 | 100 | 72 | 68.9% (n=100) | 95.7% (n=72) | 46.0% (n=100) | 63.9% (n=72) | 46.0% (n=100) | 63.9% (n=72) |
| claude-sonnet-5 | 2 | 100 | 73 | 68.6% (n=100) | 94.0% (n=73) | 45.0% (n=100) | 61.6% (n=73) | 45.0% (n=100) | 61.6% (n=73) |
| claude-sonnet-5 | 3 | 100 | 72 | 64.3% (n=100) | 89.3% (n=72) | 38.0% (n=100) | 52.8% (n=72) | 38.0% (n=100) | 52.8% (n=72) |
| claude-sonnet-5 | 4 | 100 | 61 | 52.0% (n=100) | 85.2% (n=61) | 33.0% (n=100) | 54.1% (n=61) | 33.0% (n=100) | 54.1% (n=61) |
| claude-sonnet-5 | 5 | 100 | 79 | 66.6% (n=100) | 84.3% (n=79) | 34.0% (n=100) | 43.0% (n=79) | 34.0% (n=100) | 43.0% (n=79) |
| gpt-5.6-luna | 0 | 100 | 100 | 88.6% (n=100) | 88.6% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) |
| gpt-5.6-luna | 1 | 100 | 100 | 83.3% (n=100) | 83.3% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) |
| gpt-5.6-luna | 2 | 100 | 100 | 73.8% (n=100) | 73.8% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) |
| gpt-5.6-luna | 3 | 100 | 100 | 85.7% (n=100) | 85.7% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 40.0% (n=100) | 40.0% (n=100) |
| gpt-5.6-luna | 4 | 100 | 100 | 80.6% (n=100) | 80.6% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| gpt-5.6-luna | 5 | 100 | 100 | 76.2% (n=100) | 76.2% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| claude-opus-5.5 | 0 | 100 | 100 | 93.2% (n=100) | 93.2% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) |
| claude-opus-5.5 | 1 | 100 | 99 | 91.9% (n=100) | 92.8% (n=99) | 34.0% (n=100) | 34.3% (n=99) | 33.0% (n=100) | 33.3% (n=99) |
| claude-opus-5.5 | 2 | 100 | 97 | 90.1% (n=100) | 92.9% (n=97) | 40.0% (n=100) | 41.2% (n=97) | 40.0% (n=100) | 41.2% (n=97) |
| claude-opus-5.5 | 3 | 100 | 100 | 94.3% (n=100) | 94.3% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) |
| claude-opus-5.5 | 4 | 100 | 98 | 91.4% (n=100) | 93.2% (n=98) | 41.0% (n=100) | 41.8% (n=98) | 41.0% (n=100) | 41.8% (n=98) |
| claude-opus-5.5 | 5 | 100 | 97 | 90.4% (n=100) | 93.2% (n=97) | 45.0% (n=100) | 46.4% (n=97) | 45.0% (n=100) | 46.4% (n=97) |
| claude-fable-5 | 0 | 100 | 75 | 72.3% (n=100) | 96.4% (n=75) | 48.0% (n=100) | 64.0% (n=75) | 48.0% (n=100) | 64.0% (n=75) |
| claude-fable-5 | 1 | 100 | 68 | 65.1% (n=100) | 95.8% (n=68) | 47.0% (n=100) | 69.1% (n=68) | 47.0% (n=100) | 69.1% (n=68) |
| claude-fable-5 | 2 | 100 | 59 | 54.7% (n=100) | 92.7% (n=59) | 28.0% (n=100) | 47.5% (n=59) | 28.0% (n=100) | 47.5% (n=59) |
| claude-fable-5 | 3 | 100 | 65 | 61.7% (n=100) | 95.0% (n=65) | 38.0% (n=100) | 58.5% (n=65) | 38.0% (n=100) | 58.5% (n=65) |
| claude-fable-5 | 4 | 100 | 66 | 63.3% (n=100) | 95.8% (n=66) | 36.0% (n=100) | 54.5% (n=66) | 36.0% (n=100) | 54.5% (n=66) |
| claude-fable-5 | 5 | 100 | 60 | 58.1% (n=100) | 96.9% (n=60) | 40.0% (n=100) | 66.7% (n=60) | 40.0% (n=100) | 66.7% (n=60) |
| mimo-v2.6-flash | 0 | 100 | 95 | 85.7% (n=100) | 90.2% (n=95) | 50.0% (n=100) | 52.6% (n=95) | 50.0% (n=100) | 52.6% (n=95) |
| mimo-v2.6-flash | 1 | 100 | 100 | 86.7% (n=100) | 86.7% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| mimo-v2.6-flash | 2 | 100 | 100 | 86.2% (n=100) | 86.2% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) |
| mimo-v2.6-flash | 3 | 100 | 100 | 74.1% (n=100) | 74.1% (n=100) | 37.0% (n=100) | 37.0% (n=100) | 34.0% (n=100) | 34.0% (n=100) |
| mimo-v2.6-flash | 4 | 100 | 99 | 74.5% (n=100) | 75.2% (n=99) | 36.0% (n=100) | 36.4% (n=99) | 31.0% (n=100) | 31.3% (n=99) |
| mimo-v2.6-flash | 5 | 100 | 100 | 64.0% (n=100) | 64.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 30.0% (n=100) | 30.0% (n=100) |
| mimo-v2.6-pro | 0 | 100 | 99 | 89.2% (n=100) | 89.2% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) |
| mimo-v2.6-pro | 1 | 100 | 99 | 85.6% (n=100) | 85.6% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) | 46.0% (n=100) |
| mimo-v2.6-pro | 2 | 100 | 99 | 81.5% (n=100) | 81.5% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 43.0% (n=100) | 43.0% (n=100) |
| mimo-v2.6-pro | 3 | 100 | 100 | 73.9% (n=100) | 73.9% (n=100) | 38.0% (n=100) | 38.0% (n=100) | 37.0% (n=100) | 37.0% (n=100) |
| mimo-v2.6-pro | 4 | 100 | 99 | 68.4% (n=100) | 68.4% (n=100) | 34.0% (n=100) | 34.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) |
| mimo-v2.6-pro | 5 | 100 | 97 | 61.0% (n=100) | 61.0% (n=100) | 27.0% (n=100) | 27.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) |
| claude-opus-5 | 0 | 100 | 75 | 68.7% (n=100) | 91.5% (n=75) | 40.0% (n=100) | 53.3% (n=75) | 40.0% (n=100) | 53.3% (n=75) |
| claude-opus-5 | 1 | 100 | 70 | 59.2% (n=100) | 84.6% (n=70) | 35.0% (n=100) | 50.0% (n=70) | 35.0% (n=100) | 50.0% (n=70) |
| claude-opus-5 | 2 | 100 | 71 | 57.0% (n=100) | 80.3% (n=71) | 40.0% (n=100) | 56.3% (n=71) | 40.0% (n=100) | 56.3% (n=71) |
| claude-opus-5 | 3 | 100 | 63 | 50.3% (n=100) | 79.9% (n=63) | 33.0% (n=100) | 52.4% (n=63) | 33.0% (n=100) | 52.4% (n=63) |
| claude-opus-5 | 4 | 100 | 73 | 52.2% (n=100) | 71.5% (n=73) | 36.0% (n=100) | 49.3% (n=73) | 36.0% (n=100) | 49.3% (n=73) |
| claude-opus-5 | 5 | 100 | 72 | 56.4% (n=100) | 78.3% (n=72) | 39.0% (n=100) | 54.2% (n=72) | 39.0% (n=100) | 54.2% (n=72) |
| grok-4.6 | 0 | 100 | 100 | 90.1% (n=100) | 90.1% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| grok-4.6 | 1 | 100 | 100 | 90.6% (n=100) | 90.6% (n=100) | 40.0% (n=100) | 40.0% (n=100) | 40.0% (n=100) | 40.0% (n=100) |
| grok-4.6 | 2 | 100 | 100 | 89.4% (n=100) | 89.4% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) |
| grok-4.6 | 3 | 100 | 100 | 90.1% (n=100) | 90.1% (n=100) | 37.0% (n=100) | 37.0% (n=100) | 37.0% (n=100) | 37.0% (n=100) |
| grok-4.6 | 4 | 100 | 100 | 89.6% (n=100) | 89.6% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) |
| grok-4.6 | 5 | 100 | 100 | 87.2% (n=100) | 87.2% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) | 44.0% (n=100) |
| kimi-k2.6 | 0 | 100 | 100 | 93.5% (n=100) | 93.5% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) | 48.0% (n=100) |
| kimi-k2.6 | 1 | 100 | 100 | 93.3% (n=100) | 93.3% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) | 39.0% (n=100) |
| kimi-k2.6 | 2 | 100 | 100 | 91.7% (n=100) | 91.7% (n=100) | 34.0% (n=100) | 34.0% (n=100) | 34.0% (n=100) | 34.0% (n=100) |
| kimi-k2.6 | 3 | 100 | 100 | 88.8% (n=100) | 88.8% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| kimi-k2.6 | 4 | 100 | 100 | 88.1% (n=100) | 88.1% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) | 33.0% (n=100) |
| kimi-k2.6 | 5 | 100 | 99 | 86.9% (n=100) | 87.8% (n=99) | 29.0% (n=100) | 29.3% (n=99) | 29.0% (n=100) | 29.3% (n=99) |
| minimax-m3 | 0 | 100 | 100 | 87.6% (n=100) | 87.6% (n=100) | 43.0% (n=100) | 43.0% (n=100) | 43.0% (n=100) | 43.0% (n=100) |
| minimax-m3 | 1 | 100 | 98 | 76.9% (n=100) | 76.9% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) | 41.0% (n=100) |
| minimax-m3 | 2 | 100 | 100 | 68.3% (n=100) | 68.3% (n=100) | 34.0% (n=100) | 34.0% (n=100) | 34.0% (n=100) | 34.0% (n=100) |
| minimax-m3 | 3 | 100 | 100 | 65.2% (n=100) | 65.2% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) | 32.0% (n=100) |
| minimax-m3 | 4 | 100 | 99 | 60.3% (n=100) | 60.3% (n=100) | 28.0% (n=100) | 28.0% (n=100) | 28.0% (n=100) | 28.0% (n=100) |
| minimax-m3 | 5 | 100 | 100 | 52.5% (n=100) | 52.5% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) |
| qwen3.8-27b | 0 | 100 | 100 | 92.4% (n=100) | 92.4% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) | 45.0% (n=100) |
| qwen3.8-27b | 1 | 100 | 100 | 87.4% (n=100) | 87.4% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| qwen3.8-27b | 2 | 100 | 100 | 87.6% (n=100) | 87.6% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) | 42.0% (n=100) |
| qwen3.8-27b | 3 | 100 | 100 | 86.1% (n=100) | 86.1% (n=100) | 28.0% (n=100) | 28.0% (n=100) | 28.0% (n=100) | 28.0% (n=100) |
| qwen3.8-27b | 4 | 100 | 100 | 75.5% (n=100) | 75.5% (n=100) | 22.0% (n=100) | 22.0% (n=100) | 22.0% (n=100) | 22.0% (n=100) |
| qwen3.8-27b | 5 | 100 | 100 | 72.9% (n=100) | 72.9% (n=100) | 29.0% (n=100) | 29.0% (n=100) | 29.0% (n=100) | 29.0% (n=100) |
| kimi-k3 | 0 | 100 | 96 | 90.6% (n=100) | 94.4% (n=96) | 52.0% (n=100) | 54.2% (n=96) | 52.0% (n=100) | 54.2% (n=96) |
| kimi-k3 | 1 | 100 | 84 | 77.7% (n=100) | 92.5% (n=84) | 40.0% (n=100) | 47.6% (n=84) | 40.0% (n=100) | 47.6% (n=84) |
| kimi-k3 | 2 | 100 | 99 | 88.9% (n=100) | 89.8% (n=99) | 42.0% (n=100) | 42.4% (n=99) | 42.0% (n=100) | 42.4% (n=99) |
| kimi-k3 | 3 | 100 | 100 | 76.6% (n=100) | 76.6% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) | 36.0% (n=100) |
| kimi-k3 | 4 | 100 | 63 | 42.7% (n=100) | 67.8% (n=63) | 17.0% (n=100) | 27.0% (n=63) | 17.0% (n=100) | 27.0% (n=63) |
| kimi-k3 | 5 | 100 | 1 | 0.1% (n=100) | 13.3% (n=1) | 0.0% (n=100) | 0.0% (n=1) | 0.0% (n=100) | 0.0% (n=1) |
| glm-5.3-flash | 0 | 100 | 99 | 86.6% (n=100) | 87.5% (n=99) | 35.0% (n=100) | 35.4% (n=99) | 35.0% (n=100) | 35.4% (n=99) |
| glm-5.3-flash | 1 | 100 | 98 | 79.8% (n=100) | 81.5% (n=98) | 33.0% (n=100) | 33.7% (n=98) | 33.0% (n=100) | 33.7% (n=98) |
| glm-5.3-flash | 2 | 100 | 99 | 73.5% (n=100) | 74.3% (n=99) | 29.0% (n=100) | 29.3% (n=99) | 29.0% (n=100) | 29.3% (n=99) |
| glm-5.3-flash | 3 | 100 | 100 | 61.3% (n=100) | 61.3% (n=100) | 21.0% (n=100) | 21.0% (n=100) | 21.0% (n=100) | 21.0% (n=100) |
| glm-5.3-flash | 4 | 100 | 100 | 54.1% (n=100) | 54.1% (n=100) | 19.0% (n=100) | 19.0% (n=100) | 19.0% (n=100) | 19.0% (n=100) |
| glm-5.3-flash | 5 | 100 | 100 | 44.7% (n=100) | 44.7% (n=100) | 19.0% (n=100) | 19.0% (n=100) | 18.0% (n=100) | 18.0% (n=100) |
| claude-haiku-4.5 | 0 | 100 | 100 | 83.4% (n=100) | 83.4% (n=100) | 31.0% (n=100) | 31.0% (n=100) | 31.0% (n=100) | 31.0% (n=100) |
| claude-haiku-4.5 | 1 | 100 | 100 | 78.8% (n=100) | 78.8% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) |
| claude-haiku-4.5 | 2 | 100 | 100 | 71.5% (n=100) | 71.5% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) | 25.0% (n=100) |
| claude-haiku-4.5 | 3 | 100 | 100 | 60.2% (n=100) | 60.2% (n=100) | 20.0% (n=100) | 20.0% (n=100) | 20.0% (n=100) | 20.0% (n=100) |
| claude-haiku-4.5 | 4 | 100 | 100 | 57.3% (n=100) | 57.3% (n=100) | 19.0% (n=100) | 19.0% (n=100) | 19.0% (n=100) | 19.0% (n=100) |
| claude-haiku-4.5 | 5 | 100 | 100 | 57.1% (n=100) | 57.1% (n=100) | 22.0% (n=100) | 22.0% (n=100) | 22.0% (n=100) | 22.0% (n=100) |
| glm-4.7-flash | 0 | 100 | 99 | 68.4% (n=100) | 69.0% (n=99) | 28.0% (n=100) | 28.3% (n=99) | 28.0% (n=100) | 28.3% (n=99) |
| glm-4.7-flash | 1 | 100 | 100 | 76.0% (n=100) | 76.0% (n=100) | 27.0% (n=100) | 27.0% (n=100) | 27.0% (n=100) | 27.0% (n=100) |
| glm-4.7-flash | 2 | 100 | 98 | 70.7% (n=100) | 72.1% (n=98) | 19.0% (n=100) | 19.4% (n=98) | 19.0% (n=100) | 19.4% (n=98) |
| glm-4.7-flash | 3 | 100 | 85 | 53.0% (n=100) | 61.6% (n=86) | 17.0% (n=100) | 19.8% (n=86) | 17.0% (n=100) | 19.8% (n=86) |
| glm-4.7-flash | 4 | 100 | 87 | 55.8% (n=100) | 64.1% (n=87) | 11.0% (n=100) | 12.6% (n=87) | 11.0% (n=100) | 12.6% (n=87) |
| glm-4.7-flash | 5 | 100 | 96 | 56.0% (n=100) | 58.3% (n=96) | 15.0% (n=100) | 15.6% (n=96) | 15.0% (n=100) | 15.6% (n=96) |
| claude-fable-5.1 | 0 | 100 | 1 | 1.0% (n=100) | 100.0% (n=1) | 1.0% (n=100) | 100.0% (n=1) | 1.0% (n=100) | 100.0% (n=1) |
| claude-fable-5.1 | 1 | 100 | 0 | 0.0% (n=100) | — (n=0) | 0.0% (n=100) | — (n=0) | 0.0% (n=100) | — (n=0) |
| claude-fable-5.1 | 2 | 100 | 0 | 0.0% (n=100) | — (n=0) | 0.0% (n=100) | — (n=0) | 0.0% (n=100) | — (n=0) |
| claude-fable-5.1 | 3 | 100 | 1 | 1.0% (n=100) | 100.0% (n=1) | 1.0% (n=100) | 100.0% (n=1) | 1.0% (n=100) | 100.0% (n=1) |
| claude-fable-5.1 | 4 | 100 | 5 | 4.9% (n=100) | 97.8% (n=5) | 3.0% (n=100) | 60.0% (n=5) | 3.0% (n=100) | 60.0% (n=5) |
| claude-fable-5.1 | 5 | 100 | 7 | 6.9% (n=100) | 98.1% (n=7) | 5.0% (n=100) | 71.4% (n=7) | 5.0% (n=100) | 71.4% (n=7) |
| baseline:direct-facts | 0 | 100 | 100 | 80.6% (n=100) | 80.6% (n=100) | 5.0% (n=100) | 5.0% (n=100) | 5.0% (n=100) | 5.0% (n=100) |
| baseline:direct-facts | 1 | 100 | 100 | 73.0% (n=100) | 73.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:direct-facts | 2 | 100 | 100 | 77.0% (n=100) | 77.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:direct-facts | 3 | 100 | 100 | 71.1% (n=100) | 71.1% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:direct-facts | 4 | 100 | 100 | 71.7% (n=100) | 71.7% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:direct-facts | 5 | 100 | 100 | 68.8% (n=100) | 68.8% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 0 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 1 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 2 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 3 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 4 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| baseline:story-copy | 5 | 100 | 100 | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) | 0.0% (n=100) |
| Provider | scenario | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | strata | 600 | 600 | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) |
| baseline:forward-chain | strata | 600 | 600 | 99.8% (n=600) | 99.8% (n=600) | 98.3% (n=600) | 98.3% (n=600) | 98.3% (n=600) | 98.3% (n=600) |
| gpt-6-astra | strata | 600 | 550 | 90.6% (n=600) | 98.8% (n=550) | 80.3% (n=600) | 87.6% (n=550) | 80.3% (n=600) | 87.6% (n=550) |
| gpt-6-sol | strata | 600 | 600 | 96.6% (n=600) | 96.6% (n=600) | 74.3% (n=600) | 74.3% (n=600) | 74.2% (n=600) | 74.2% (n=600) |
| gemini-3.7-flash | strata | 600 | 600 | 91.3% (n=600) | 91.3% (n=600) | 68.7% (n=600) | 68.7% (n=600) | 68.7% (n=600) | 68.7% (n=600) |
| gpt-5.6-sol | strata | 600 | 600 | 91.0% (n=600) | 91.0% (n=600) | 68.5% (n=600) | 68.5% (n=600) | 68.5% (n=600) | 68.5% (n=600) |
| gemini-3.8-flash | strata | 600 | 598 | 86.2% (n=600) | 86.5% (n=598) | 59.0% (n=600) | 59.2% (n=598) | 59.0% (n=600) | 59.2% (n=598) |
| glm-5.3 | strata | 600 | 563 | 80.1% (n=600) | 85.2% (n=564) | 53.3% (n=600) | 56.7% (n=564) | 53.2% (n=600) | 56.6% (n=564) |
| gpt-5.6-terra | strata | 600 | 600 | 81.8% (n=600) | 81.8% (n=600) | 50.7% (n=600) | 50.7% (n=600) | 50.7% (n=600) | 50.7% (n=600) |
| deepseek-4.1-flash | strata | 600 | 595 | 78.7% (n=600) | 79.4% (n=595) | 48.7% (n=600) | 49.1% (n=595) | 48.7% (n=600) | 49.1% (n=595) |
| gemini-3.6-flash | strata | 600 | 600 | 79.5% (n=600) | 79.5% (n=600) | 43.7% (n=600) | 43.7% (n=600) | 43.7% (n=600) | 43.7% (n=600) |
| deepseek-v4-pro | strata | 600 | 598 | 91.5% (n=600) | 91.5% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 42.7% (n=600) |
| gpt-6-luna | strata | 600 | 600 | 81.5% (n=600) | 81.5% (n=600) | 42.5% (n=600) | 42.5% (n=600) | 42.3% (n=600) | 42.3% (n=600) |
| claude-sonnet-5 | strata | 600 | 445 | 66.9% (n=600) | 90.2% (n=445) | 41.8% (n=600) | 56.4% (n=445) | 41.8% (n=600) | 56.4% (n=445) |
| gpt-5.6-luna | strata | 600 | 600 | 81.4% (n=600) | 81.4% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 41.7% (n=600) | 41.7% (n=600) |
| claude-opus-5.5 | strata | 600 | 591 | 91.9% (n=600) | 93.3% (n=591) | 40.0% (n=600) | 40.6% (n=591) | 39.8% (n=600) | 40.4% (n=591) |
| claude-fable-5 | strata | 600 | 393 | 62.5% (n=600) | 95.5% (n=393) | 39.5% (n=600) | 60.3% (n=393) | 39.5% (n=600) | 60.3% (n=393) |
| mimo-v2.6-flash | strata | 600 | 594 | 78.5% (n=600) | 79.3% (n=594) | 40.5% (n=600) | 40.9% (n=594) | 38.8% (n=600) | 39.2% (n=594) |
| mimo-v2.6-pro | strata | 600 | 593 | 76.6% (n=600) | 76.6% (n=600) | 39.2% (n=600) | 39.2% (n=600) | 38.0% (n=600) | 38.0% (n=600) |
| claude-opus-5 | strata | 600 | 424 | 57.3% (n=600) | 81.1% (n=424) | 37.2% (n=600) | 52.6% (n=424) | 37.2% (n=600) | 52.6% (n=424) |
| grok-4.6 | strata | 600 | 600 | 89.5% (n=600) | 89.5% (n=600) | 37.0% (n=600) | 37.0% (n=600) | 37.0% (n=600) | 37.0% (n=600) |
| kimi-k2.6 | strata | 600 | 599 | 90.4% (n=600) | 90.6% (n=599) | 36.5% (n=600) | 36.6% (n=599) | 36.5% (n=600) | 36.6% (n=599) |
| minimax-m3 | strata | 600 | 597 | 68.5% (n=600) | 68.5% (n=600) | 33.8% (n=600) | 33.8% (n=600) | 33.8% (n=600) | 33.8% (n=600) |
| qwen3.8-27b | strata | 600 | 600 | 83.6% (n=600) | 83.6% (n=600) | 33.7% (n=600) | 33.7% (n=600) | 33.7% (n=600) | 33.7% (n=600) |
| kimi-k3 | strata | 600 | 443 | 62.8% (n=600) | 85.0% (n=443) | 31.2% (n=600) | 42.2% (n=443) | 31.2% (n=600) | 42.2% (n=443) |
| glm-5.3-flash | strata | 600 | 596 | 66.7% (n=600) | 67.1% (n=596) | 26.0% (n=600) | 26.2% (n=596) | 25.8% (n=600) | 26.0% (n=596) |
| claude-haiku-4.5 | strata | 600 | 600 | 68.0% (n=600) | 68.0% (n=600) | 23.7% (n=600) | 23.7% (n=600) | 23.7% (n=600) | 23.7% (n=600) |
| glm-4.7-flash | strata | 600 | 565 | 63.3% (n=600) | 67.1% (n=566) | 19.5% (n=600) | 20.7% (n=566) | 19.5% (n=600) | 20.7% (n=566) |
| claude-fable-5.1 | strata | 600 | 14 | 2.3% (n=600) | 98.2% (n=14) | 1.7% (n=600) | 71.4% (n=14) | 1.7% (n=600) | 71.4% (n=14) |
| baseline:direct-facts | strata | 600 | 600 | 73.7% (n=600) | 73.7% (n=600) | 0.8% (n=600) | 0.8% (n=600) | 0.8% (n=600) | 0.8% (n=600) |
| baseline:story-copy | strata | 600 | 600 | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) |
| Provider | condition | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | explicit | 240 | 240 | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) |
| baseline:constraint-solver | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| baseline:constraint-solver | implicit | 240 | 240 | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) |
| baseline:forward-chain | explicit | 240 | 240 | 99.8% (n=240) | 99.8% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) |
| baseline:forward-chain | formal | 120 | 120 | 99.8% (n=120) | 99.8% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| baseline:forward-chain | implicit | 240 | 240 | 99.8% (n=240) | 99.8% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) |
| gpt-6-astra | explicit | 240 | 220 | 90.8% (n=240) | 99.1% (n=220) | 81.2% (n=240) | 88.6% (n=220) | 81.2% (n=240) | 88.6% (n=220) |
| gpt-6-astra | formal | 120 | 114 | 95.0% (n=120) | 100.0% (n=114) | 95.0% (n=120) | 100.0% (n=114) | 95.0% (n=120) | 100.0% (n=114) |
| gpt-6-astra | implicit | 240 | 216 | 88.0% (n=240) | 97.8% (n=216) | 72.1% (n=240) | 80.1% (n=216) | 72.1% (n=240) | 80.1% (n=216) |
| gpt-6-sol | explicit | 240 | 240 | 95.0% (n=240) | 95.0% (n=240) | 61.3% (n=240) | 61.3% (n=240) | 60.8% (n=240) | 60.8% (n=240) |
| gpt-6-sol | formal | 120 | 120 | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) |
| gpt-6-sol | implicit | 240 | 240 | 97.1% (n=240) | 97.1% (n=240) | 75.0% (n=240) | 75.0% (n=240) | 75.0% (n=240) | 75.0% (n=240) |
| gemini-3.7-flash | explicit | 240 | 240 | 96.9% (n=240) | 96.9% (n=240) | 68.8% (n=240) | 68.8% (n=240) | 68.8% (n=240) | 68.8% (n=240) |
| gemini-3.7-flash | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.7-flash | implicit | 240 | 240 | 81.4% (n=240) | 81.4% (n=240) | 52.9% (n=240) | 52.9% (n=240) | 52.9% (n=240) | 52.9% (n=240) |
| gpt-5.6-sol | explicit | 240 | 240 | 97.2% (n=240) | 97.2% (n=240) | 67.1% (n=240) | 67.1% (n=240) | 67.1% (n=240) | 67.1% (n=240) |
| gpt-5.6-sol | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gpt-5.6-sol | implicit | 240 | 240 | 80.4% (n=240) | 80.4% (n=240) | 54.2% (n=240) | 54.2% (n=240) | 54.2% (n=240) | 54.2% (n=240) |
| gemini-3.8-flash | explicit | 240 | 239 | 95.0% (n=240) | 95.3% (n=239) | 58.3% (n=240) | 58.6% (n=239) | 58.3% (n=240) | 58.6% (n=239) |
| gemini-3.8-flash | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.8-flash | implicit | 240 | 239 | 70.6% (n=240) | 70.9% (n=239) | 39.2% (n=240) | 39.3% (n=239) | 39.2% (n=240) | 39.3% (n=239) |
| glm-5.3 | explicit | 240 | 216 | 77.2% (n=240) | 85.7% (n=216) | 48.3% (n=240) | 53.7% (n=216) | 47.9% (n=240) | 53.2% (n=216) |
| glm-5.3 | formal | 120 | 120 | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| glm-5.3 | implicit | 240 | 227 | 73.9% (n=240) | 77.8% (n=228) | 35.8% (n=240) | 37.7% (n=228) | 35.8% (n=240) | 37.7% (n=228) |
| gpt-5.6-terra | explicit | 240 | 240 | 96.3% (n=240) | 96.3% (n=240) | 52.5% (n=240) | 52.5% (n=240) | 52.5% (n=240) | 52.5% (n=240) |
| gpt-5.6-terra | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gpt-5.6-terra | implicit | 240 | 240 | 58.1% (n=240) | 58.1% (n=240) | 24.2% (n=240) | 24.2% (n=240) | 24.2% (n=240) | 24.2% (n=240) |
| deepseek-4.1-flash | explicit | 240 | 236 | 93.7% (n=240) | 95.3% (n=236) | 53.8% (n=240) | 54.7% (n=236) | 53.8% (n=240) | 54.7% (n=236) |
| deepseek-4.1-flash | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| deepseek-4.1-flash | implicit | 240 | 239 | 53.1% (n=240) | 53.3% (n=239) | 17.9% (n=240) | 18.0% (n=239) | 17.9% (n=240) | 18.0% (n=239) |
| gemini-3.6-flash | explicit | 240 | 240 | 83.7% (n=240) | 83.7% (n=240) | 42.9% (n=240) | 42.9% (n=240) | 42.9% (n=240) | 42.9% (n=240) |
| gemini-3.6-flash | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.6-flash | implicit | 240 | 240 | 65.0% (n=240) | 65.0% (n=240) | 16.2% (n=240) | 16.2% (n=240) | 16.2% (n=240) | 16.2% (n=240) |
| deepseek-v4-pro | explicit | 240 | 239 | 92.7% (n=240) | 92.7% (n=240) | 38.8% (n=240) | 38.8% (n=240) | 38.8% (n=240) | 38.8% (n=240) |
| deepseek-v4-pro | formal | 120 | 120 | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) |
| deepseek-v4-pro | implicit | 240 | 239 | 87.3% (n=240) | 87.3% (n=240) | 19.2% (n=240) | 19.2% (n=240) | 19.2% (n=240) | 19.2% (n=240) |
| gpt-6-luna | explicit | 240 | 240 | 92.2% (n=240) | 92.2% (n=240) | 33.3% (n=240) | 33.3% (n=240) | 33.3% (n=240) | 33.3% (n=240) |
| gpt-6-luna | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 99.2% (n=120) | 99.2% (n=120) |
| gpt-6-luna | implicit | 240 | 240 | 61.5% (n=240) | 61.5% (n=240) | 22.9% (n=240) | 22.9% (n=240) | 22.9% (n=240) | 22.9% (n=240) |
| claude-sonnet-5 | explicit | 240 | 193 | 74.8% (n=240) | 93.1% (n=193) | 34.2% (n=240) | 42.5% (n=193) | 34.2% (n=240) | 42.5% (n=193) |
| claude-sonnet-5 | formal | 120 | 120 | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) |
| claude-sonnet-5 | implicit | 240 | 132 | 45.8% (n=240) | 83.3% (n=132) | 23.8% (n=240) | 43.2% (n=132) | 23.8% (n=240) | 43.2% (n=132) |
| gpt-5.6-luna | explicit | 240 | 240 | 94.5% (n=240) | 94.5% (n=240) | 40.0% (n=240) | 40.0% (n=240) | 38.3% (n=240) | 38.3% (n=240) |
| gpt-5.6-luna | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| gpt-5.6-luna | implicit | 240 | 240 | 58.9% (n=240) | 58.9% (n=240) | 16.7% (n=240) | 16.7% (n=240) | 16.7% (n=240) | 16.7% (n=240) |
| claude-opus-5.5 | explicit | 240 | 235 | 90.8% (n=240) | 92.8% (n=235) | 28.3% (n=240) | 28.9% (n=235) | 27.9% (n=240) | 28.5% (n=235) |
| claude-opus-5.5 | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| claude-opus-5.5 | implicit | 240 | 236 | 88.8% (n=240) | 90.3% (n=236) | 21.7% (n=240) | 22.0% (n=236) | 21.7% (n=240) | 22.0% (n=236) |
| claude-fable-5 | explicit | 240 | 161 | 62.8% (n=240) | 93.7% (n=161) | 34.6% (n=240) | 51.6% (n=161) | 34.6% (n=240) | 51.6% (n=161) |
| claude-fable-5 | formal | 120 | 89 | 74.2% (n=120) | 100.0% (n=89) | 74.2% (n=120) | 100.0% (n=89) | 74.2% (n=120) | 100.0% (n=89) |
| claude-fable-5 | implicit | 240 | 143 | 56.4% (n=240) | 94.7% (n=143) | 27.1% (n=240) | 45.5% (n=143) | 27.1% (n=240) | 45.5% (n=143) |
| mimo-v2.6-flash | explicit | 240 | 237 | 89.1% (n=240) | 90.2% (n=237) | 40.0% (n=240) | 40.5% (n=237) | 38.3% (n=240) | 38.8% (n=237) |
| mimo-v2.6-flash | formal | 120 | 120 | 96.3% (n=120) | 96.3% (n=120) | 95.0% (n=120) | 95.0% (n=120) | 91.7% (n=120) | 91.7% (n=120) |
| mimo-v2.6-flash | implicit | 240 | 237 | 59.0% (n=240) | 59.8% (n=237) | 13.8% (n=240) | 13.9% (n=237) | 12.9% (n=240) | 13.1% (n=237) |
| mimo-v2.6-pro | explicit | 240 | 233 | 84.6% (n=240) | 84.6% (n=240) | 38.8% (n=240) | 38.8% (n=240) | 36.7% (n=240) | 36.7% (n=240) |
| mimo-v2.6-pro | formal | 120 | 120 | 97.3% (n=120) | 97.3% (n=120) | 94.2% (n=120) | 94.2% (n=120) | 92.5% (n=120) | 92.5% (n=120) |
| mimo-v2.6-pro | implicit | 240 | 240 | 58.3% (n=240) | 58.3% (n=240) | 12.1% (n=240) | 12.1% (n=240) | 12.1% (n=240) | 12.1% (n=240) |
| claude-opus-5 | explicit | 240 | 230 | 82.2% (n=240) | 85.8% (n=230) | 58.3% (n=240) | 60.9% (n=230) | 58.3% (n=240) | 60.9% (n=230) |
| claude-opus-5 | formal | 120 | 0 | 0.0% (n=120) | — (n=0) | 0.0% (n=120) | — (n=0) | 0.0% (n=120) | — (n=0) |
| claude-opus-5 | implicit | 240 | 194 | 61.0% (n=240) | 75.4% (n=194) | 34.6% (n=240) | 42.8% (n=194) | 34.6% (n=240) | 42.8% (n=194) |
| grok-4.6 | explicit | 240 | 240 | 88.0% (n=240) | 88.0% (n=240) | 21.2% (n=240) | 21.2% (n=240) | 21.2% (n=240) | 21.2% (n=240) |
| grok-4.6 | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| grok-4.6 | implicit | 240 | 240 | 85.7% (n=240) | 85.7% (n=240) | 21.2% (n=240) | 21.2% (n=240) | 21.2% (n=240) | 21.2% (n=240) |
| kimi-k2.6 | explicit | 240 | 239 | 92.0% (n=240) | 92.4% (n=239) | 23.8% (n=240) | 23.8% (n=239) | 23.8% (n=240) | 23.8% (n=239) |
| kimi-k2.6 | formal | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| kimi-k2.6 | implicit | 240 | 240 | 84.0% (n=240) | 84.0% (n=240) | 17.5% (n=240) | 17.5% (n=240) | 17.5% (n=240) | 17.5% (n=240) |
| minimax-m3 | explicit | 240 | 238 | 69.5% (n=240) | 69.5% (n=240) | 24.6% (n=240) | 24.6% (n=240) | 24.6% (n=240) | 24.6% (n=240) |
| minimax-m3 | formal | 120 | 120 | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) |
| minimax-m3 | implicit | 240 | 239 | 52.9% (n=240) | 52.9% (n=240) | 11.2% (n=240) | 11.2% (n=240) | 11.2% (n=240) | 11.2% (n=240) |
| qwen3.8-27b | explicit | 240 | 240 | 86.5% (n=240) | 86.5% (n=240) | 24.2% (n=240) | 24.2% (n=240) | 24.2% (n=240) | 24.2% (n=240) |
| qwen3.8-27b | formal | 120 | 120 | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) |
| qwen3.8-27b | implicit | 240 | 240 | 74.6% (n=240) | 74.6% (n=240) | 12.1% (n=240) | 12.1% (n=240) | 12.1% (n=240) | 12.1% (n=240) |
| kimi-k3 | explicit | 240 | 171 | 60.6% (n=240) | 85.0% (n=171) | 18.8% (n=240) | 26.3% (n=171) | 18.8% (n=240) | 26.3% (n=171) |
| kimi-k3 | formal | 120 | 93 | 77.5% (n=120) | 100.0% (n=93) | 77.5% (n=120) | 100.0% (n=93) | 77.5% (n=120) | 100.0% (n=93) |
| kimi-k3 | implicit | 240 | 179 | 57.6% (n=240) | 77.2% (n=179) | 20.4% (n=240) | 27.4% (n=179) | 20.4% (n=240) | 27.4% (n=179) |
| glm-5.3-flash | explicit | 240 | 236 | 64.0% (n=240) | 65.1% (n=236) | 18.3% (n=240) | 18.6% (n=236) | 17.9% (n=240) | 18.2% (n=236) |
| glm-5.3-flash | formal | 120 | 120 | 99.0% (n=120) | 99.0% (n=120) | 83.3% (n=120) | 83.3% (n=120) | 83.3% (n=120) | 83.3% (n=120) |
| glm-5.3-flash | implicit | 240 | 240 | 53.2% (n=240) | 53.2% (n=240) | 5.0% (n=240) | 5.0% (n=240) | 5.0% (n=240) | 5.0% (n=240) |
| claude-haiku-4.5 | explicit | 240 | 240 | 68.0% (n=240) | 68.0% (n=240) | 5.8% (n=240) | 5.8% (n=240) | 5.8% (n=240) | 5.8% (n=240) |
| claude-haiku-4.5 | formal | 120 | 120 | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| claude-haiku-4.5 | implicit | 240 | 240 | 53.0% (n=240) | 53.0% (n=240) | 4.2% (n=240) | 4.2% (n=240) | 4.2% (n=240) | 4.2% (n=240) |
| glm-4.7-flash | explicit | 240 | 221 | 59.1% (n=240) | 63.9% (n=222) | 6.7% (n=240) | 7.2% (n=222) | 6.7% (n=240) | 7.2% (n=222) |
| glm-4.7-flash | formal | 120 | 120 | 90.4% (n=120) | 90.4% (n=120) | 77.5% (n=120) | 77.5% (n=120) | 77.5% (n=120) | 77.5% (n=120) |
| glm-4.7-flash | implicit | 240 | 224 | 54.0% (n=240) | 57.8% (n=224) | 3.3% (n=240) | 3.6% (n=224) | 3.3% (n=240) | 3.6% (n=224) |
| claude-fable-5.1 | explicit | 240 | 2 | 0.8% (n=240) | 100.0% (n=2) | 0.8% (n=240) | 100.0% (n=2) | 0.8% (n=240) | 100.0% (n=2) |
| claude-fable-5.1 | formal | 120 | 3 | 2.5% (n=120) | 100.0% (n=3) | 2.5% (n=120) | 100.0% (n=3) | 2.5% (n=120) | 100.0% (n=3) |
| claude-fable-5.1 | implicit | 240 | 9 | 3.6% (n=240) | 97.3% (n=9) | 2.1% (n=240) | 55.6% (n=9) | 2.1% (n=240) | 55.6% (n=9) |
| baseline:direct-facts | explicit | 240 | 240 | 73.7% (n=240) | 73.7% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) |
| baseline:direct-facts | formal | 120 | 120 | 73.7% (n=120) | 73.7% (n=120) | 0.8% (n=120) | 0.8% (n=120) | 0.8% (n=120) | 0.8% (n=120) |
| baseline:direct-facts | implicit | 240 | 240 | 73.7% (n=240) | 73.7% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) |
| baseline:story-copy | explicit | 240 | 240 | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) |
| baseline:story-copy | formal | 120 | 120 | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) |
| baseline:story-copy | implicit | 240 | 240 | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) |
| Provider | realization | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| baseline:constraint-solver | 1 | 240 | 240 | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) |
| baseline:constraint-solver | 2 | 240 | 240 | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) | 100.0% (n=240) |
| baseline:forward-chain | 0 | 120 | 120 | 99.8% (n=120) | 99.8% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| baseline:forward-chain | 1 | 240 | 240 | 99.8% (n=240) | 99.8% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) |
| baseline:forward-chain | 2 | 240 | 240 | 99.8% (n=240) | 99.8% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) | 98.3% (n=240) |
| gpt-6-astra | 0 | 120 | 114 | 95.0% (n=120) | 100.0% (n=114) | 95.0% (n=120) | 100.0% (n=114) | 95.0% (n=120) | 100.0% (n=114) |
| gpt-6-astra | 1 | 240 | 223 | 91.4% (n=240) | 98.4% (n=223) | 79.2% (n=240) | 85.2% (n=223) | 79.2% (n=240) | 85.2% (n=223) |
| gpt-6-astra | 2 | 240 | 213 | 87.5% (n=240) | 98.5% (n=213) | 74.2% (n=240) | 83.6% (n=213) | 74.2% (n=240) | 83.6% (n=213) |
| gpt-6-sol | 0 | 120 | 120 | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) | 99.2% (n=120) |
| gpt-6-sol | 1 | 240 | 240 | 96.1% (n=240) | 96.1% (n=240) | 70.4% (n=240) | 70.4% (n=240) | 70.4% (n=240) | 70.4% (n=240) |
| gpt-6-sol | 2 | 240 | 240 | 96.0% (n=240) | 96.0% (n=240) | 65.8% (n=240) | 65.8% (n=240) | 65.4% (n=240) | 65.4% (n=240) |
| gemini-3.7-flash | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.7-flash | 1 | 240 | 240 | 89.6% (n=240) | 89.6% (n=240) | 60.0% (n=240) | 60.0% (n=240) | 60.0% (n=240) | 60.0% (n=240) |
| gemini-3.7-flash | 2 | 240 | 240 | 88.7% (n=240) | 88.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) |
| gpt-5.6-sol | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gpt-5.6-sol | 1 | 240 | 240 | 87.7% (n=240) | 87.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) | 61.7% (n=240) |
| gpt-5.6-sol | 2 | 240 | 240 | 89.8% (n=240) | 89.8% (n=240) | 59.6% (n=240) | 59.6% (n=240) | 59.6% (n=240) | 59.6% (n=240) |
| gemini-3.8-flash | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.8-flash | 1 | 240 | 239 | 81.8% (n=240) | 82.1% (n=239) | 49.6% (n=240) | 49.8% (n=239) | 49.6% (n=240) | 49.8% (n=239) |
| gemini-3.8-flash | 2 | 240 | 239 | 83.8% (n=240) | 84.1% (n=239) | 47.9% (n=240) | 48.1% (n=239) | 47.9% (n=240) | 48.1% (n=239) |
| glm-5.3 | 0 | 120 | 120 | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| glm-5.3 | 1 | 240 | 221 | 74.8% (n=240) | 80.8% (n=222) | 44.2% (n=240) | 47.7% (n=222) | 44.2% (n=240) | 47.7% (n=222) |
| glm-5.3 | 2 | 240 | 222 | 76.2% (n=240) | 82.4% (n=222) | 40.0% (n=240) | 43.2% (n=222) | 39.6% (n=240) | 42.8% (n=222) |
| gpt-5.6-terra | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gpt-5.6-terra | 1 | 240 | 240 | 77.6% (n=240) | 77.6% (n=240) | 39.6% (n=240) | 39.6% (n=240) | 39.6% (n=240) | 39.6% (n=240) |
| gpt-5.6-terra | 2 | 240 | 240 | 76.8% (n=240) | 76.8% (n=240) | 37.1% (n=240) | 37.1% (n=240) | 37.1% (n=240) | 37.1% (n=240) |
| deepseek-4.1-flash | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| deepseek-4.1-flash | 1 | 240 | 236 | 73.9% (n=240) | 75.1% (n=236) | 37.1% (n=240) | 37.7% (n=236) | 37.1% (n=240) | 37.7% (n=236) |
| deepseek-4.1-flash | 2 | 240 | 239 | 72.9% (n=240) | 73.2% (n=239) | 34.6% (n=240) | 34.7% (n=239) | 34.6% (n=240) | 34.7% (n=239) |
| gemini-3.6-flash | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| gemini-3.6-flash | 1 | 240 | 240 | 74.5% (n=240) | 74.5% (n=240) | 30.0% (n=240) | 30.0% (n=240) | 30.0% (n=240) | 30.0% (n=240) |
| gemini-3.6-flash | 2 | 240 | 240 | 74.2% (n=240) | 74.2% (n=240) | 29.2% (n=240) | 29.2% (n=240) | 29.2% (n=240) | 29.2% (n=240) |
| deepseek-v4-pro | 0 | 120 | 120 | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) |
| deepseek-v4-pro | 1 | 240 | 238 | 89.0% (n=240) | 89.0% (n=240) | 29.2% (n=240) | 29.2% (n=240) | 29.2% (n=240) | 29.2% (n=240) |
| deepseek-v4-pro | 2 | 240 | 240 | 90.9% (n=240) | 90.9% (n=240) | 28.7% (n=240) | 28.7% (n=240) | 28.7% (n=240) | 28.7% (n=240) |
| gpt-6-luna | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 99.2% (n=120) | 99.2% (n=120) |
| gpt-6-luna | 1 | 240 | 240 | 75.7% (n=240) | 75.7% (n=240) | 25.4% (n=240) | 25.4% (n=240) | 25.4% (n=240) | 25.4% (n=240) |
| gpt-6-luna | 2 | 240 | 240 | 77.9% (n=240) | 77.9% (n=240) | 30.8% (n=240) | 30.8% (n=240) | 30.8% (n=240) | 30.8% (n=240) |
| claude-sonnet-5 | 0 | 120 | 120 | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) | 93.3% (n=120) |
| claude-sonnet-5 | 1 | 240 | 162 | 59.7% (n=240) | 88.4% (n=162) | 28.3% (n=240) | 42.0% (n=162) | 28.3% (n=240) | 42.0% (n=162) |
| claude-sonnet-5 | 2 | 240 | 163 | 61.0% (n=240) | 89.8% (n=163) | 29.6% (n=240) | 43.6% (n=163) | 29.6% (n=240) | 43.6% (n=163) |
| gpt-5.6-luna | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| gpt-5.6-luna | 1 | 240 | 240 | 75.0% (n=240) | 75.0% (n=240) | 25.0% (n=240) | 25.0% (n=240) | 23.8% (n=240) | 23.8% (n=240) |
| gpt-5.6-luna | 2 | 240 | 240 | 78.5% (n=240) | 78.5% (n=240) | 31.7% (n=240) | 31.7% (n=240) | 31.2% (n=240) | 31.2% (n=240) |
| claude-opus-5.5 | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| claude-opus-5.5 | 1 | 240 | 237 | 90.4% (n=240) | 91.6% (n=237) | 27.1% (n=240) | 27.4% (n=237) | 26.7% (n=240) | 27.0% (n=237) |
| claude-opus-5.5 | 2 | 240 | 234 | 89.3% (n=240) | 91.5% (n=234) | 22.9% (n=240) | 23.5% (n=234) | 22.9% (n=240) | 23.5% (n=234) |
| claude-fable-5 | 0 | 120 | 89 | 74.2% (n=120) | 100.0% (n=89) | 74.2% (n=120) | 100.0% (n=89) | 74.2% (n=120) | 100.0% (n=89) |
| claude-fable-5 | 1 | 240 | 151 | 59.5% (n=240) | 94.5% (n=151) | 33.3% (n=240) | 53.0% (n=151) | 33.3% (n=240) | 53.0% (n=151) |
| claude-fable-5 | 2 | 240 | 153 | 59.8% (n=240) | 93.8% (n=153) | 28.3% (n=240) | 44.4% (n=153) | 28.3% (n=240) | 44.4% (n=153) |
| mimo-v2.6-flash | 0 | 120 | 120 | 96.3% (n=120) | 96.3% (n=120) | 95.0% (n=120) | 95.0% (n=120) | 91.7% (n=120) | 91.7% (n=120) |
| mimo-v2.6-flash | 1 | 240 | 236 | 75.6% (n=240) | 76.8% (n=236) | 28.7% (n=240) | 29.2% (n=236) | 26.7% (n=240) | 27.1% (n=236) |
| mimo-v2.6-flash | 2 | 240 | 238 | 72.6% (n=240) | 73.2% (n=238) | 25.0% (n=240) | 25.2% (n=238) | 24.6% (n=240) | 24.8% (n=238) |
| mimo-v2.6-pro | 0 | 120 | 120 | 97.3% (n=120) | 97.3% (n=120) | 94.2% (n=120) | 94.2% (n=120) | 92.5% (n=120) | 92.5% (n=120) |
| mimo-v2.6-pro | 1 | 240 | 236 | 70.6% (n=240) | 70.6% (n=240) | 25.8% (n=240) | 25.8% (n=240) | 24.6% (n=240) | 24.6% (n=240) |
| mimo-v2.6-pro | 2 | 240 | 237 | 72.2% (n=240) | 72.2% (n=240) | 25.0% (n=240) | 25.0% (n=240) | 24.2% (n=240) | 24.2% (n=240) |
| claude-opus-5 | 0 | 120 | 0 | 0.0% (n=120) | — (n=0) | 0.0% (n=120) | — (n=0) | 0.0% (n=120) | — (n=0) |
| claude-opus-5 | 1 | 240 | 215 | 71.0% (n=240) | 79.2% (n=215) | 46.2% (n=240) | 51.6% (n=215) | 46.2% (n=240) | 51.6% (n=215) |
| claude-opus-5 | 2 | 240 | 209 | 72.3% (n=240) | 83.0% (n=209) | 46.7% (n=240) | 53.6% (n=209) | 46.7% (n=240) | 53.6% (n=209) |
| grok-4.6 | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| grok-4.6 | 1 | 240 | 240 | 86.5% (n=240) | 86.5% (n=240) | 22.9% (n=240) | 22.9% (n=240) | 22.9% (n=240) | 22.9% (n=240) |
| grok-4.6 | 2 | 240 | 240 | 87.1% (n=240) | 87.1% (n=240) | 19.6% (n=240) | 19.6% (n=240) | 19.6% (n=240) | 19.6% (n=240) |
| kimi-k2.6 | 0 | 120 | 120 | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) | 100.0% (n=120) |
| kimi-k2.6 | 1 | 240 | 240 | 88.2% (n=240) | 88.2% (n=240) | 20.4% (n=240) | 20.4% (n=240) | 20.4% (n=240) | 20.4% (n=240) |
| kimi-k2.6 | 2 | 240 | 239 | 87.8% (n=240) | 88.2% (n=239) | 20.8% (n=240) | 20.9% (n=239) | 20.8% (n=240) | 20.9% (n=239) |
| minimax-m3 | 0 | 120 | 120 | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) | 97.5% (n=120) |
| minimax-m3 | 1 | 240 | 239 | 59.4% (n=240) | 59.4% (n=240) | 16.2% (n=240) | 16.2% (n=240) | 16.2% (n=240) | 16.2% (n=240) |
| minimax-m3 | 2 | 240 | 238 | 63.0% (n=240) | 63.0% (n=240) | 19.6% (n=240) | 19.6% (n=240) | 19.6% (n=240) | 19.6% (n=240) |
| qwen3.8-27b | 0 | 120 | 120 | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) | 95.8% (n=120) |
| qwen3.8-27b | 1 | 240 | 240 | 82.3% (n=240) | 82.3% (n=240) | 18.8% (n=240) | 18.8% (n=240) | 18.8% (n=240) | 18.8% (n=240) |
| qwen3.8-27b | 2 | 240 | 240 | 78.8% (n=240) | 78.8% (n=240) | 17.5% (n=240) | 17.5% (n=240) | 17.5% (n=240) | 17.5% (n=240) |
| kimi-k3 | 0 | 120 | 93 | 77.5% (n=120) | 100.0% (n=93) | 77.5% (n=120) | 100.0% (n=93) | 77.5% (n=120) | 100.0% (n=93) |
| kimi-k3 | 1 | 240 | 175 | 58.2% (n=240) | 79.9% (n=175) | 19.2% (n=240) | 26.3% (n=175) | 19.2% (n=240) | 26.3% (n=175) |
| kimi-k3 | 2 | 240 | 175 | 60.0% (n=240) | 82.2% (n=175) | 20.0% (n=240) | 27.4% (n=175) | 20.0% (n=240) | 27.4% (n=175) |
| glm-5.3-flash | 0 | 120 | 120 | 99.0% (n=120) | 99.0% (n=120) | 83.3% (n=120) | 83.3% (n=120) | 83.3% (n=120) | 83.3% (n=120) |
| glm-5.3-flash | 1 | 240 | 239 | 60.6% (n=240) | 60.8% (n=239) | 11.2% (n=240) | 11.3% (n=239) | 10.8% (n=240) | 10.9% (n=239) |
| glm-5.3-flash | 2 | 240 | 237 | 56.6% (n=240) | 57.3% (n=237) | 12.1% (n=240) | 12.2% (n=237) | 12.1% (n=240) | 12.2% (n=237) |
| claude-haiku-4.5 | 0 | 120 | 120 | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) | 98.3% (n=120) |
| claude-haiku-4.5 | 1 | 240 | 240 | 60.1% (n=240) | 60.1% (n=240) | 3.3% (n=240) | 3.3% (n=240) | 3.3% (n=240) | 3.3% (n=240) |
| claude-haiku-4.5 | 2 | 240 | 240 | 60.8% (n=240) | 60.8% (n=240) | 6.7% (n=240) | 6.7% (n=240) | 6.7% (n=240) | 6.7% (n=240) |
| glm-4.7-flash | 0 | 120 | 120 | 90.4% (n=120) | 90.4% (n=120) | 77.5% (n=120) | 77.5% (n=120) | 77.5% (n=120) | 77.5% (n=120) |
| glm-4.7-flash | 1 | 240 | 220 | 56.5% (n=240) | 61.4% (n=221) | 4.2% (n=240) | 4.5% (n=221) | 4.2% (n=240) | 4.5% (n=221) |
| glm-4.7-flash | 2 | 240 | 225 | 56.5% (n=240) | 60.3% (n=225) | 5.8% (n=240) | 6.2% (n=225) | 5.8% (n=240) | 6.2% (n=225) |
| claude-fable-5.1 | 0 | 120 | 3 | 2.5% (n=120) | 100.0% (n=3) | 2.5% (n=120) | 100.0% (n=3) | 2.5% (n=120) | 100.0% (n=3) |
| claude-fable-5.1 | 1 | 240 | 7 | 2.8% (n=240) | 96.5% (n=7) | 1.2% (n=240) | 42.9% (n=7) | 1.2% (n=240) | 42.9% (n=7) |
| claude-fable-5.1 | 2 | 240 | 4 | 1.7% (n=240) | 100.0% (n=4) | 1.7% (n=240) | 100.0% (n=4) | 1.7% (n=240) | 100.0% (n=4) |
| baseline:direct-facts | 0 | 120 | 120 | 73.7% (n=120) | 73.7% (n=120) | 0.8% (n=120) | 0.8% (n=120) | 0.8% (n=120) | 0.8% (n=120) |
| baseline:direct-facts | 1 | 240 | 240 | 73.7% (n=240) | 73.7% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) |
| baseline:direct-facts | 2 | 240 | 240 | 73.7% (n=240) | 73.7% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) | 0.8% (n=240) |
| baseline:story-copy | 0 | 120 | 120 | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) | 0.0% (n=120) |
| baseline:story-copy | 1 | 240 | 240 | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) |
| baseline:story-copy | 2 | 240 | 240 | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) | 0.0% (n=240) |
| Provider | genre | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | hotel | 110 | 110 | 100.0% (n=110) | 100.0% (n=110) | 100.0% (n=110) | 100.0% (n=110) | 100.0% (n=110) | 100.0% (n=110) |
| baseline:constraint-solver | manor | 105 | 105 | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) |
| baseline:constraint-solver | maritime | 90 | 90 | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) |
| baseline:constraint-solver | observatory | 125 | 125 | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) |
| baseline:constraint-solver | theater | 170 | 170 | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) |
| baseline:forward-chain | hotel | 110 | 110 | 99.2% (n=110) | 99.2% (n=110) | 90.9% (n=110) | 90.9% (n=110) | 90.9% (n=110) | 90.9% (n=110) |
| baseline:forward-chain | manor | 105 | 105 | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) | 100.0% (n=105) |
| baseline:forward-chain | maritime | 90 | 90 | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) | 100.0% (n=90) |
| baseline:forward-chain | observatory | 125 | 125 | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) | 100.0% (n=125) |
| baseline:forward-chain | theater | 170 | 170 | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) | 100.0% (n=170) |
| gpt-6-astra | hotel | 110 | 104 | 93.3% (n=110) | 98.7% (n=104) | 83.6% (n=110) | 88.5% (n=104) | 83.6% (n=110) | 88.5% (n=104) |
| gpt-6-astra | manor | 105 | 95 | 89.4% (n=105) | 98.9% (n=95) | 79.0% (n=105) | 87.4% (n=95) | 79.0% (n=105) | 87.4% (n=95) |
| gpt-6-astra | maritime | 90 | 85 | 93.0% (n=90) | 98.5% (n=85) | 78.9% (n=90) | 83.5% (n=85) | 78.9% (n=90) | 83.5% (n=85) |
| gpt-6-astra | observatory | 125 | 114 | 90.5% (n=125) | 99.2% (n=114) | 83.2% (n=125) | 91.2% (n=114) | 83.2% (n=125) | 91.2% (n=114) |
| gpt-6-astra | theater | 170 | 152 | 88.2% (n=170) | 98.6% (n=152) | 77.6% (n=170) | 86.8% (n=152) | 77.6% (n=170) | 86.8% (n=152) |
| gpt-6-sol | hotel | 110 | 110 | 96.0% (n=110) | 96.0% (n=110) | 74.5% (n=110) | 74.5% (n=110) | 74.5% (n=110) | 74.5% (n=110) |
| gpt-6-sol | manor | 105 | 105 | 95.6% (n=105) | 95.6% (n=105) | 71.4% (n=105) | 71.4% (n=105) | 71.4% (n=105) | 71.4% (n=105) |
| gpt-6-sol | maritime | 90 | 90 | 96.5% (n=90) | 96.5% (n=90) | 68.9% (n=90) | 68.9% (n=90) | 67.8% (n=90) | 67.8% (n=90) |
| gpt-6-sol | observatory | 125 | 125 | 97.1% (n=125) | 97.1% (n=125) | 77.6% (n=125) | 77.6% (n=125) | 77.6% (n=125) | 77.6% (n=125) |
| gpt-6-sol | theater | 170 | 170 | 97.4% (n=170) | 97.4% (n=170) | 76.5% (n=170) | 76.5% (n=170) | 76.5% (n=170) | 76.5% (n=170) |
| gemini-3.7-flash | hotel | 110 | 110 | 89.5% (n=110) | 89.5% (n=110) | 66.4% (n=110) | 66.4% (n=110) | 66.4% (n=110) | 66.4% (n=110) |
| gemini-3.7-flash | manor | 105 | 105 | 92.8% (n=105) | 92.8% (n=105) | 69.5% (n=105) | 69.5% (n=105) | 69.5% (n=105) | 69.5% (n=105) |
| gemini-3.7-flash | maritime | 90 | 90 | 89.7% (n=90) | 89.7% (n=90) | 63.3% (n=90) | 63.3% (n=90) | 63.3% (n=90) | 63.3% (n=90) |
| gemini-3.7-flash | observatory | 125 | 125 | 91.7% (n=125) | 91.7% (n=125) | 67.2% (n=125) | 67.2% (n=125) | 67.2% (n=125) | 67.2% (n=125) |
| gemini-3.7-flash | theater | 170 | 170 | 92.1% (n=170) | 92.1% (n=170) | 73.5% (n=170) | 73.5% (n=170) | 73.5% (n=170) | 73.5% (n=170) |
| gpt-5.6-sol | hotel | 110 | 110 | 92.3% (n=110) | 92.3% (n=110) | 76.4% (n=110) | 76.4% (n=110) | 76.4% (n=110) | 76.4% (n=110) |
| gpt-5.6-sol | manor | 105 | 105 | 90.0% (n=105) | 90.0% (n=105) | 65.7% (n=105) | 65.7% (n=105) | 65.7% (n=105) | 65.7% (n=105) |
| gpt-5.6-sol | maritime | 90 | 90 | 87.8% (n=90) | 87.8% (n=90) | 50.0% (n=90) | 50.0% (n=90) | 50.0% (n=90) | 50.0% (n=90) |
| gpt-5.6-sol | observatory | 125 | 125 | 90.2% (n=125) | 90.2% (n=125) | 75.2% (n=125) | 75.2% (n=125) | 75.2% (n=125) | 75.2% (n=125) |
| gpt-5.6-sol | theater | 170 | 170 | 93.1% (n=170) | 93.1% (n=170) | 70.0% (n=170) | 70.0% (n=170) | 70.0% (n=170) | 70.0% (n=170) |
| gemini-3.8-flash | hotel | 110 | 109 | 85.2% (n=110) | 85.9% (n=109) | 57.3% (n=110) | 57.8% (n=109) | 57.3% (n=110) | 57.8% (n=109) |
| gemini-3.8-flash | manor | 105 | 104 | 88.5% (n=105) | 89.4% (n=104) | 61.0% (n=105) | 61.5% (n=104) | 61.0% (n=105) | 61.5% (n=104) |
| gemini-3.8-flash | maritime | 90 | 90 | 86.7% (n=90) | 86.7% (n=90) | 61.1% (n=90) | 61.1% (n=90) | 61.1% (n=90) | 61.1% (n=90) |
| gemini-3.8-flash | observatory | 125 | 125 | 85.1% (n=125) | 85.1% (n=125) | 56.8% (n=125) | 56.8% (n=125) | 56.8% (n=125) | 56.8% (n=125) |
| gemini-3.8-flash | theater | 170 | 170 | 86.1% (n=170) | 86.1% (n=170) | 59.4% (n=170) | 59.4% (n=170) | 59.4% (n=170) | 59.4% (n=170) |
| glm-5.3 | hotel | 110 | 106 | 81.8% (n=110) | 84.9% (n=106) | 52.7% (n=110) | 54.7% (n=106) | 52.7% (n=110) | 54.7% (n=106) |
| glm-5.3 | manor | 105 | 100 | 78.6% (n=105) | 82.6% (n=100) | 54.3% (n=105) | 57.0% (n=100) | 53.3% (n=105) | 56.0% (n=100) |
| glm-5.3 | maritime | 90 | 83 | 79.3% (n=90) | 86.0% (n=83) | 47.8% (n=90) | 51.8% (n=83) | 47.8% (n=90) | 51.8% (n=83) |
| glm-5.3 | observatory | 125 | 116 | 77.6% (n=125) | 83.6% (n=116) | 54.4% (n=125) | 58.6% (n=116) | 54.4% (n=125) | 58.6% (n=116) |
| glm-5.3 | theater | 170 | 158 | 82.1% (n=170) | 87.8% (n=159) | 55.3% (n=170) | 59.1% (n=159) | 55.3% (n=170) | 59.1% (n=159) |
| gpt-5.6-terra | hotel | 110 | 110 | 80.3% (n=110) | 80.3% (n=110) | 50.0% (n=110) | 50.0% (n=110) | 50.0% (n=110) | 50.0% (n=110) |
| gpt-5.6-terra | manor | 105 | 105 | 82.6% (n=105) | 82.6% (n=105) | 52.4% (n=105) | 52.4% (n=105) | 52.4% (n=105) | 52.4% (n=105) |
| gpt-5.6-terra | maritime | 90 | 90 | 80.0% (n=90) | 80.0% (n=90) | 44.4% (n=90) | 44.4% (n=90) | 44.4% (n=90) | 44.4% (n=90) |
| gpt-5.6-terra | observatory | 125 | 125 | 82.8% (n=125) | 82.8% (n=125) | 54.4% (n=125) | 54.4% (n=125) | 54.4% (n=125) | 54.4% (n=125) |
| gpt-5.6-terra | theater | 170 | 170 | 82.4% (n=170) | 82.4% (n=170) | 50.6% (n=170) | 50.6% (n=170) | 50.6% (n=170) | 50.6% (n=170) |
| deepseek-4.1-flash | hotel | 110 | 108 | 80.9% (n=110) | 82.4% (n=108) | 48.2% (n=110) | 49.1% (n=108) | 48.2% (n=110) | 49.1% (n=108) |
| deepseek-4.1-flash | manor | 105 | 105 | 80.9% (n=105) | 80.9% (n=105) | 55.2% (n=105) | 55.2% (n=105) | 55.2% (n=105) | 55.2% (n=105) |
| deepseek-4.1-flash | maritime | 90 | 90 | 83.1% (n=90) | 83.1% (n=90) | 46.7% (n=90) | 46.7% (n=90) | 46.7% (n=90) | 46.7% (n=90) |
| deepseek-4.1-flash | observatory | 125 | 125 | 73.0% (n=125) | 73.0% (n=125) | 46.4% (n=125) | 46.4% (n=125) | 46.4% (n=125) | 46.4% (n=125) |
| deepseek-4.1-flash | theater | 170 | 167 | 77.8% (n=170) | 79.2% (n=167) | 47.6% (n=170) | 48.5% (n=167) | 47.6% (n=170) | 48.5% (n=167) |
| gemini-3.6-flash | hotel | 110 | 110 | 80.7% (n=110) | 80.7% (n=110) | 44.5% (n=110) | 44.5% (n=110) | 44.5% (n=110) | 44.5% (n=110) |
| gemini-3.6-flash | manor | 105 | 105 | 80.6% (n=105) | 80.6% (n=105) | 42.9% (n=105) | 42.9% (n=105) | 42.9% (n=105) | 42.9% (n=105) |
| gemini-3.6-flash | maritime | 90 | 90 | 79.6% (n=90) | 79.6% (n=90) | 41.1% (n=90) | 41.1% (n=90) | 41.1% (n=90) | 41.1% (n=90) |
| gemini-3.6-flash | observatory | 125 | 125 | 80.6% (n=125) | 80.6% (n=125) | 45.6% (n=125) | 45.6% (n=125) | 45.6% (n=125) | 45.6% (n=125) |
| gemini-3.6-flash | theater | 170 | 170 | 77.2% (n=170) | 77.2% (n=170) | 43.5% (n=170) | 43.5% (n=170) | 43.5% (n=170) | 43.5% (n=170) |
| deepseek-v4-pro | hotel | 110 | 110 | 94.3% (n=110) | 94.3% (n=110) | 42.7% (n=110) | 42.7% (n=110) | 42.7% (n=110) | 42.7% (n=110) |
| deepseek-v4-pro | manor | 105 | 105 | 94.1% (n=105) | 94.1% (n=105) | 52.4% (n=105) | 52.4% (n=105) | 52.4% (n=105) | 52.4% (n=105) |
| deepseek-v4-pro | maritime | 90 | 90 | 88.7% (n=90) | 88.7% (n=90) | 34.4% (n=90) | 34.4% (n=90) | 34.4% (n=90) | 34.4% (n=90) |
| deepseek-v4-pro | observatory | 125 | 125 | 92.2% (n=125) | 92.2% (n=125) | 44.8% (n=125) | 44.8% (n=125) | 44.8% (n=125) | 44.8% (n=125) |
| deepseek-v4-pro | theater | 170 | 168 | 89.0% (n=170) | 89.0% (n=170) | 39.4% (n=170) | 39.4% (n=170) | 39.4% (n=170) | 39.4% (n=170) |
| gpt-6-luna | hotel | 110 | 110 | 78.1% (n=110) | 78.1% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) |
| gpt-6-luna | manor | 105 | 105 | 82.4% (n=105) | 82.4% (n=105) | 49.5% (n=105) | 49.5% (n=105) | 48.6% (n=105) | 48.6% (n=105) |
| gpt-6-luna | maritime | 90 | 90 | 77.8% (n=90) | 77.8% (n=90) | 31.1% (n=90) | 31.1% (n=90) | 31.1% (n=90) | 31.1% (n=90) |
| gpt-6-luna | observatory | 125 | 125 | 85.0% (n=125) | 85.0% (n=125) | 49.6% (n=125) | 49.6% (n=125) | 49.6% (n=125) | 49.6% (n=125) |
| gpt-6-luna | theater | 170 | 170 | 82.4% (n=170) | 82.4% (n=170) | 43.5% (n=170) | 43.5% (n=170) | 43.5% (n=170) | 43.5% (n=170) |
| claude-sonnet-5 | hotel | 110 | 106 | 85.8% (n=110) | 89.1% (n=106) | 49.1% (n=110) | 50.9% (n=106) | 49.1% (n=110) | 50.9% (n=106) |
| claude-sonnet-5 | manor | 105 | 62 | 54.2% (n=105) | 91.8% (n=62) | 40.0% (n=105) | 67.7% (n=62) | 40.0% (n=105) | 67.7% (n=62) |
| claude-sonnet-5 | maritime | 90 | 53 | 54.9% (n=90) | 93.3% (n=53) | 33.3% (n=90) | 56.6% (n=53) | 33.3% (n=90) | 56.6% (n=53) |
| claude-sonnet-5 | observatory | 125 | 59 | 44.0% (n=125) | 93.2% (n=59) | 32.0% (n=125) | 67.8% (n=59) | 32.0% (n=125) | 67.8% (n=59) |
| claude-sonnet-5 | theater | 170 | 165 | 85.8% (n=170) | 88.4% (n=165) | 50.0% (n=170) | 51.5% (n=165) | 50.0% (n=170) | 51.5% (n=165) |
| gpt-5.6-luna | hotel | 110 | 110 | 84.2% (n=110) | 84.2% (n=110) | 43.6% (n=110) | 43.6% (n=110) | 42.7% (n=110) | 42.7% (n=110) |
| gpt-5.6-luna | manor | 105 | 105 | 82.7% (n=105) | 82.7% (n=105) | 44.8% (n=105) | 44.8% (n=105) | 44.8% (n=105) | 44.8% (n=105) |
| gpt-5.6-luna | maritime | 90 | 90 | 79.5% (n=90) | 79.5% (n=90) | 26.7% (n=90) | 26.7% (n=90) | 26.7% (n=90) | 26.7% (n=90) |
| gpt-5.6-luna | observatory | 125 | 125 | 79.7% (n=125) | 79.7% (n=125) | 48.8% (n=125) | 48.8% (n=125) | 47.2% (n=125) | 47.2% (n=125) |
| gpt-5.6-luna | theater | 170 | 170 | 80.9% (n=170) | 80.9% (n=170) | 44.7% (n=170) | 44.7% (n=170) | 42.9% (n=170) | 42.9% (n=170) |
| claude-opus-5.5 | hotel | 110 | 110 | 93.2% (n=110) | 93.2% (n=110) | 40.0% (n=110) | 40.0% (n=110) | 40.0% (n=110) | 40.0% (n=110) |
| claude-opus-5.5 | manor | 105 | 105 | 94.5% (n=105) | 94.5% (n=105) | 41.0% (n=105) | 41.0% (n=105) | 41.0% (n=105) | 41.0% (n=105) |
| claude-opus-5.5 | maritime | 90 | 88 | 91.6% (n=90) | 93.7% (n=88) | 41.1% (n=90) | 42.0% (n=88) | 41.1% (n=90) | 42.0% (n=88) |
| claude-opus-5.5 | observatory | 125 | 121 | 89.5% (n=125) | 92.4% (n=121) | 42.4% (n=125) | 43.8% (n=121) | 42.4% (n=125) | 43.8% (n=121) |
| claude-opus-5.5 | theater | 170 | 167 | 91.3% (n=170) | 92.9% (n=167) | 37.1% (n=170) | 37.7% (n=167) | 36.5% (n=170) | 37.1% (n=167) |
| claude-fable-5 | hotel | 110 | 59 | 51.1% (n=110) | 95.3% (n=59) | 29.1% (n=110) | 54.2% (n=59) | 29.1% (n=110) | 54.2% (n=59) |
| claude-fable-5 | manor | 105 | 70 | 64.1% (n=105) | 96.2% (n=70) | 42.9% (n=105) | 64.3% (n=70) | 42.9% (n=105) | 64.3% (n=70) |
| claude-fable-5 | maritime | 90 | 67 | 71.5% (n=90) | 96.0% (n=67) | 46.7% (n=90) | 62.7% (n=67) | 46.7% (n=90) | 62.7% (n=67) |
| claude-fable-5 | observatory | 125 | 109 | 81.8% (n=125) | 93.8% (n=109) | 50.4% (n=125) | 57.8% (n=109) | 50.4% (n=125) | 57.8% (n=109) |
| claude-fable-5 | theater | 170 | 88 | 50.0% (n=170) | 96.7% (n=88) | 32.4% (n=170) | 62.5% (n=88) | 32.4% (n=170) | 62.5% (n=88) |
| mimo-v2.6-flash | hotel | 110 | 108 | 78.8% (n=110) | 80.2% (n=108) | 42.7% (n=110) | 43.5% (n=108) | 41.8% (n=110) | 42.6% (n=108) |
| mimo-v2.6-flash | manor | 105 | 104 | 83.0% (n=105) | 83.8% (n=104) | 45.7% (n=105) | 46.2% (n=104) | 43.8% (n=105) | 44.2% (n=104) |
| mimo-v2.6-flash | maritime | 90 | 89 | 78.3% (n=90) | 79.2% (n=89) | 38.9% (n=90) | 39.3% (n=89) | 38.9% (n=90) | 39.3% (n=89) |
| mimo-v2.6-flash | observatory | 125 | 125 | 77.0% (n=125) | 77.0% (n=125) | 36.8% (n=125) | 36.8% (n=125) | 35.2% (n=125) | 35.2% (n=125) |
| mimo-v2.6-flash | theater | 170 | 168 | 76.8% (n=170) | 77.7% (n=168) | 39.4% (n=170) | 39.9% (n=168) | 36.5% (n=170) | 36.9% (n=168) |
| mimo-v2.6-pro | hotel | 110 | 110 | 80.0% (n=110) | 80.0% (n=110) | 41.8% (n=110) | 41.8% (n=110) | 41.8% (n=110) | 41.8% (n=110) |
| mimo-v2.6-pro | manor | 105 | 105 | 83.0% (n=105) | 83.0% (n=105) | 44.8% (n=105) | 44.8% (n=105) | 41.9% (n=105) | 41.9% (n=105) |
| mimo-v2.6-pro | maritime | 90 | 87 | 77.5% (n=90) | 77.5% (n=90) | 36.7% (n=90) | 36.7% (n=90) | 36.7% (n=90) | 36.7% (n=90) |
| mimo-v2.6-pro | observatory | 125 | 122 | 71.9% (n=125) | 71.9% (n=125) | 40.8% (n=125) | 40.8% (n=125) | 40.0% (n=125) | 40.0% (n=125) |
| mimo-v2.6-pro | theater | 170 | 169 | 73.4% (n=170) | 73.4% (n=170) | 34.1% (n=170) | 34.1% (n=170) | 32.4% (n=170) | 32.4% (n=170) |
| claude-opus-5 | hotel | 110 | 85 | 65.5% (n=110) | 84.7% (n=85) | 49.1% (n=110) | 63.5% (n=85) | 49.1% (n=110) | 63.5% (n=85) |
| claude-opus-5 | manor | 105 | 67 | 47.5% (n=105) | 74.4% (n=67) | 29.5% (n=105) | 46.3% (n=67) | 29.5% (n=105) | 46.3% (n=67) |
| claude-opus-5 | maritime | 90 | 70 | 63.7% (n=90) | 81.9% (n=70) | 33.3% (n=90) | 42.9% (n=70) | 33.3% (n=90) | 42.9% (n=70) |
| claude-opus-5 | observatory | 125 | 72 | 47.1% (n=125) | 81.8% (n=72) | 34.4% (n=125) | 59.7% (n=72) | 34.4% (n=125) | 59.7% (n=72) |
| claude-opus-5 | theater | 170 | 130 | 62.2% (n=170) | 81.3% (n=130) | 38.2% (n=170) | 50.0% (n=130) | 38.2% (n=170) | 50.0% (n=130) |
| grok-4.6 | hotel | 110 | 110 | 89.9% (n=110) | 89.9% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) |
| grok-4.6 | manor | 105 | 105 | 89.3% (n=105) | 89.3% (n=105) | 34.3% (n=105) | 34.3% (n=105) | 34.3% (n=105) | 34.3% (n=105) |
| grok-4.6 | maritime | 90 | 90 | 90.0% (n=90) | 90.0% (n=90) | 43.3% (n=90) | 43.3% (n=90) | 43.3% (n=90) | 43.3% (n=90) |
| grok-4.6 | observatory | 125 | 125 | 89.6% (n=125) | 89.6% (n=125) | 35.2% (n=125) | 35.2% (n=125) | 35.2% (n=125) | 35.2% (n=125) |
| grok-4.6 | theater | 170 | 170 | 88.9% (n=170) | 88.9% (n=170) | 37.6% (n=170) | 37.6% (n=170) | 37.6% (n=170) | 37.6% (n=170) |
| kimi-k2.6 | hotel | 110 | 110 | 90.5% (n=110) | 90.5% (n=110) | 34.5% (n=110) | 34.5% (n=110) | 34.5% (n=110) | 34.5% (n=110) |
| kimi-k2.6 | manor | 105 | 105 | 91.2% (n=105) | 91.2% (n=105) | 39.0% (n=105) | 39.0% (n=105) | 39.0% (n=105) | 39.0% (n=105) |
| kimi-k2.6 | maritime | 90 | 90 | 93.0% (n=90) | 93.0% (n=90) | 40.0% (n=90) | 40.0% (n=90) | 40.0% (n=90) | 40.0% (n=90) |
| kimi-k2.6 | observatory | 125 | 125 | 89.8% (n=125) | 89.8% (n=125) | 37.6% (n=125) | 37.6% (n=125) | 37.6% (n=125) | 37.6% (n=125) |
| kimi-k2.6 | theater | 170 | 169 | 89.0% (n=170) | 89.5% (n=169) | 33.5% (n=170) | 33.7% (n=169) | 33.5% (n=170) | 33.7% (n=169) |
| minimax-m3 | hotel | 110 | 109 | 68.3% (n=110) | 68.3% (n=110) | 37.3% (n=110) | 37.3% (n=110) | 37.3% (n=110) | 37.3% (n=110) |
| minimax-m3 | manor | 105 | 105 | 72.7% (n=105) | 72.7% (n=105) | 38.1% (n=105) | 38.1% (n=105) | 38.1% (n=105) | 38.1% (n=105) |
| minimax-m3 | maritime | 90 | 90 | 63.2% (n=90) | 63.2% (n=90) | 25.6% (n=90) | 25.6% (n=90) | 25.6% (n=90) | 25.6% (n=90) |
| minimax-m3 | observatory | 125 | 123 | 61.9% (n=125) | 61.9% (n=125) | 32.0% (n=125) | 32.0% (n=125) | 32.0% (n=125) | 32.0% (n=125) |
| minimax-m3 | theater | 170 | 170 | 73.5% (n=170) | 73.5% (n=170) | 34.7% (n=170) | 34.7% (n=170) | 34.7% (n=170) | 34.7% (n=170) |
| qwen3.8-27b | hotel | 110 | 110 | 84.0% (n=110) | 84.0% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) | 35.5% (n=110) |
| qwen3.8-27b | manor | 105 | 105 | 85.7% (n=105) | 85.7% (n=105) | 33.3% (n=105) | 33.3% (n=105) | 33.3% (n=105) | 33.3% (n=105) |
| qwen3.8-27b | maritime | 90 | 90 | 83.7% (n=90) | 83.7% (n=90) | 31.1% (n=90) | 31.1% (n=90) | 31.1% (n=90) | 31.1% (n=90) |
| qwen3.8-27b | observatory | 125 | 125 | 82.6% (n=125) | 82.6% (n=125) | 32.0% (n=125) | 32.0% (n=125) | 32.0% (n=125) | 32.0% (n=125) |
| qwen3.8-27b | theater | 170 | 170 | 82.8% (n=170) | 82.8% (n=170) | 35.3% (n=170) | 35.3% (n=170) | 35.3% (n=170) | 35.3% (n=170) |
| kimi-k3 | hotel | 110 | 93 | 70.5% (n=110) | 83.4% (n=93) | 36.4% (n=110) | 43.0% (n=93) | 36.4% (n=110) | 43.0% (n=93) |
| kimi-k3 | manor | 105 | 76 | 63.4% (n=105) | 87.6% (n=76) | 34.3% (n=105) | 47.4% (n=76) | 34.3% (n=105) | 47.4% (n=76) |
| kimi-k3 | maritime | 90 | 61 | 59.6% (n=90) | 87.9% (n=61) | 23.3% (n=90) | 34.4% (n=61) | 23.3% (n=90) | 34.4% (n=61) |
| kimi-k3 | observatory | 125 | 95 | 60.6% (n=125) | 79.7% (n=95) | 32.0% (n=125) | 42.1% (n=95) | 32.0% (n=125) | 42.1% (n=95) |
| kimi-k3 | theater | 170 | 118 | 60.7% (n=170) | 87.5% (n=118) | 29.4% (n=170) | 42.4% (n=118) | 29.4% (n=170) | 42.4% (n=118) |
| glm-5.3-flash | hotel | 110 | 110 | 68.3% (n=110) | 68.3% (n=110) | 25.5% (n=110) | 25.5% (n=110) | 25.5% (n=110) | 25.5% (n=110) |
| glm-5.3-flash | manor | 105 | 105 | 70.8% (n=105) | 70.8% (n=105) | 35.2% (n=105) | 35.2% (n=105) | 35.2% (n=105) | 35.2% (n=105) |
| glm-5.3-flash | maritime | 90 | 89 | 66.5% (n=90) | 67.3% (n=89) | 21.1% (n=90) | 21.3% (n=89) | 21.1% (n=90) | 21.3% (n=89) |
| glm-5.3-flash | observatory | 125 | 125 | 65.0% (n=125) | 65.0% (n=125) | 25.6% (n=125) | 25.6% (n=125) | 25.6% (n=125) | 25.6% (n=125) |
| glm-5.3-flash | theater | 170 | 167 | 64.4% (n=170) | 65.5% (n=167) | 23.5% (n=170) | 24.0% (n=167) | 22.9% (n=170) | 23.4% (n=167) |
| claude-haiku-4.5 | hotel | 110 | 110 | 68.4% (n=110) | 68.4% (n=110) | 25.5% (n=110) | 25.5% (n=110) | 25.5% (n=110) | 25.5% (n=110) |
| claude-haiku-4.5 | manor | 105 | 105 | 75.4% (n=105) | 75.4% (n=105) | 28.6% (n=105) | 28.6% (n=105) | 28.6% (n=105) | 28.6% (n=105) |
| claude-haiku-4.5 | maritime | 90 | 90 | 70.3% (n=90) | 70.3% (n=90) | 23.3% (n=90) | 23.3% (n=90) | 23.3% (n=90) | 23.3% (n=90) |
| claude-haiku-4.5 | observatory | 125 | 125 | 60.7% (n=125) | 60.7% (n=125) | 20.0% (n=125) | 20.0% (n=125) | 20.0% (n=125) | 20.0% (n=125) |
| claude-haiku-4.5 | theater | 170 | 170 | 67.5% (n=170) | 67.5% (n=170) | 22.4% (n=170) | 22.4% (n=170) | 22.4% (n=170) | 22.4% (n=170) |
| glm-4.7-flash | hotel | 110 | 95 | 56.2% (n=110) | 65.1% (n=95) | 18.2% (n=110) | 21.1% (n=95) | 18.2% (n=110) | 21.1% (n=95) |
| glm-4.7-flash | manor | 105 | 102 | 68.4% (n=105) | 70.4% (n=102) | 24.8% (n=105) | 25.5% (n=102) | 24.8% (n=105) | 25.5% (n=102) |
| glm-4.7-flash | maritime | 90 | 84 | 65.6% (n=90) | 70.3% (n=84) | 16.7% (n=90) | 17.9% (n=84) | 16.7% (n=90) | 17.9% (n=84) |
| glm-4.7-flash | observatory | 125 | 120 | 67.1% (n=125) | 69.9% (n=120) | 20.8% (n=125) | 21.7% (n=120) | 20.8% (n=125) | 21.7% (n=120) |
| glm-4.7-flash | theater | 170 | 164 | 60.7% (n=170) | 62.5% (n=165) | 17.6% (n=170) | 18.2% (n=165) | 17.6% (n=170) | 18.2% (n=165) |
| claude-fable-5.1 | hotel | 110 | 1 | 0.8% (n=110) | 90.9% (n=1) | 0.0% (n=110) | 0.0% (n=1) | 0.0% (n=110) | 0.0% (n=1) |
| claude-fable-5.1 | manor | 105 | 3 | 2.8% (n=105) | 99.3% (n=3) | 1.9% (n=105) | 66.7% (n=3) | 1.9% (n=105) | 66.7% (n=3) |
| claude-fable-5.1 | maritime | 90 | 4 | 4.4% (n=90) | 98.1% (n=4) | 3.3% (n=90) | 75.0% (n=4) | 3.3% (n=90) | 75.0% (n=4) |
| claude-fable-5.1 | observatory | 125 | 2 | 1.6% (n=125) | 100.0% (n=2) | 1.6% (n=125) | 100.0% (n=2) | 1.6% (n=125) | 100.0% (n=2) |
| claude-fable-5.1 | theater | 170 | 4 | 2.3% (n=170) | 98.6% (n=4) | 1.8% (n=170) | 75.0% (n=4) | 1.8% (n=170) | 75.0% (n=4) |
| baseline:direct-facts | hotel | 110 | 110 | 74.6% (n=110) | 74.6% (n=110) | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) |
| baseline:direct-facts | manor | 105 | 105 | 73.4% (n=105) | 73.4% (n=105) | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) |
| baseline:direct-facts | maritime | 90 | 90 | 76.6% (n=90) | 76.6% (n=90) | 5.6% (n=90) | 5.6% (n=90) | 5.6% (n=90) | 5.6% (n=90) |
| baseline:direct-facts | observatory | 125 | 125 | 73.6% (n=125) | 73.6% (n=125) | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) |
| baseline:direct-facts | theater | 170 | 170 | 71.8% (n=170) | 71.8% (n=170) | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) |
| baseline:story-copy | hotel | 110 | 110 | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) | 0.0% (n=110) |
| baseline:story-copy | manor | 105 | 105 | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) | 0.0% (n=105) |
| baseline:story-copy | maritime | 90 | 90 | 0.0% (n=90) | 0.0% (n=90) | 0.0% (n=90) | 0.0% (n=90) | 0.0% (n=90) | 0.0% (n=90) |
| baseline:story-copy | observatory | 125 | 125 | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) | 0.0% (n=125) |
| baseline:story-copy | theater | 170 | 170 | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) | 0.0% (n=170) |
| Provider | renderer | Expected | Answered | Fact F1 · all items | Fact F1 · scored only | Exact world · all items | Exact world · scored only | Task success · all items | Task success · scored only |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | template | 600 | 600 | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) | 100.0% (n=600) |
| baseline:forward-chain | template | 600 | 600 | 99.8% (n=600) | 99.8% (n=600) | 98.3% (n=600) | 98.3% (n=600) | 98.3% (n=600) | 98.3% (n=600) |
| gpt-6-astra | template | 600 | 550 | 90.6% (n=600) | 98.8% (n=550) | 80.3% (n=600) | 87.6% (n=550) | 80.3% (n=600) | 87.6% (n=550) |
| gpt-6-sol | template | 600 | 600 | 96.6% (n=600) | 96.6% (n=600) | 74.3% (n=600) | 74.3% (n=600) | 74.2% (n=600) | 74.2% (n=600) |
| gemini-3.7-flash | template | 600 | 600 | 91.3% (n=600) | 91.3% (n=600) | 68.7% (n=600) | 68.7% (n=600) | 68.7% (n=600) | 68.7% (n=600) |
| gpt-5.6-sol | template | 600 | 600 | 91.0% (n=600) | 91.0% (n=600) | 68.5% (n=600) | 68.5% (n=600) | 68.5% (n=600) | 68.5% (n=600) |
| gemini-3.8-flash | template | 600 | 598 | 86.2% (n=600) | 86.5% (n=598) | 59.0% (n=600) | 59.2% (n=598) | 59.0% (n=600) | 59.2% (n=598) |
| glm-5.3 | template | 600 | 563 | 80.1% (n=600) | 85.2% (n=564) | 53.3% (n=600) | 56.7% (n=564) | 53.2% (n=600) | 56.6% (n=564) |
| gpt-5.6-terra | template | 600 | 600 | 81.8% (n=600) | 81.8% (n=600) | 50.7% (n=600) | 50.7% (n=600) | 50.7% (n=600) | 50.7% (n=600) |
| deepseek-4.1-flash | template | 600 | 595 | 78.7% (n=600) | 79.4% (n=595) | 48.7% (n=600) | 49.1% (n=595) | 48.7% (n=600) | 49.1% (n=595) |
| gemini-3.6-flash | template | 600 | 600 | 79.5% (n=600) | 79.5% (n=600) | 43.7% (n=600) | 43.7% (n=600) | 43.7% (n=600) | 43.7% (n=600) |
| deepseek-v4-pro | template | 600 | 598 | 91.5% (n=600) | 91.5% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 42.7% (n=600) |
| gpt-6-luna | template | 600 | 600 | 81.5% (n=600) | 81.5% (n=600) | 42.5% (n=600) | 42.5% (n=600) | 42.3% (n=600) | 42.3% (n=600) |
| claude-sonnet-5 | template | 600 | 445 | 66.9% (n=600) | 90.2% (n=445) | 41.8% (n=600) | 56.4% (n=445) | 41.8% (n=600) | 56.4% (n=445) |
| gpt-5.6-luna | template | 600 | 600 | 81.4% (n=600) | 81.4% (n=600) | 42.7% (n=600) | 42.7% (n=600) | 41.7% (n=600) | 41.7% (n=600) |
| claude-opus-5.5 | template | 600 | 591 | 91.9% (n=600) | 93.3% (n=591) | 40.0% (n=600) | 40.6% (n=591) | 39.8% (n=600) | 40.4% (n=591) |
| claude-fable-5 | template | 600 | 393 | 62.5% (n=600) | 95.5% (n=393) | 39.5% (n=600) | 60.3% (n=393) | 39.5% (n=600) | 60.3% (n=393) |
| mimo-v2.6-flash | template | 600 | 594 | 78.5% (n=600) | 79.3% (n=594) | 40.5% (n=600) | 40.9% (n=594) | 38.8% (n=600) | 39.2% (n=594) |
| mimo-v2.6-pro | template | 600 | 593 | 76.6% (n=600) | 76.6% (n=600) | 39.2% (n=600) | 39.2% (n=600) | 38.0% (n=600) | 38.0% (n=600) |
| claude-opus-5 | template | 600 | 424 | 57.3% (n=600) | 81.1% (n=424) | 37.2% (n=600) | 52.6% (n=424) | 37.2% (n=600) | 52.6% (n=424) |
| grok-4.6 | template | 600 | 600 | 89.5% (n=600) | 89.5% (n=600) | 37.0% (n=600) | 37.0% (n=600) | 37.0% (n=600) | 37.0% (n=600) |
| kimi-k2.6 | template | 600 | 599 | 90.4% (n=600) | 90.6% (n=599) | 36.5% (n=600) | 36.6% (n=599) | 36.5% (n=600) | 36.6% (n=599) |
| minimax-m3 | template | 600 | 597 | 68.5% (n=600) | 68.5% (n=600) | 33.8% (n=600) | 33.8% (n=600) | 33.8% (n=600) | 33.8% (n=600) |
| qwen3.8-27b | template | 600 | 600 | 83.6% (n=600) | 83.6% (n=600) | 33.7% (n=600) | 33.7% (n=600) | 33.7% (n=600) | 33.7% (n=600) |
| kimi-k3 | template | 600 | 443 | 62.8% (n=600) | 85.0% (n=443) | 31.2% (n=600) | 42.2% (n=443) | 31.2% (n=600) | 42.2% (n=443) |
| glm-5.3-flash | template | 600 | 596 | 66.7% (n=600) | 67.1% (n=596) | 26.0% (n=600) | 26.2% (n=596) | 25.8% (n=600) | 26.0% (n=596) |
| claude-haiku-4.5 | template | 600 | 600 | 68.0% (n=600) | 68.0% (n=600) | 23.7% (n=600) | 23.7% (n=600) | 23.7% (n=600) | 23.7% (n=600) |
| glm-4.7-flash | template | 600 | 565 | 63.3% (n=600) | 67.1% (n=566) | 19.5% (n=600) | 20.7% (n=566) | 19.5% (n=600) | 20.7% (n=566) |
| claude-fable-5.1 | template | 600 | 14 | 2.3% (n=600) | 98.2% (n=14) | 1.7% (n=600) | 71.4% (n=14) | 1.7% (n=600) | 71.4% (n=14) |
| baseline:direct-facts | template | 600 | 600 | 73.7% (n=600) | 73.7% (n=600) | 0.8% (n=600) | 0.8% (n=600) | 0.8% (n=600) | 0.8% (n=600) |
| baseline:story-copy | template | 600 | 600 | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) | 0.0% (n=600) |
Model pairs · fact F1 difference (percentage points, left − right)
Differences are left minus right, in percentage points. Complete scored pairs only; condition pairs require all configured realizations on both sides. Missing pairs are excluded.
{
"caution": "Descriptive intervals, not multiplicity-adjusted; few families give unstable uncertainty estimates.",
"confidence": 0.95,
"method": "family-cluster percentile bootstrap",
"resamples": 2000,
"seed": 0,
"strata_intervals": false
}| Provider | Left | Right | Metric | Difference (pp) | 95% CI (pp) | Expected pairs | Complete pairs | Missing pairs | Families |
|---|---|---|---|---|---|---|---|---|---|
| All providers | baseline:constraint-solver | baseline:direct-facts | fact_f1 | 26.3% | 24.5% to 28.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:direct-facts | exact_world | 99.2% | 97.5% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:direct-facts | task_success | 99.2% | 97.5% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:direct-facts | decision_correct | 65.8% | 57.5% to 74.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:forward-chain | fact_f1 | 0.2% | 0.0% to 0.4% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:forward-chain | exact_world | 1.7% | 0.0% to 4.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:forward-chain | task_success | 1.7% | 0.0% to 4.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:forward-chain | decision_correct | 0.8% | 0.0% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:story-copy | fact_f1 | 100.0% | 100.0% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:story-copy | exact_world | 100.0% | 100.0% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:story-copy | task_success | 100.0% | 100.0% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | baseline:story-copy | decision_correct | 100.0% | 100.0% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | claude-fable-5 | fact_f1 | 4.5% | 3.6% to 5.5% | 600 | 393 | 207 | 120 |
| All providers | baseline:constraint-solver | claude-fable-5 | exact_world | 39.7% | 34.0% to 45.2% | 600 | 393 | 207 | 120 |
| All providers | baseline:constraint-solver | claude-fable-5 | task_success | 39.7% | 34.0% to 45.2% | 600 | 393 | 207 | 120 |
| All providers | baseline:constraint-solver | claude-fable-5 | decision_correct | 1.3% | 0.2% to 2.7% | 600 | 393 | 207 | 120 |
| All providers | baseline:constraint-solver | claude-fable-5.1 | fact_f1 | 1.8% | 0.3% to 3.5% | 600 | 14 | 586 | 13 |
| All providers | baseline:constraint-solver | claude-fable-5.1 | exact_world | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:constraint-solver | claude-fable-5.1 | task_success | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:constraint-solver | claude-fable-5.1 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:constraint-solver | claude-haiku-4.5 | fact_f1 | 32.0% | 29.1% to 34.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | claude-haiku-4.5 | exact_world | 76.3% | 74.3% to 78.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | claude-haiku-4.5 | task_success | 76.3% | 74.3% to 78.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | claude-haiku-4.5 | decision_correct | 4.5% | 2.7% to 6.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | claude-opus-5 | fact_f1 | 18.9% | 15.7% to 22.1% | 600 | 424 | 176 | 119 |
| All providers | baseline:constraint-solver | claude-opus-5 | exact_world | 47.4% | 41.7% to 52.8% | 600 | 424 | 176 | 119 |
| All providers | baseline:constraint-solver | claude-opus-5 | task_success | 47.4% | 41.7% to 52.8% | 600 | 424 | 176 | 119 |
| All providers | baseline:constraint-solver | claude-opus-5 | decision_correct | 8.5% | 5.9% to 11.4% | 600 | 424 | 176 | 119 |
| All providers | baseline:constraint-solver | claude-opus-5.5 | fact_f1 | 6.7% | 5.9% to 7.6% | 600 | 591 | 9 | 120 |
| All providers | baseline:constraint-solver | claude-opus-5.5 | exact_world | 59.4% | 55.7% to 62.8% | 600 | 591 | 9 | 120 |
| All providers | baseline:constraint-solver | claude-opus-5.5 | task_success | 59.6% | 55.8% to 63.0% | 600 | 591 | 9 | 120 |
| All providers | baseline:constraint-solver | claude-opus-5.5 | decision_correct | 4.7% | 2.9% to 6.9% | 600 | 591 | 9 | 120 |
| All providers | baseline:constraint-solver | claude-sonnet-5 | fact_f1 | 9.8% | 7.6% to 11.9% | 600 | 445 | 155 | 120 |
| All providers | baseline:constraint-solver | claude-sonnet-5 | exact_world | 43.6% | 39.1% to 48.7% | 600 | 445 | 155 | 120 |
| All providers | baseline:constraint-solver | claude-sonnet-5 | task_success | 43.6% | 39.1% to 48.7% | 600 | 445 | 155 | 120 |
| All providers | baseline:constraint-solver | claude-sonnet-5 | decision_correct | 4.0% | 2.3% to 5.9% | 600 | 445 | 155 | 120 |
| All providers | baseline:constraint-solver | deepseek-4.1-flash | fact_f1 | 20.6% | 18.3% to 22.9% | 600 | 595 | 5 | 120 |
| All providers | baseline:constraint-solver | deepseek-4.1-flash | exact_world | 50.9% | 46.9% to 55.1% | 600 | 595 | 5 | 120 |
| All providers | baseline:constraint-solver | deepseek-4.1-flash | task_success | 50.9% | 46.9% to 55.1% | 600 | 595 | 5 | 120 |
| All providers | baseline:constraint-solver | deepseek-4.1-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 595 | 5 | 120 |
| All providers | baseline:constraint-solver | deepseek-v4-pro | fact_f1 | 8.5% | 7.3% to 9.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | deepseek-v4-pro | exact_world | 57.3% | 53.2% to 61.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | deepseek-v4-pro | task_success | 57.3% | 53.2% to 61.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | deepseek-v4-pro | decision_correct | 4.7% | 2.8% to 6.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.6-flash | fact_f1 | 20.5% | 17.9% to 23.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.6-flash | exact_world | 56.3% | 52.2% to 60.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.6-flash | task_success | 56.3% | 52.2% to 60.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.6-flash | decision_correct | 1.0% | 0.2% to 2.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.7-flash | fact_f1 | 8.7% | 7.0% to 10.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.7-flash | exact_world | 31.3% | 27.2% to 35.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.7-flash | task_success | 31.3% | 27.2% to 35.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.7-flash | decision_correct | 0.5% | 0.0% to 1.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gemini-3.8-flash | fact_f1 | 13.5% | 11.3% to 15.6% | 600 | 598 | 2 | 120 |
| All providers | baseline:constraint-solver | gemini-3.8-flash | exact_world | 40.8% | 36.4% to 45.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:constraint-solver | gemini-3.8-flash | task_success | 40.8% | 36.4% to 45.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:constraint-solver | gemini-3.8-flash | decision_correct | 0.3% | 0.0% to 0.8% | 600 | 598 | 2 | 120 |
| All providers | baseline:constraint-solver | glm-4.7-flash | fact_f1 | 32.9% | 30.1% to 35.6% | 600 | 566 | 34 | 120 |
| All providers | baseline:constraint-solver | glm-4.7-flash | exact_world | 79.3% | 77.0% to 81.5% | 600 | 566 | 34 | 120 |
| All providers | baseline:constraint-solver | glm-4.7-flash | task_success | 79.3% | 77.0% to 81.5% | 600 | 566 | 34 | 120 |
| All providers | baseline:constraint-solver | glm-4.7-flash | decision_correct | 17.5% | 14.2% to 20.9% | 600 | 566 | 34 | 120 |
| All providers | baseline:constraint-solver | glm-5.3 | fact_f1 | 14.8% | 11.9% to 17.8% | 600 | 564 | 36 | 120 |
| All providers | baseline:constraint-solver | glm-5.3 | exact_world | 43.3% | 38.5% to 47.8% | 600 | 564 | 36 | 120 |
| All providers | baseline:constraint-solver | glm-5.3 | task_success | 43.4% | 38.6% to 47.9% | 600 | 564 | 36 | 120 |
| All providers | baseline:constraint-solver | glm-5.3 | decision_correct | 3.7% | 2.3% to 5.4% | 600 | 564 | 36 | 120 |
| All providers | baseline:constraint-solver | glm-5.3-flash | fact_f1 | 32.9% | 29.5% to 36.1% | 600 | 596 | 4 | 120 |
| All providers | baseline:constraint-solver | glm-5.3-flash | exact_world | 73.8% | 70.6% to 76.8% | 600 | 596 | 4 | 120 |
| All providers | baseline:constraint-solver | glm-5.3-flash | task_success | 74.0% | 70.7% to 77.0% | 600 | 596 | 4 | 120 |
| All providers | baseline:constraint-solver | glm-5.3-flash | decision_correct | 1.7% | 0.8% to 2.7% | 600 | 596 | 4 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-luna | fact_f1 | 18.6% | 16.6% to 20.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-luna | exact_world | 57.3% | 54.0% to 60.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-luna | task_success | 58.3% | 55.0% to 61.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-luna | decision_correct | 2.8% | 1.5% to 4.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-sol | fact_f1 | 9.0% | 7.2% to 10.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-sol | exact_world | 31.5% | 27.2% to 36.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-sol | task_success | 31.5% | 27.2% to 36.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-sol | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-terra | fact_f1 | 18.2% | 16.2% to 20.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-terra | exact_world | 49.3% | 45.7% to 53.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-terra | task_success | 49.3% | 45.7% to 53.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-5.6-terra | decision_correct | 0.2% | 0.0% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-astra | fact_f1 | 1.2% | 0.8% to 1.7% | 600 | 550 | 50 | 115 |
| All providers | baseline:constraint-solver | gpt-6-astra | exact_world | 12.4% | 8.7% to 16.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:constraint-solver | gpt-6-astra | task_success | 12.4% | 8.7% to 16.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:constraint-solver | gpt-6-astra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 550 | 50 | 115 |
| All providers | baseline:constraint-solver | gpt-6-luna | fact_f1 | 18.5% | 16.3% to 20.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-luna | exact_world | 57.5% | 53.7% to 61.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-luna | task_success | 57.7% | 54.0% to 61.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-luna | decision_correct | 1.7% | 0.7% to 2.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-sol | fact_f1 | 3.4% | 2.6% to 4.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-sol | exact_world | 25.7% | 21.5% to 29.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-sol | task_success | 25.8% | 21.7% to 30.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | gpt-6-sol | decision_correct | 1.5% | 0.7% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | grok-4.6 | fact_f1 | 10.5% | 9.3% to 11.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | grok-4.6 | exact_world | 63.0% | 59.2% to 66.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | grok-4.6 | task_success | 63.0% | 59.2% to 66.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | grok-4.6 | decision_correct | 3.3% | 1.3% to 5.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | kimi-k2.6 | fact_f1 | 9.4% | 8.3% to 10.7% | 600 | 599 | 1 | 120 |
| All providers | baseline:constraint-solver | kimi-k2.6 | exact_world | 63.4% | 59.9% to 66.9% | 600 | 599 | 1 | 120 |
| All providers | baseline:constraint-solver | kimi-k2.6 | task_success | 63.4% | 59.9% to 66.9% | 600 | 599 | 1 | 120 |
| All providers | baseline:constraint-solver | kimi-k2.6 | decision_correct | 2.2% | 1.0% to 3.5% | 600 | 599 | 1 | 120 |
| All providers | baseline:constraint-solver | kimi-k3 | fact_f1 | 15.0% | 12.2% to 17.9% | 600 | 443 | 157 | 96 |
| All providers | baseline:constraint-solver | kimi-k3 | exact_world | 57.8% | 53.0% to 62.4% | 600 | 443 | 157 | 96 |
| All providers | baseline:constraint-solver | kimi-k3 | task_success | 57.8% | 53.0% to 62.4% | 600 | 443 | 157 | 96 |
| All providers | baseline:constraint-solver | kimi-k3 | decision_correct | 0.2% | 0.0% to 0.7% | 600 | 443 | 157 | 96 |
| All providers | baseline:constraint-solver | mimo-v2.6-flash | fact_f1 | 20.7% | 18.3% to 22.9% | 600 | 594 | 6 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-flash | exact_world | 59.1% | 55.6% to 62.3% | 600 | 594 | 6 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-flash | task_success | 60.8% | 57.2% to 64.0% | 600 | 594 | 6 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-flash | decision_correct | 10.1% | 7.5% to 12.9% | 600 | 594 | 6 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-pro | fact_f1 | 23.4% | 20.9% to 26.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-pro | exact_world | 60.8% | 57.3% to 64.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-pro | task_success | 62.0% | 58.5% to 65.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | mimo-v2.6-pro | decision_correct | 11.5% | 8.8% to 14.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | minimax-m3 | fact_f1 | 31.5% | 28.3% to 34.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | minimax-m3 | exact_world | 66.2% | 62.8% to 69.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | minimax-m3 | task_success | 66.2% | 62.8% to 69.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | minimax-m3 | decision_correct | 3.3% | 1.8% to 5.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | qwen3.8-27b | fact_f1 | 16.4% | 14.3% to 18.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | qwen3.8-27b | exact_world | 66.3% | 62.5% to 69.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | qwen3.8-27b | task_success | 66.3% | 62.5% to 69.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:constraint-solver | qwen3.8-27b | decision_correct | 3.0% | 1.5% to 4.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:forward-chain | fact_f1 | -26.2% | -28.0% to -24.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:forward-chain | exact_world | -97.5% | -100.0% to -94.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:forward-chain | task_success | -97.5% | -100.0% to -94.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:forward-chain | decision_correct | -65.0% | -73.3% to -56.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:story-copy | fact_f1 | 73.7% | 71.8% to 75.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:story-copy | exact_world | 0.8% | 0.0% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:story-copy | task_success | 0.8% | 0.0% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | baseline:story-copy | decision_correct | 34.2% | 25.8% to 42.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | claude-fable-5 | fact_f1 | -21.4% | -23.6% to -19.2% | 600 | 393 | 207 | 120 |
| All providers | baseline:direct-facts | claude-fable-5 | exact_world | -59.3% | -65.2% to -53.2% | 600 | 393 | 207 | 120 |
| All providers | baseline:direct-facts | claude-fable-5 | task_success | -59.3% | -65.2% to -53.2% | 600 | 393 | 207 | 120 |
| All providers | baseline:direct-facts | claude-fable-5 | decision_correct | -65.6% | -74.9% to -56.1% | 600 | 393 | 207 | 120 |
| All providers | baseline:direct-facts | claude-fable-5.1 | fact_f1 | -27.0% | -34.2% to -21.1% | 600 | 14 | 586 | 13 |
| All providers | baseline:direct-facts | claude-fable-5.1 | exact_world | -71.4% | -92.3% to -50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:direct-facts | claude-fable-5.1 | task_success | -71.4% | -92.3% to -50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:direct-facts | claude-fable-5.1 | decision_correct | -64.3% | -92.3% to -37.5% | 600 | 14 | 586 | 13 |
| All providers | baseline:direct-facts | claude-haiku-4.5 | fact_f1 | 5.6% | 2.8% to 8.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | claude-haiku-4.5 | exact_world | -22.8% | -24.8% to -20.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | claude-haiku-4.5 | task_success | -22.8% | -24.8% to -20.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | claude-haiku-4.5 | decision_correct | -61.3% | -69.5% to -53.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | claude-opus-5 | fact_f1 | -7.0% | -10.4% to -3.9% | 600 | 424 | 176 | 119 |
| All providers | baseline:direct-facts | claude-opus-5 | exact_world | -51.7% | -57.3% to -46.1% | 600 | 424 | 176 | 119 |
| All providers | baseline:direct-facts | claude-opus-5 | task_success | -51.7% | -57.3% to -46.1% | 600 | 424 | 176 | 119 |
| All providers | baseline:direct-facts | claude-opus-5 | decision_correct | -58.5% | -67.5% to -49.4% | 600 | 424 | 176 | 119 |
| All providers | baseline:direct-facts | claude-opus-5.5 | fact_f1 | -19.5% | -21.7% to -17.5% | 600 | 591 | 9 | 120 |
| All providers | baseline:direct-facts | claude-opus-5.5 | exact_world | -39.8% | -44.0% to -35.4% | 600 | 591 | 9 | 120 |
| All providers | baseline:direct-facts | claude-opus-5.5 | task_success | -39.6% | -43.8% to -35.3% | 600 | 591 | 9 | 120 |
| All providers | baseline:direct-facts | claude-opus-5.5 | decision_correct | -60.9% | -69.0% to -52.9% | 600 | 591 | 9 | 120 |
| All providers | baseline:direct-facts | claude-sonnet-5 | fact_f1 | -16.6% | -19.4% to -13.9% | 600 | 445 | 155 | 120 |
| All providers | baseline:direct-facts | claude-sonnet-5 | exact_world | -55.3% | -60.3% to -49.7% | 600 | 445 | 155 | 120 |
| All providers | baseline:direct-facts | claude-sonnet-5 | task_success | -55.3% | -60.3% to -49.7% | 600 | 445 | 155 | 120 |
| All providers | baseline:direct-facts | claude-sonnet-5 | decision_correct | -65.2% | -74.0% to -56.8% | 600 | 445 | 155 | 120 |
| All providers | baseline:direct-facts | deepseek-4.1-flash | fact_f1 | -5.6% | -8.3% to -2.8% | 600 | 595 | 5 | 120 |
| All providers | baseline:direct-facts | deepseek-4.1-flash | exact_world | -48.2% | -52.5% to -43.6% | 600 | 595 | 5 | 120 |
| All providers | baseline:direct-facts | deepseek-4.1-flash | task_success | -48.2% | -52.5% to -43.6% | 600 | 595 | 5 | 120 |
| All providers | baseline:direct-facts | deepseek-4.1-flash | decision_correct | -65.5% | -73.9% to -57.2% | 600 | 595 | 5 | 120 |
| All providers | baseline:direct-facts | deepseek-v4-pro | fact_f1 | -17.8% | -19.8% to -15.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | deepseek-v4-pro | exact_world | -41.8% | -46.3% to -37.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | deepseek-v4-pro | task_success | -41.8% | -46.3% to -37.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | deepseek-v4-pro | decision_correct | -61.2% | -69.2% to -52.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.6-flash | fact_f1 | -5.8% | -8.7% to -2.9% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.6-flash | exact_world | -42.8% | -47.0% to -38.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.6-flash | task_success | -42.8% | -47.0% to -38.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.6-flash | decision_correct | -64.8% | -73.0% to -56.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.7-flash | fact_f1 | -17.6% | -19.9% to -15.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.7-flash | exact_world | -67.8% | -72.2% to -63.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.7-flash | task_success | -67.8% | -72.2% to -63.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.7-flash | decision_correct | -65.3% | -73.5% to -57.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gemini-3.8-flash | fact_f1 | -12.8% | -15.3% to -10.5% | 600 | 598 | 2 | 120 |
| All providers | baseline:direct-facts | gemini-3.8-flash | exact_world | -58.4% | -62.8% to -54.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:direct-facts | gemini-3.8-flash | task_success | -58.4% | -62.8% to -54.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:direct-facts | gemini-3.8-flash | decision_correct | -65.4% | -73.6% to -57.1% | 600 | 598 | 2 | 120 |
| All providers | baseline:direct-facts | glm-4.7-flash | fact_f1 | 6.9% | 3.7% to 10.0% | 600 | 566 | 34 | 120 |
| All providers | baseline:direct-facts | glm-4.7-flash | exact_world | -19.8% | -22.5% to -16.9% | 600 | 566 | 34 | 120 |
| All providers | baseline:direct-facts | glm-4.7-flash | task_success | -19.8% | -22.5% to -16.9% | 600 | 566 | 34 | 120 |
| All providers | baseline:direct-facts | glm-4.7-flash | decision_correct | -46.5% | -55.5% to -37.3% | 600 | 566 | 34 | 120 |
| All providers | baseline:direct-facts | glm-5.3 | fact_f1 | -11.4% | -14.7% to -8.1% | 600 | 564 | 36 | 120 |
| All providers | baseline:direct-facts | glm-5.3 | exact_world | -56.0% | -61.1% to -51.4% | 600 | 564 | 36 | 120 |
| All providers | baseline:direct-facts | glm-5.3 | task_success | -55.9% | -61.0% to -51.3% | 600 | 564 | 36 | 120 |
| All providers | baseline:direct-facts | glm-5.3 | decision_correct | -62.2% | -70.7% to -53.7% | 600 | 564 | 36 | 120 |
| All providers | baseline:direct-facts | glm-5.3-flash | fact_f1 | 6.6% | 3.0% to 10.0% | 600 | 596 | 4 | 120 |
| All providers | baseline:direct-facts | glm-5.3-flash | exact_world | -25.5% | -29.1% to -22.1% | 600 | 596 | 4 | 120 |
| All providers | baseline:direct-facts | glm-5.3-flash | task_success | -25.3% | -28.9% to -22.0% | 600 | 596 | 4 | 120 |
| All providers | baseline:direct-facts | glm-5.3-flash | decision_correct | -64.3% | -72.7% to -55.8% | 600 | 596 | 4 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-luna | fact_f1 | -7.7% | -10.3% to -5.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-luna | exact_world | -41.8% | -45.7% to -38.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-luna | task_success | -40.8% | -44.7% to -37.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-luna | decision_correct | -63.0% | -71.2% to -54.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-sol | fact_f1 | -17.3% | -19.9% to -14.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-sol | exact_world | -67.7% | -72.5% to -62.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-sol | task_success | -67.7% | -72.5% to -62.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-sol | decision_correct | -65.8% | -74.2% to -57.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-terra | fact_f1 | -8.1% | -11.0% to -5.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-terra | exact_world | -49.8% | -54.0% to -45.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-terra | task_success | -49.8% | -54.0% to -45.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-5.6-terra | decision_correct | -65.7% | -74.0% to -57.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-astra | fact_f1 | -24.6% | -26.7% to -22.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:direct-facts | gpt-6-astra | exact_world | -86.7% | -91.2% to -81.7% | 600 | 550 | 50 | 115 |
| All providers | baseline:direct-facts | gpt-6-astra | task_success | -86.7% | -91.2% to -81.7% | 600 | 550 | 50 | 115 |
| All providers | baseline:direct-facts | gpt-6-astra | decision_correct | -66.0% | -74.7% to -57.1% | 600 | 550 | 50 | 115 |
| All providers | baseline:direct-facts | gpt-6-luna | fact_f1 | -7.8% | -10.7% to -5.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-luna | exact_world | -41.7% | -45.8% to -37.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-luna | task_success | -41.5% | -45.5% to -37.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-luna | decision_correct | -64.2% | -72.3% to -55.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-sol | fact_f1 | -23.0% | -25.2% to -20.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-sol | exact_world | -73.5% | -78.0% to -68.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-sol | task_success | -73.3% | -77.8% to -68.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | gpt-6-sol | decision_correct | -64.3% | -72.7% to -56.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | grok-4.6 | fact_f1 | -15.8% | -18.2% to -13.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | grok-4.6 | exact_world | -36.2% | -40.0% to -32.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | grok-4.6 | task_success | -36.2% | -40.0% to -32.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | grok-4.6 | decision_correct | -62.5% | -70.8% to -53.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | kimi-k2.6 | fact_f1 | -16.9% | -18.9% to -14.8% | 600 | 599 | 1 | 120 |
| All providers | baseline:direct-facts | kimi-k2.6 | exact_world | -35.7% | -39.3% to -32.2% | 600 | 599 | 1 | 120 |
| All providers | baseline:direct-facts | kimi-k2.6 | task_success | -35.7% | -39.3% to -32.2% | 600 | 599 | 1 | 120 |
| All providers | baseline:direct-facts | kimi-k2.6 | decision_correct | -63.8% | -72.2% to -55.2% | 600 | 599 | 1 | 120 |
| All providers | baseline:direct-facts | kimi-k3 | fact_f1 | -10.2% | -13.4% to -7.1% | 600 | 443 | 157 | 96 |
| All providers | baseline:direct-facts | kimi-k3 | exact_world | -41.1% | -46.3% to -36.0% | 600 | 443 | 157 | 96 |
| All providers | baseline:direct-facts | kimi-k3 | task_success | -41.1% | -46.3% to -36.0% | 600 | 443 | 157 | 96 |
| All providers | baseline:direct-facts | kimi-k3 | decision_correct | -64.3% | -74.2% to -54.3% | 600 | 443 | 157 | 96 |
| All providers | baseline:direct-facts | mimo-v2.6-flash | fact_f1 | -5.6% | -8.4% to -3.0% | 600 | 594 | 6 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-flash | exact_world | -40.1% | -43.8% to -36.4% | 600 | 594 | 6 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-flash | task_success | -38.4% | -42.1% to -34.8% | 600 | 594 | 6 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-flash | decision_correct | -55.4% | -63.9% to -46.3% | 600 | 594 | 6 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-pro | fact_f1 | -2.9% | -5.9% to 0.1% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-pro | exact_world | -38.3% | -42.0% to -34.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-pro | task_success | -37.2% | -41.0% to -33.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | mimo-v2.6-pro | decision_correct | -54.3% | -63.5% to -45.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | minimax-m3 | fact_f1 | 5.2% | 1.9% to 8.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | minimax-m3 | exact_world | -33.0% | -36.5% to -29.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | minimax-m3 | task_success | -33.0% | -36.5% to -29.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | minimax-m3 | decision_correct | -62.5% | -71.0% to -54.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | qwen3.8-27b | fact_f1 | -9.9% | -12.3% to -7.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | qwen3.8-27b | exact_world | -32.8% | -37.0% to -28.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | qwen3.8-27b | task_success | -32.8% | -37.0% to -28.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:direct-facts | qwen3.8-27b | decision_correct | -62.8% | -71.0% to -54.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | baseline:story-copy | fact_f1 | 99.8% | 99.6% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | baseline:story-copy | exact_world | 98.3% | 95.8% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | baseline:story-copy | task_success | 98.3% | 95.8% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | baseline:story-copy | decision_correct | 99.2% | 97.5% to 100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | claude-fable-5 | fact_f1 | 4.4% | 3.5% to 5.4% | 600 | 393 | 207 | 120 |
| All providers | baseline:forward-chain | claude-fable-5 | exact_world | 38.7% | 32.9% to 44.3% | 600 | 393 | 207 | 120 |
| All providers | baseline:forward-chain | claude-fable-5 | task_success | 38.7% | 32.9% to 44.3% | 600 | 393 | 207 | 120 |
| All providers | baseline:forward-chain | claude-fable-5 | decision_correct | 0.8% | -0.8% to 2.5% | 600 | 393 | 207 | 120 |
| All providers | baseline:forward-chain | claude-fable-5.1 | fact_f1 | 1.8% | 0.3% to 3.5% | 600 | 14 | 586 | 13 |
| All providers | baseline:forward-chain | claude-fable-5.1 | exact_world | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:forward-chain | claude-fable-5.1 | task_success | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:forward-chain | claude-fable-5.1 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:forward-chain | claude-haiku-4.5 | fact_f1 | 31.8% | 28.9% to 34.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | claude-haiku-4.5 | exact_world | 74.7% | 71.7% to 77.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | claude-haiku-4.5 | task_success | 74.7% | 71.7% to 77.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | claude-haiku-4.5 | decision_correct | 3.7% | 2.2% to 5.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | claude-opus-5 | fact_f1 | 18.8% | 15.6% to 21.9% | 600 | 424 | 176 | 119 |
| All providers | baseline:forward-chain | claude-opus-5 | exact_world | 45.5% | 38.7% to 51.8% | 600 | 424 | 176 | 119 |
| All providers | baseline:forward-chain | claude-opus-5 | task_success | 45.5% | 38.7% to 51.8% | 600 | 424 | 176 | 119 |
| All providers | baseline:forward-chain | claude-opus-5 | decision_correct | 7.5% | 4.1% to 10.7% | 600 | 424 | 176 | 119 |
| All providers | baseline:forward-chain | claude-opus-5.5 | fact_f1 | 6.6% | 5.7% to 7.5% | 600 | 591 | 9 | 120 |
| All providers | baseline:forward-chain | claude-opus-5.5 | exact_world | 57.7% | 53.2% to 61.9% | 600 | 591 | 9 | 120 |
| All providers | baseline:forward-chain | claude-opus-5.5 | task_success | 57.9% | 53.4% to 62.0% | 600 | 591 | 9 | 120 |
| All providers | baseline:forward-chain | claude-opus-5.5 | decision_correct | 3.9% | 1.3% to 6.1% | 600 | 591 | 9 | 120 |
| All providers | baseline:forward-chain | claude-sonnet-5 | fact_f1 | 9.6% | 7.5% to 11.8% | 600 | 445 | 155 | 120 |
| All providers | baseline:forward-chain | claude-sonnet-5 | exact_world | 41.6% | 36.4% to 46.9% | 600 | 445 | 155 | 120 |
| All providers | baseline:forward-chain | claude-sonnet-5 | task_success | 41.6% | 36.4% to 46.9% | 600 | 445 | 155 | 120 |
| All providers | baseline:forward-chain | claude-sonnet-5 | decision_correct | 2.9% | 0.7% to 5.0% | 600 | 445 | 155 | 120 |
| All providers | baseline:forward-chain | deepseek-4.1-flash | fact_f1 | 20.5% | 18.1% to 22.8% | 600 | 595 | 5 | 120 |
| All providers | baseline:forward-chain | deepseek-4.1-flash | exact_world | 49.4% | 44.6% to 54.1% | 600 | 595 | 5 | 120 |
| All providers | baseline:forward-chain | deepseek-4.1-flash | task_success | 49.4% | 44.6% to 54.1% | 600 | 595 | 5 | 120 |
| All providers | baseline:forward-chain | deepseek-4.1-flash | decision_correct | -0.7% | -2.0% to 0.0% | 600 | 595 | 5 | 120 |
| All providers | baseline:forward-chain | deepseek-v4-pro | fact_f1 | 8.4% | 7.1% to 9.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | deepseek-v4-pro | exact_world | 55.7% | 50.5% to 60.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | deepseek-v4-pro | task_success | 55.7% | 50.5% to 60.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | deepseek-v4-pro | decision_correct | 3.8% | 1.5% to 6.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.6-flash | fact_f1 | 20.4% | 17.7% to 23.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.6-flash | exact_world | 54.7% | 49.8% to 59.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.6-flash | task_success | 54.7% | 49.8% to 59.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.6-flash | decision_correct | 0.2% | -0.8% to 1.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.7-flash | fact_f1 | 8.5% | 6.9% to 10.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.7-flash | exact_world | 29.7% | 25.5% to 33.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.7-flash | task_success | 29.7% | 25.5% to 33.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.7-flash | decision_correct | -0.3% | -1.5% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gemini-3.8-flash | fact_f1 | 13.3% | 11.2% to 15.4% | 600 | 598 | 2 | 120 |
| All providers | baseline:forward-chain | gemini-3.8-flash | exact_world | 39.1% | 34.2% to 43.7% | 600 | 598 | 2 | 120 |
| All providers | baseline:forward-chain | gemini-3.8-flash | task_success | 39.1% | 34.2% to 43.7% | 600 | 598 | 2 | 120 |
| All providers | baseline:forward-chain | gemini-3.8-flash | decision_correct | -0.5% | -2.0% to 0.5% | 600 | 598 | 2 | 120 |
| All providers | baseline:forward-chain | glm-4.7-flash | fact_f1 | 32.8% | 29.9% to 35.5% | 600 | 566 | 34 | 120 |
| All providers | baseline:forward-chain | glm-4.7-flash | exact_world | 78.3% | 75.3% to 81.0% | 600 | 566 | 34 | 120 |
| All providers | baseline:forward-chain | glm-4.7-flash | task_success | 78.3% | 75.3% to 81.0% | 600 | 566 | 34 | 120 |
| All providers | baseline:forward-chain | glm-4.7-flash | decision_correct | 17.3% | 14.0% to 20.7% | 600 | 566 | 34 | 120 |
| All providers | baseline:forward-chain | glm-5.3 | fact_f1 | 14.7% | 11.7% to 17.7% | 600 | 564 | 36 | 120 |
| All providers | baseline:forward-chain | glm-5.3 | exact_world | 41.8% | 36.5% to 46.7% | 600 | 564 | 36 | 120 |
| All providers | baseline:forward-chain | glm-5.3 | task_success | 42.0% | 36.8% to 46.9% | 600 | 564 | 36 | 120 |
| All providers | baseline:forward-chain | glm-5.3 | decision_correct | 3.0% | 0.7% to 5.0% | 600 | 564 | 36 | 120 |
| All providers | baseline:forward-chain | glm-5.3-flash | fact_f1 | 32.7% | 29.4% to 36.0% | 600 | 596 | 4 | 120 |
| All providers | baseline:forward-chain | glm-5.3-flash | exact_world | 72.1% | 68.3% to 75.7% | 600 | 596 | 4 | 120 |
| All providers | baseline:forward-chain | glm-5.3-flash | task_success | 72.3% | 68.4% to 75.9% | 600 | 596 | 4 | 120 |
| All providers | baseline:forward-chain | glm-5.3-flash | decision_correct | 0.8% | -1.4% to 2.5% | 600 | 596 | 4 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-luna | fact_f1 | 18.5% | 16.4% to 20.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-luna | exact_world | 55.7% | 51.2% to 59.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-luna | task_success | 56.7% | 52.2% to 60.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-luna | decision_correct | 2.0% | -0.0% to 3.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-sol | fact_f1 | 8.8% | 7.0% to 10.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-sol | exact_world | 29.8% | 24.7% to 35.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-sol | task_success | 29.8% | 24.7% to 35.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-sol | decision_correct | -0.8% | -2.5% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-terra | fact_f1 | 18.1% | 16.0% to 20.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-terra | exact_world | 47.7% | 43.0% to 51.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-terra | task_success | 47.7% | 43.0% to 51.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-5.6-terra | decision_correct | -0.7% | -2.0% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-astra | fact_f1 | 1.1% | 0.5% to 1.6% | 600 | 550 | 50 | 115 |
| All providers | baseline:forward-chain | gpt-6-astra | exact_world | 10.7% | 6.2% to 15.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:forward-chain | gpt-6-astra | task_success | 10.7% | 6.2% to 15.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:forward-chain | gpt-6-astra | decision_correct | -0.7% | -2.2% to 0.0% | 600 | 550 | 50 | 115 |
| All providers | baseline:forward-chain | gpt-6-luna | fact_f1 | 18.4% | 16.2% to 20.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-luna | exact_world | 55.8% | 51.7% to 59.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-luna | task_success | 56.0% | 51.7% to 59.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-luna | decision_correct | 0.8% | -1.3% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-sol | fact_f1 | 3.2% | 2.4% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-sol | exact_world | 24.0% | 19.5% to 28.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-sol | task_success | 24.2% | 19.7% to 28.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | gpt-6-sol | decision_correct | 0.7% | -1.2% to 2.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | grok-4.6 | fact_f1 | 10.4% | 9.1% to 11.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | grok-4.6 | exact_world | 61.3% | 57.0% to 65.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | grok-4.6 | task_success | 61.3% | 57.0% to 65.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | grok-4.6 | decision_correct | 2.5% | 0.7% to 4.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | kimi-k2.6 | fact_f1 | 9.3% | 8.2% to 10.5% | 600 | 599 | 1 | 120 |
| All providers | baseline:forward-chain | kimi-k2.6 | exact_world | 61.8% | 57.8% to 65.6% | 600 | 599 | 1 | 120 |
| All providers | baseline:forward-chain | kimi-k2.6 | task_success | 61.8% | 57.8% to 65.6% | 600 | 599 | 1 | 120 |
| All providers | baseline:forward-chain | kimi-k2.6 | decision_correct | 1.3% | -1.0% to 3.3% | 600 | 599 | 1 | 120 |
| All providers | baseline:forward-chain | kimi-k3 | fact_f1 | 14.8% | 12.0% to 17.7% | 600 | 443 | 157 | 96 |
| All providers | baseline:forward-chain | kimi-k3 | exact_world | 55.5% | 50.1% to 60.5% | 600 | 443 | 157 | 96 |
| All providers | baseline:forward-chain | kimi-k3 | task_success | 55.5% | 50.1% to 60.5% | 600 | 443 | 157 | 96 |
| All providers | baseline:forward-chain | kimi-k3 | decision_correct | -0.9% | -2.8% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | baseline:forward-chain | mimo-v2.6-flash | fact_f1 | 20.5% | 18.2% to 22.7% | 600 | 594 | 6 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-flash | exact_world | 57.4% | 53.0% to 61.3% | 600 | 594 | 6 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-flash | task_success | 59.1% | 54.6% to 63.1% | 600 | 594 | 6 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-flash | decision_correct | 9.3% | 6.2% to 12.3% | 600 | 594 | 6 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-pro | fact_f1 | 23.2% | 20.7% to 25.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-pro | exact_world | 59.2% | 54.8% to 63.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-pro | task_success | 60.3% | 56.0% to 64.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | mimo-v2.6-pro | decision_correct | 10.7% | 7.2% to 14.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | minimax-m3 | fact_f1 | 31.4% | 28.2% to 34.6% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | minimax-m3 | exact_world | 64.5% | 60.2% to 68.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | minimax-m3 | task_success | 64.5% | 60.2% to 68.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | minimax-m3 | decision_correct | 2.5% | 0.7% to 4.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | qwen3.8-27b | fact_f1 | 16.2% | 14.1% to 18.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | qwen3.8-27b | exact_world | 64.7% | 60.0% to 68.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | qwen3.8-27b | task_success | 64.7% | 60.0% to 68.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:forward-chain | qwen3.8-27b | decision_correct | 2.2% | 0.7% to 3.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | claude-fable-5 | fact_f1 | -95.5% | -96.4% to -94.5% | 600 | 393 | 207 | 120 |
| All providers | baseline:story-copy | claude-fable-5 | exact_world | -60.3% | -66.0% to -54.8% | 600 | 393 | 207 | 120 |
| All providers | baseline:story-copy | claude-fable-5 | task_success | -60.3% | -66.0% to -54.8% | 600 | 393 | 207 | 120 |
| All providers | baseline:story-copy | claude-fable-5 | decision_correct | -98.7% | -99.8% to -97.3% | 600 | 393 | 207 | 120 |
| All providers | baseline:story-copy | claude-fable-5.1 | fact_f1 | -98.2% | -99.7% to -96.5% | 600 | 14 | 586 | 13 |
| All providers | baseline:story-copy | claude-fable-5.1 | exact_world | -71.4% | -92.3% to -50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:story-copy | claude-fable-5.1 | task_success | -71.4% | -92.3% to -50.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:story-copy | claude-fable-5.1 | decision_correct | -100.0% | -100.0% to -100.0% | 600 | 14 | 586 | 13 |
| All providers | baseline:story-copy | claude-haiku-4.5 | fact_f1 | -68.0% | -70.9% to -65.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | claude-haiku-4.5 | exact_world | -23.7% | -25.7% to -21.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | claude-haiku-4.5 | task_success | -23.7% | -25.7% to -21.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | claude-haiku-4.5 | decision_correct | -95.5% | -97.3% to -93.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | claude-opus-5 | fact_f1 | -81.1% | -84.3% to -77.9% | 600 | 424 | 176 | 119 |
| All providers | baseline:story-copy | claude-opus-5 | exact_world | -52.6% | -58.3% to -47.2% | 600 | 424 | 176 | 119 |
| All providers | baseline:story-copy | claude-opus-5 | task_success | -52.6% | -58.3% to -47.2% | 600 | 424 | 176 | 119 |
| All providers | baseline:story-copy | claude-opus-5 | decision_correct | -91.5% | -94.1% to -88.6% | 600 | 424 | 176 | 119 |
| All providers | baseline:story-copy | claude-opus-5.5 | fact_f1 | -93.3% | -94.1% to -92.4% | 600 | 591 | 9 | 120 |
| All providers | baseline:story-copy | claude-opus-5.5 | exact_world | -40.6% | -44.3% to -37.2% | 600 | 591 | 9 | 120 |
| All providers | baseline:story-copy | claude-opus-5.5 | task_success | -40.4% | -44.2% to -37.0% | 600 | 591 | 9 | 120 |
| All providers | baseline:story-copy | claude-opus-5.5 | decision_correct | -95.3% | -97.1% to -93.1% | 600 | 591 | 9 | 120 |
| All providers | baseline:story-copy | claude-sonnet-5 | fact_f1 | -90.2% | -92.4% to -88.1% | 600 | 445 | 155 | 120 |
| All providers | baseline:story-copy | claude-sonnet-5 | exact_world | -56.4% | -60.9% to -51.3% | 600 | 445 | 155 | 120 |
| All providers | baseline:story-copy | claude-sonnet-5 | task_success | -56.4% | -60.9% to -51.3% | 600 | 445 | 155 | 120 |
| All providers | baseline:story-copy | claude-sonnet-5 | decision_correct | -96.0% | -97.7% to -94.1% | 600 | 445 | 155 | 120 |
| All providers | baseline:story-copy | deepseek-4.1-flash | fact_f1 | -79.4% | -81.7% to -77.1% | 600 | 595 | 5 | 120 |
| All providers | baseline:story-copy | deepseek-4.1-flash | exact_world | -49.1% | -53.1% to -44.9% | 600 | 595 | 5 | 120 |
| All providers | baseline:story-copy | deepseek-4.1-flash | task_success | -49.1% | -53.1% to -44.9% | 600 | 595 | 5 | 120 |
| All providers | baseline:story-copy | deepseek-4.1-flash | decision_correct | -100.0% | -100.0% to -100.0% | 600 | 595 | 5 | 120 |
| All providers | baseline:story-copy | deepseek-v4-pro | fact_f1 | -91.5% | -92.7% to -90.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | deepseek-v4-pro | exact_world | -42.7% | -46.8% to -38.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | deepseek-v4-pro | task_success | -42.7% | -46.8% to -38.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | deepseek-v4-pro | decision_correct | -95.3% | -97.2% to -93.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.6-flash | fact_f1 | -79.5% | -82.1% to -76.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.6-flash | exact_world | -43.7% | -47.8% to -39.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.6-flash | task_success | -43.7% | -47.8% to -39.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.6-flash | decision_correct | -99.0% | -99.8% to -97.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.7-flash | fact_f1 | -91.3% | -93.0% to -89.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.7-flash | exact_world | -68.7% | -72.8% to -64.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.7-flash | task_success | -68.7% | -72.8% to -64.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.7-flash | decision_correct | -99.5% | -100.0% to -98.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gemini-3.8-flash | fact_f1 | -86.5% | -88.7% to -84.4% | 600 | 598 | 2 | 120 |
| All providers | baseline:story-copy | gemini-3.8-flash | exact_world | -59.2% | -63.6% to -55.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:story-copy | gemini-3.8-flash | task_success | -59.2% | -63.6% to -55.0% | 600 | 598 | 2 | 120 |
| All providers | baseline:story-copy | gemini-3.8-flash | decision_correct | -99.7% | -100.0% to -99.2% | 600 | 598 | 2 | 120 |
| All providers | baseline:story-copy | glm-4.7-flash | fact_f1 | -67.1% | -69.9% to -64.4% | 600 | 566 | 34 | 120 |
| All providers | baseline:story-copy | glm-4.7-flash | exact_world | -20.7% | -23.0% to -18.5% | 600 | 566 | 34 | 120 |
| All providers | baseline:story-copy | glm-4.7-flash | task_success | -20.7% | -23.0% to -18.5% | 600 | 566 | 34 | 120 |
| All providers | baseline:story-copy | glm-4.7-flash | decision_correct | -82.5% | -85.8% to -79.1% | 600 | 566 | 34 | 120 |
| All providers | baseline:story-copy | glm-5.3 | fact_f1 | -85.2% | -88.1% to -82.2% | 600 | 564 | 36 | 120 |
| All providers | baseline:story-copy | glm-5.3 | exact_world | -56.7% | -61.5% to -52.2% | 600 | 564 | 36 | 120 |
| All providers | baseline:story-copy | glm-5.3 | task_success | -56.6% | -61.4% to -52.1% | 600 | 564 | 36 | 120 |
| All providers | baseline:story-copy | glm-5.3 | decision_correct | -96.3% | -97.7% to -94.6% | 600 | 564 | 36 | 120 |
| All providers | baseline:story-copy | glm-5.3-flash | fact_f1 | -67.1% | -70.5% to -63.9% | 600 | 596 | 4 | 120 |
| All providers | baseline:story-copy | glm-5.3-flash | exact_world | -26.2% | -29.4% to -23.2% | 600 | 596 | 4 | 120 |
| All providers | baseline:story-copy | glm-5.3-flash | task_success | -26.0% | -29.3% to -23.0% | 600 | 596 | 4 | 120 |
| All providers | baseline:story-copy | glm-5.3-flash | decision_correct | -98.3% | -99.2% to -97.3% | 600 | 596 | 4 | 120 |
| All providers | baseline:story-copy | gpt-5.6-luna | fact_f1 | -81.4% | -83.4% to -79.4% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-luna | exact_world | -42.7% | -46.0% to -39.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-luna | task_success | -41.7% | -45.0% to -38.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-luna | decision_correct | -97.2% | -98.5% to -95.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-sol | fact_f1 | -91.0% | -92.8% to -89.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-sol | exact_world | -68.5% | -72.8% to -63.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-sol | task_success | -68.5% | -72.8% to -63.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-sol | decision_correct | -100.0% | -100.0% to -100.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-terra | fact_f1 | -81.8% | -83.8% to -79.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-terra | exact_world | -50.7% | -54.3% to -47.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-terra | task_success | -50.7% | -54.3% to -47.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-5.6-terra | decision_correct | -99.8% | -100.0% to -99.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-astra | fact_f1 | -98.8% | -99.2% to -98.3% | 600 | 550 | 50 | 115 |
| All providers | baseline:story-copy | gpt-6-astra | exact_world | -87.6% | -91.3% to -83.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:story-copy | gpt-6-astra | task_success | -87.6% | -91.3% to -83.5% | 600 | 550 | 50 | 115 |
| All providers | baseline:story-copy | gpt-6-astra | decision_correct | -100.0% | -100.0% to -100.0% | 600 | 550 | 50 | 115 |
| All providers | baseline:story-copy | gpt-6-luna | fact_f1 | -81.5% | -83.7% to -79.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-luna | exact_world | -42.5% | -46.3% to -39.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-luna | task_success | -42.3% | -46.0% to -38.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-luna | decision_correct | -98.3% | -99.3% to -97.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-sol | fact_f1 | -96.6% | -97.4% to -95.8% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-sol | exact_world | -74.3% | -78.5% to -70.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-sol | task_success | -74.2% | -78.3% to -70.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | gpt-6-sol | decision_correct | -98.5% | -99.3% to -97.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | grok-4.6 | fact_f1 | -89.5% | -90.7% to -88.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | grok-4.6 | exact_world | -37.0% | -40.8% to -33.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | grok-4.6 | task_success | -37.0% | -40.8% to -33.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | grok-4.6 | decision_correct | -96.7% | -98.7% to -94.5% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | kimi-k2.6 | fact_f1 | -90.6% | -91.7% to -89.3% | 600 | 599 | 1 | 120 |
| All providers | baseline:story-copy | kimi-k2.6 | exact_world | -36.6% | -40.1% to -33.1% | 600 | 599 | 1 | 120 |
| All providers | baseline:story-copy | kimi-k2.6 | task_success | -36.6% | -40.1% to -33.1% | 600 | 599 | 1 | 120 |
| All providers | baseline:story-copy | kimi-k2.6 | decision_correct | -97.8% | -99.0% to -96.5% | 600 | 599 | 1 | 120 |
| All providers | baseline:story-copy | kimi-k3 | fact_f1 | -85.0% | -87.8% to -82.1% | 600 | 443 | 157 | 96 |
| All providers | baseline:story-copy | kimi-k3 | exact_world | -42.2% | -47.0% to -37.6% | 600 | 443 | 157 | 96 |
| All providers | baseline:story-copy | kimi-k3 | task_success | -42.2% | -47.0% to -37.6% | 600 | 443 | 157 | 96 |
| All providers | baseline:story-copy | kimi-k3 | decision_correct | -99.8% | -100.0% to -99.3% | 600 | 443 | 157 | 96 |
| All providers | baseline:story-copy | mimo-v2.6-flash | fact_f1 | -79.3% | -81.7% to -77.1% | 600 | 594 | 6 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-flash | exact_world | -40.9% | -44.4% to -37.7% | 600 | 594 | 6 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-flash | task_success | -39.2% | -42.8% to -36.0% | 600 | 594 | 6 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-flash | decision_correct | -89.9% | -92.5% to -87.1% | 600 | 594 | 6 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-pro | fact_f1 | -76.6% | -79.1% to -74.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-pro | exact_world | -39.2% | -42.7% to -35.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-pro | task_success | -38.0% | -41.5% to -34.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | mimo-v2.6-pro | decision_correct | -88.5% | -91.2% to -85.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | minimax-m3 | fact_f1 | -68.5% | -71.7% to -65.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | minimax-m3 | exact_world | -33.8% | -37.2% to -30.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | minimax-m3 | task_success | -33.8% | -37.2% to -30.7% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | minimax-m3 | decision_correct | -96.7% | -98.2% to -95.0% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | qwen3.8-27b | fact_f1 | -83.6% | -85.7% to -81.3% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | qwen3.8-27b | exact_world | -33.7% | -37.5% to -30.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | qwen3.8-27b | task_success | -33.7% | -37.5% to -30.2% | 600 | 600 | 0 | 120 |
| All providers | baseline:story-copy | qwen3.8-27b | decision_correct | -97.0% | -98.5% to -95.2% | 600 | 600 | 0 | 120 |
| All providers | claude-fable-5 | claude-fable-5.1 | fact_f1 | -1.3% | -3.0% to 0.0% | 600 | 9 | 591 | 9 |
| All providers | claude-fable-5 | claude-fable-5.1 | exact_world | -11.1% | -33.3% to 0.0% | 600 | 9 | 591 | 9 |
| All providers | claude-fable-5 | claude-fable-5.1 | task_success | -11.1% | -33.3% to 0.0% | 600 | 9 | 591 | 9 |
| All providers | claude-fable-5 | claude-fable-5.1 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 9 | 591 | 9 |
| All providers | claude-fable-5 | claude-haiku-4.5 | fact_f1 | 25.5% | 21.9% to 29.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | claude-haiku-4.5 | exact_world | 33.8% | 27.9% to 39.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | claude-haiku-4.5 | task_success | 33.8% | 27.9% to 39.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | claude-haiku-4.5 | decision_correct | 3.1% | 0.5% to 5.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | claude-opus-5 | fact_f1 | 12.0% | 7.7% to 16.4% | 600 | 268 | 332 | 112 |
| All providers | claude-fable-5 | claude-opus-5 | exact_world | -4.1% | -14.1% to 5.5% | 600 | 268 | 332 | 112 |
| All providers | claude-fable-5 | claude-opus-5 | task_success | -4.1% | -14.1% to 5.5% | 600 | 268 | 332 | 112 |
| All providers | claude-fable-5 | claude-opus-5 | decision_correct | 6.3% | 2.9% to 9.9% | 600 | 268 | 332 | 112 |
| All providers | claude-fable-5 | claude-opus-5.5 | fact_f1 | 1.8% | 0.5% to 3.0% | 600 | 388 | 212 | 120 |
| All providers | claude-fable-5 | claude-opus-5.5 | exact_world | 17.8% | 11.3% to 24.3% | 600 | 388 | 212 | 120 |
| All providers | claude-fable-5 | claude-opus-5.5 | task_success | 17.8% | 11.3% to 24.3% | 600 | 388 | 212 | 120 |
| All providers | claude-fable-5 | claude-opus-5.5 | decision_correct | 4.1% | 1.3% to 7.1% | 600 | 388 | 212 | 120 |
| All providers | claude-fable-5 | claude-sonnet-5 | fact_f1 | 5.4% | 2.7% to 8.1% | 600 | 292 | 308 | 117 |
| All providers | claude-fable-5 | claude-sonnet-5 | exact_world | 9.6% | 2.4% to 16.8% | 600 | 292 | 308 | 117 |
| All providers | claude-fable-5 | claude-sonnet-5 | task_success | 9.6% | 2.4% to 16.8% | 600 | 292 | 308 | 117 |
| All providers | claude-fable-5 | claude-sonnet-5 | decision_correct | 3.8% | 1.4% to 6.0% | 600 | 292 | 308 | 117 |
| All providers | claude-fable-5 | deepseek-4.1-flash | fact_f1 | 12.0% | 9.3% to 14.7% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | deepseek-4.1-flash | exact_world | 5.6% | -1.1% to 12.1% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | deepseek-4.1-flash | task_success | 5.6% | -1.1% to 12.1% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | deepseek-4.1-flash | decision_correct | -1.3% | -2.7% to -0.2% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | deepseek-v4-pro | fact_f1 | 2.8% | 1.1% to 4.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | deepseek-v4-pro | exact_world | 14.0% | 7.2% to 20.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | deepseek-v4-pro | task_success | 14.0% | 7.2% to 20.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | deepseek-v4-pro | decision_correct | 3.6% | 0.8% to 6.5% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.6-flash | fact_f1 | 14.0% | 10.8% to 17.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.6-flash | exact_world | 14.0% | 7.2% to 20.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.6-flash | task_success | 14.0% | 7.2% to 20.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.6-flash | decision_correct | -0.8% | -2.2% to 0.3% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.7-flash | fact_f1 | 3.4% | 1.5% to 5.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.7-flash | exact_world | -9.7% | -16.5% to -3.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.7-flash | task_success | -9.7% | -16.5% to -3.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.7-flash | decision_correct | -1.0% | -2.5% to 0.3% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gemini-3.8-flash | fact_f1 | 6.9% | 4.4% to 9.2% | 600 | 391 | 209 | 120 |
| All providers | claude-fable-5 | gemini-3.8-flash | exact_world | -2.8% | -9.5% to 4.0% | 600 | 391 | 209 | 120 |
| All providers | claude-fable-5 | gemini-3.8-flash | task_success | -2.8% | -9.5% to 4.0% | 600 | 391 | 209 | 120 |
| All providers | claude-fable-5 | gemini-3.8-flash | decision_correct | -0.8% | -2.3% to 0.5% | 600 | 391 | 209 | 120 |
| All providers | claude-fable-5 | glm-4.7-flash | fact_f1 | 26.6% | 23.5% to 29.9% | 600 | 369 | 231 | 119 |
| All providers | claude-fable-5 | glm-4.7-flash | exact_world | 37.4% | 31.9% to 43.4% | 600 | 369 | 231 | 119 |
| All providers | claude-fable-5 | glm-4.7-flash | task_success | 37.4% | 31.9% to 43.4% | 600 | 369 | 231 | 119 |
| All providers | claude-fable-5 | glm-4.7-flash | decision_correct | 16.3% | 12.0% to 20.6% | 600 | 369 | 231 | 119 |
| All providers | claude-fable-5 | glm-5.3 | fact_f1 | 10.2% | 6.8% to 13.8% | 600 | 368 | 232 | 119 |
| All providers | claude-fable-5 | glm-5.3 | exact_world | 3.8% | -3.5% to 11.1% | 600 | 368 | 232 | 119 |
| All providers | claude-fable-5 | glm-5.3 | task_success | 4.1% | -3.1% to 11.4% | 600 | 368 | 232 | 119 |
| All providers | claude-fable-5 | glm-5.3 | decision_correct | 2.7% | 0.5% to 4.9% | 600 | 368 | 232 | 119 |
| All providers | claude-fable-5 | glm-5.3-flash | fact_f1 | 25.7% | 21.4% to 29.5% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | glm-5.3-flash | exact_world | 31.5% | 25.1% to 38.0% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | glm-5.3-flash | task_success | 31.8% | 25.4% to 38.2% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | glm-5.3-flash | decision_correct | -0.5% | -2.1% to 1.0% | 600 | 390 | 210 | 120 |
| All providers | claude-fable-5 | gpt-5.6-luna | fact_f1 | 11.2% | 8.8% to 13.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-luna | exact_world | 14.5% | 8.0% to 21.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-luna | task_success | 15.8% | 9.3% to 22.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-luna | decision_correct | 1.8% | -0.5% to 4.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-sol | fact_f1 | 3.7% | 1.5% to 5.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-sol | exact_world | -8.4% | -16.2% to -0.5% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-sol | task_success | -8.4% | -16.2% to -0.5% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-sol | decision_correct | -1.3% | -2.7% to -0.2% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-terra | fact_f1 | 12.2% | 9.4% to 14.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-terra | exact_world | 8.9% | 1.9% to 15.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-terra | task_success | 8.9% | 1.9% to 15.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-5.6-terra | decision_correct | -1.3% | -2.7% to -0.2% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-astra | fact_f1 | -3.6% | -4.7% to -2.5% | 600 | 362 | 238 | 115 |
| All providers | claude-fable-5 | gpt-6-astra | exact_world | -28.2% | -34.5% to -21.4% | 600 | 362 | 238 | 115 |
| All providers | claude-fable-5 | gpt-6-astra | task_success | -28.2% | -34.5% to -21.4% | 600 | 362 | 238 | 115 |
| All providers | claude-fable-5 | gpt-6-astra | decision_correct | -1.4% | -2.9% to -0.3% | 600 | 362 | 238 | 115 |
| All providers | claude-fable-5 | gpt-6-luna | fact_f1 | 12.0% | 9.3% to 14.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-luna | exact_world | 15.5% | 9.4% to 21.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-luna | task_success | 15.8% | 9.6% to 22.0% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-luna | decision_correct | 0.5% | -1.3% to 2.3% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-sol | fact_f1 | -1.8% | -2.9% to -0.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-sol | exact_world | -14.8% | -21.0% to -8.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-sol | task_success | -14.5% | -20.8% to -8.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | gpt-6-sol | decision_correct | 0.0% | -1.7% to 1.5% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | grok-4.6 | fact_f1 | 5.1% | 3.6% to 6.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | grok-4.6 | exact_world | 20.6% | 14.8% to 26.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | grok-4.6 | task_success | 20.6% | 14.8% to 26.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | grok-4.6 | decision_correct | 2.8% | 0.2% to 5.6% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | kimi-k2.6 | fact_f1 | 3.7% | 2.4% to 5.1% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | kimi-k2.6 | exact_world | 21.6% | 16.1% to 26.9% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | kimi-k2.6 | task_success | 21.6% | 16.1% to 26.9% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | kimi-k2.6 | decision_correct | 1.5% | -0.5% to 3.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | kimi-k3 | fact_f1 | 10.1% | 6.6% to 13.5% | 600 | 293 | 307 | 94 |
| All providers | claude-fable-5 | kimi-k3 | exact_world | 16.4% | 10.3% to 22.6% | 600 | 293 | 307 | 94 |
| All providers | claude-fable-5 | kimi-k3 | task_success | 16.4% | 10.3% to 22.6% | 600 | 293 | 307 | 94 |
| All providers | claude-fable-5 | kimi-k3 | decision_correct | -1.0% | -2.8% to 0.4% | 600 | 293 | 307 | 94 |
| All providers | claude-fable-5 | mimo-v2.6-flash | fact_f1 | 14.2% | 11.2% to 16.9% | 600 | 387 | 213 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-flash | exact_world | 15.8% | 10.2% to 21.4% | 600 | 387 | 213 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-flash | task_success | 17.3% | 11.6% to 23.1% | 600 | 387 | 213 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-flash | decision_correct | 9.0% | 5.8% to 12.7% | 600 | 387 | 213 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-pro | fact_f1 | 16.7% | 13.5% to 19.9% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-pro | exact_world | 15.5% | 9.2% to 21.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-pro | task_success | 17.0% | 10.7% to 23.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | mimo-v2.6-pro | decision_correct | 10.7% | 7.1% to 14.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | minimax-m3 | fact_f1 | 24.6% | 20.7% to 28.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | minimax-m3 | exact_world | 22.6% | 16.4% to 28.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | minimax-m3 | task_success | 22.6% | 16.4% to 28.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | minimax-m3 | decision_correct | 2.8% | 0.3% to 5.4% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | qwen3.8-27b | fact_f1 | 10.1% | 7.6% to 12.7% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | qwen3.8-27b | exact_world | 25.2% | 18.3% to 32.0% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | qwen3.8-27b | task_success | 25.2% | 18.3% to 32.0% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5 | qwen3.8-27b | decision_correct | 2.3% | 0.0% to 4.8% | 600 | 393 | 207 | 120 |
| All providers | claude-fable-5.1 | claude-haiku-4.5 | fact_f1 | 39.4% | 24.0% to 53.9% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-haiku-4.5 | exact_world | 50.0% | 26.7% to 73.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-haiku-4.5 | task_success | 50.0% | 26.7% to 73.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-haiku-4.5 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-opus-5 | fact_f1 | 38.7% | 10.3% to 65.2% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | claude-opus-5 | exact_world | 22.2% | 0.0% to 50.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | claude-opus-5 | task_success | 22.2% | 0.0% to 50.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | claude-opus-5 | decision_correct | 33.3% | 11.1% to 62.5% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | claude-opus-5.5 | fact_f1 | 4.7% | 0.3% to 12.9% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-opus-5.5 | exact_world | 14.3% | 0.0% to 35.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-opus-5.5 | task_success | 14.3% | 0.0% to 35.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-opus-5.5 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | claude-sonnet-5 | fact_f1 | 19.2% | -0.7% to 39.5% | 600 | 8 | 592 | 8 |
| All providers | claude-fable-5.1 | claude-sonnet-5 | exact_world | 12.5% | -25.0% to 50.0% | 600 | 8 | 592 | 8 |
| All providers | claude-fable-5.1 | claude-sonnet-5 | task_success | 12.5% | -25.0% to 50.0% | 600 | 8 | 592 | 8 |
| All providers | claude-fable-5.1 | claude-sonnet-5 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 8 | 592 | 8 |
| All providers | claude-fable-5.1 | deepseek-4.1-flash | fact_f1 | 46.1% | 27.9% to 62.1% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-4.1-flash | exact_world | 42.9% | 18.8% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-4.1-flash | task_success | 42.9% | 18.8% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-4.1-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-v4-pro | fact_f1 | 18.2% | 7.1% to 29.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-v4-pro | exact_world | 50.0% | 26.7% to 73.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-v4-pro | task_success | 50.0% | 26.7% to 73.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | deepseek-v4-pro | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.6-flash | fact_f1 | 34.8% | 18.9% to 49.9% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.6-flash | exact_world | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.6-flash | task_success | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.6-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.7-flash | fact_f1 | 20.7% | 7.7% to 36.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.7-flash | exact_world | 28.6% | 6.7% to 53.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.7-flash | task_success | 28.6% | 6.7% to 53.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.7-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.8-flash | fact_f1 | 36.0% | 20.6% to 49.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.8-flash | exact_world | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.8-flash | task_success | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gemini-3.8-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-4.7-flash | fact_f1 | 31.1% | 14.5% to 50.2% | 600 | 13 | 587 | 12 |
| All providers | claude-fable-5.1 | glm-4.7-flash | exact_world | 53.8% | 25.0% to 76.9% | 600 | 13 | 587 | 12 |
| All providers | claude-fable-5.1 | glm-4.7-flash | task_success | 53.8% | 25.0% to 76.9% | 600 | 13 | 587 | 12 |
| All providers | claude-fable-5.1 | glm-4.7-flash | decision_correct | 15.4% | 0.0% to 38.5% | 600 | 13 | 587 | 12 |
| All providers | claude-fable-5.1 | glm-5.3 | fact_f1 | 18.5% | 4.3% to 33.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3 | exact_world | 35.7% | 7.1% to 64.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3 | task_success | 35.7% | 7.1% to 64.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3 | decision_correct | 14.3% | 0.0% to 33.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3-flash | fact_f1 | 33.5% | 14.9% to 53.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3-flash | exact_world | 57.1% | 33.3% to 78.6% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3-flash | task_success | 57.1% | 33.3% to 78.6% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | glm-5.3-flash | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-luna | fact_f1 | 23.7% | 5.8% to 44.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-luna | exact_world | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-luna | task_success | 28.6% | 7.7% to 50.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-luna | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-sol | fact_f1 | 14.3% | -0.3% to 29.3% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-sol | exact_world | 7.1% | -14.3% to 35.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-sol | task_success | 7.1% | -14.3% to 35.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-sol | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-terra | fact_f1 | 39.8% | 18.5% to 61.6% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-terra | exact_world | 35.7% | 5.9% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-terra | task_success | 35.7% | 5.9% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-5.6-terra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-astra | fact_f1 | -0.9% | -2.9% to 0.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | gpt-6-astra | exact_world | -11.1% | -37.5% to 0.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | gpt-6-astra | task_success | -11.1% | -37.5% to 0.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | gpt-6-astra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 9 | 591 | 8 |
| All providers | claude-fable-5.1 | gpt-6-luna | fact_f1 | 29.8% | 10.1% to 51.9% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-luna | exact_world | 21.4% | -13.3% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-luna | task_success | 21.4% | -13.3% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-luna | decision_correct | 7.1% | 0.0% to 23.1% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-sol | fact_f1 | 0.5% | -1.8% to 3.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-sol | exact_world | -7.1% | -35.3% to 26.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-sol | task_success | -7.1% | -35.3% to 26.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | gpt-6-sol | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | grok-4.6 | fact_f1 | 11.3% | 4.1% to 20.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | grok-4.6 | exact_world | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | grok-4.6 | task_success | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | grok-4.6 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | kimi-k2.6 | fact_f1 | 19.1% | 7.7% to 30.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | kimi-k2.6 | exact_world | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | kimi-k2.6 | task_success | 35.7% | 12.5% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | kimi-k2.6 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | kimi-k3 | fact_f1 | 40.3% | 3.3% to 64.9% | 600 | 5 | 595 | 4 |
| All providers | claude-fable-5.1 | kimi-k3 | exact_world | 40.0% | 0.0% to 75.0% | 600 | 5 | 595 | 4 |
| All providers | claude-fable-5.1 | kimi-k3 | task_success | 40.0% | 0.0% to 75.0% | 600 | 5 | 595 | 4 |
| All providers | claude-fable-5.1 | kimi-k3 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 5 | 595 | 4 |
| All providers | claude-fable-5.1 | mimo-v2.6-flash | fact_f1 | 27.4% | 13.8% to 41.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-flash | exact_world | 42.9% | 18.8% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-flash | task_success | 42.9% | 18.8% to 69.2% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-flash | decision_correct | 14.3% | 0.0% to 35.7% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-pro | fact_f1 | 43.5% | 26.0% to 58.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-pro | exact_world | 42.9% | 21.4% to 68.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-pro | task_success | 42.9% | 21.4% to 68.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | mimo-v2.6-pro | decision_correct | 21.4% | 0.0% to 42.9% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | minimax-m3 | fact_f1 | 40.0% | 19.8% to 58.4% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | minimax-m3 | exact_world | 35.7% | 14.3% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | minimax-m3 | task_success | 35.7% | 14.3% to 61.5% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | minimax-m3 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | qwen3.8-27b | fact_f1 | 20.5% | 7.8% to 35.1% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | qwen3.8-27b | exact_world | 42.9% | 21.4% to 68.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | qwen3.8-27b | task_success | 42.9% | 21.4% to 68.8% | 600 | 14 | 586 | 13 |
| All providers | claude-fable-5.1 | qwen3.8-27b | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 14 | 586 | 13 |
| All providers | claude-haiku-4.5 | claude-opus-5 | fact_f1 | -19.6% | -23.7% to -15.7% | 600 | 424 | 176 | 119 |
| All providers | claude-haiku-4.5 | claude-opus-5 | exact_world | -47.4% | -53.2% to -41.6% | 600 | 424 | 176 | 119 |
| All providers | claude-haiku-4.5 | claude-opus-5 | task_success | -47.4% | -53.2% to -41.6% | 600 | 424 | 176 | 119 |
| All providers | claude-haiku-4.5 | claude-opus-5 | decision_correct | 2.8% | -1.2% to 6.7% | 600 | 424 | 176 | 119 |
| All providers | claude-haiku-4.5 | claude-opus-5.5 | fact_f1 | -25.0% | -28.0% to -21.9% | 600 | 591 | 9 | 120 |
| All providers | claude-haiku-4.5 | claude-opus-5.5 | exact_world | -16.6% | -20.7% to -12.4% | 600 | 591 | 9 | 120 |
| All providers | claude-haiku-4.5 | claude-opus-5.5 | task_success | -16.4% | -20.5% to -12.2% | 600 | 591 | 9 | 120 |
| All providers | claude-haiku-4.5 | claude-opus-5.5 | decision_correct | 0.2% | -2.4% to 2.7% | 600 | 591 | 9 | 120 |
| All providers | claude-haiku-4.5 | claude-sonnet-5 | fact_f1 | -17.4% | -20.1% to -14.6% | 600 | 445 | 155 | 120 |
| All providers | claude-haiku-4.5 | claude-sonnet-5 | exact_world | -25.4% | -30.0% to -20.6% | 600 | 445 | 155 | 120 |
| All providers | claude-haiku-4.5 | claude-sonnet-5 | task_success | -25.4% | -30.0% to -20.6% | 600 | 445 | 155 | 120 |
| All providers | claude-haiku-4.5 | claude-sonnet-5 | decision_correct | -0.7% | -3.1% to 1.8% | 600 | 445 | 155 | 120 |
| All providers | claude-haiku-4.5 | deepseek-4.1-flash | fact_f1 | -11.2% | -13.8% to -8.3% | 600 | 595 | 5 | 120 |
| All providers | claude-haiku-4.5 | deepseek-4.1-flash | exact_world | -25.2% | -29.0% to -21.1% | 600 | 595 | 5 | 120 |
| All providers | claude-haiku-4.5 | deepseek-4.1-flash | task_success | -25.2% | -29.0% to -21.1% | 600 | 595 | 5 | 120 |
| All providers | claude-haiku-4.5 | deepseek-4.1-flash | decision_correct | -4.4% | -6.2% to -2.7% | 600 | 595 | 5 | 120 |
| All providers | claude-haiku-4.5 | deepseek-v4-pro | fact_f1 | -23.4% | -26.3% to -20.6% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | deepseek-v4-pro | exact_world | -19.0% | -22.7% to -15.5% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | deepseek-v4-pro | task_success | -19.0% | -22.7% to -15.5% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | deepseek-v4-pro | decision_correct | 0.2% | -2.0% to 2.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.6-flash | fact_f1 | -11.4% | -13.9% to -8.9% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.6-flash | exact_world | -20.0% | -23.7% to -16.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.6-flash | task_success | -20.0% | -23.7% to -16.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.6-flash | decision_correct | -3.5% | -5.0% to -2.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.7-flash | fact_f1 | -23.3% | -25.7% to -20.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.7-flash | exact_world | -45.0% | -48.8% to -41.2% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.7-flash | task_success | -45.0% | -48.8% to -41.2% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.7-flash | decision_correct | -4.0% | -5.5% to -2.5% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.8-flash | fact_f1 | -18.5% | -20.8% to -16.2% | 600 | 598 | 2 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.8-flash | exact_world | -35.5% | -39.9% to -31.1% | 600 | 598 | 2 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.8-flash | task_success | -35.5% | -39.9% to -31.1% | 600 | 598 | 2 | 120 |
| All providers | claude-haiku-4.5 | gemini-3.8-flash | decision_correct | -4.2% | -5.9% to -2.7% | 600 | 598 | 2 | 120 |
| All providers | claude-haiku-4.5 | glm-4.7-flash | fact_f1 | 2.6% | -0.6% to 5.9% | 600 | 566 | 34 | 120 |
| All providers | claude-haiku-4.5 | glm-4.7-flash | exact_world | 4.4% | 1.8% to 7.1% | 600 | 566 | 34 | 120 |
| All providers | claude-haiku-4.5 | glm-4.7-flash | task_success | 4.4% | 1.8% to 7.1% | 600 | 566 | 34 | 120 |
| All providers | claude-haiku-4.5 | glm-4.7-flash | decision_correct | 14.3% | 11.0% to 17.7% | 600 | 566 | 34 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3 | fact_f1 | -16.2% | -19.5% to -13.0% | 600 | 564 | 36 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3 | exact_world | -31.9% | -36.5% to -27.9% | 600 | 564 | 36 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3 | task_success | -31.7% | -36.3% to -27.7% | 600 | 564 | 36 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3 | decision_correct | -0.5% | -2.9% to 1.8% | 600 | 564 | 36 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3-flash | fact_f1 | 0.9% | -2.0% to 3.9% | 600 | 596 | 4 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3-flash | exact_world | -2.3% | -5.4% to 0.5% | 600 | 596 | 4 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3-flash | task_success | -2.2% | -5.2% to 0.7% | 600 | 596 | 4 | 120 |
| All providers | claude-haiku-4.5 | glm-5.3-flash | decision_correct | -2.9% | -5.2% to -0.7% | 600 | 596 | 4 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-luna | fact_f1 | -13.3% | -16.3% to -10.1% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-luna | exact_world | -19.0% | -22.8% to -15.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-luna | task_success | -18.0% | -21.8% to -14.2% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-luna | decision_correct | -1.7% | -3.7% to 0.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-sol | fact_f1 | -23.0% | -25.8% to -20.1% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-sol | exact_world | -44.8% | -49.7% to -39.5% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-sol | task_success | -44.8% | -49.7% to -39.5% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-sol | decision_correct | -4.5% | -6.5% to -2.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-terra | fact_f1 | -13.7% | -16.5% to -10.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-terra | exact_world | -27.0% | -30.7% to -23.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-terra | task_success | -27.0% | -30.7% to -23.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-5.6-terra | decision_correct | -4.3% | -6.2% to -2.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-astra | fact_f1 | -29.4% | -32.3% to -26.4% | 600 | 550 | 50 | 115 |
| All providers | claude-haiku-4.5 | gpt-6-astra | exact_world | -62.9% | -67.3% to -58.2% | 600 | 550 | 50 | 115 |
| All providers | claude-haiku-4.5 | gpt-6-astra | task_success | -62.9% | -67.3% to -58.2% | 600 | 550 | 50 | 115 |
| All providers | claude-haiku-4.5 | gpt-6-astra | decision_correct | -4.4% | -6.4% to -2.7% | 600 | 550 | 50 | 115 |
| All providers | claude-haiku-4.5 | gpt-6-luna | fact_f1 | -13.4% | -16.4% to -10.4% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-luna | exact_world | -18.8% | -22.8% to -15.2% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-luna | task_success | -18.7% | -22.7% to -15.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-luna | decision_correct | -2.8% | -5.2% to -0.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-sol | fact_f1 | -28.6% | -31.7% to -25.4% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-sol | exact_world | -50.7% | -54.8% to -46.0% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-sol | task_success | -50.5% | -54.7% to -45.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | gpt-6-sol | decision_correct | -3.0% | -4.8% to -1.2% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | grok-4.6 | fact_f1 | -21.4% | -24.5% to -18.4% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | grok-4.6 | exact_world | -13.3% | -17.3% to -9.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | grok-4.6 | task_success | -13.3% | -17.3% to -9.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | grok-4.6 | decision_correct | -1.2% | -3.2% to 0.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | kimi-k2.6 | fact_f1 | -22.5% | -25.1% to -19.8% | 600 | 599 | 1 | 120 |
| All providers | claude-haiku-4.5 | kimi-k2.6 | exact_world | -12.9% | -16.2% to -9.7% | 600 | 599 | 1 | 120 |
| All providers | claude-haiku-4.5 | kimi-k2.6 | task_success | -12.9% | -16.2% to -9.7% | 600 | 599 | 1 | 120 |
| All providers | claude-haiku-4.5 | kimi-k2.6 | decision_correct | -2.3% | -4.5% to -0.3% | 600 | 599 | 1 | 120 |
| All providers | claude-haiku-4.5 | kimi-k3 | fact_f1 | -14.1% | -17.1% to -11.1% | 600 | 443 | 157 | 96 |
| All providers | claude-haiku-4.5 | kimi-k3 | exact_world | -17.2% | -21.5% to -12.7% | 600 | 443 | 157 | 96 |
| All providers | claude-haiku-4.5 | kimi-k3 | task_success | -17.2% | -21.5% to -12.7% | 600 | 443 | 157 | 96 |
| All providers | claude-haiku-4.5 | kimi-k3 | decision_correct | -4.7% | -7.1% to -2.7% | 600 | 443 | 157 | 96 |
| All providers | claude-haiku-4.5 | mimo-v2.6-flash | fact_f1 | -11.4% | -14.1% to -8.6% | 600 | 594 | 6 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-flash | exact_world | -17.3% | -20.7% to -14.2% | 600 | 594 | 6 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-flash | task_success | -15.7% | -19.0% to -12.5% | 600 | 594 | 6 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-flash | decision_correct | 5.6% | 2.5% to 8.8% | 600 | 594 | 6 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-pro | fact_f1 | -8.6% | -11.3% to -5.6% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-pro | exact_world | -15.5% | -18.7% to -12.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-pro | task_success | -14.3% | -17.7% to -10.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | mimo-v2.6-pro | decision_correct | 7.0% | 3.3% to 10.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | minimax-m3 | fact_f1 | -0.4% | -3.5% to 2.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | minimax-m3 | exact_world | -10.2% | -13.5% to -6.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | minimax-m3 | task_success | -10.2% | -13.5% to -6.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | minimax-m3 | decision_correct | -1.2% | -3.0% to 0.7% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | qwen3.8-27b | fact_f1 | -15.6% | -18.2% to -12.8% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | qwen3.8-27b | exact_world | -10.0% | -13.7% to -6.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | qwen3.8-27b | task_success | -10.0% | -13.7% to -6.3% | 600 | 600 | 0 | 120 |
| All providers | claude-haiku-4.5 | qwen3.8-27b | decision_correct | -1.5% | -3.5% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | claude-opus-5 | claude-opus-5.5 | fact_f1 | -10.5% | -13.8% to -7.2% | 600 | 416 | 184 | 119 |
| All providers | claude-opus-5 | claude-opus-5.5 | exact_world | 27.2% | 20.8% to 33.2% | 600 | 416 | 184 | 119 |
| All providers | claude-opus-5 | claude-opus-5.5 | task_success | 27.4% | 21.0% to 33.5% | 600 | 416 | 184 | 119 |
| All providers | claude-opus-5 | claude-opus-5.5 | decision_correct | -2.6% | -6.2% to 0.8% | 600 | 416 | 184 | 119 |
| All providers | claude-opus-5 | claude-sonnet-5 | fact_f1 | -6.6% | -11.0% to -2.4% | 600 | 314 | 286 | 106 |
| All providers | claude-opus-5 | claude-sonnet-5 | exact_world | 11.5% | 3.4% to 19.4% | 600 | 314 | 286 | 106 |
| All providers | claude-opus-5 | claude-sonnet-5 | task_success | 11.5% | 3.4% to 19.4% | 600 | 314 | 286 | 106 |
| All providers | claude-opus-5 | claude-sonnet-5 | decision_correct | -4.8% | -8.8% to -0.9% | 600 | 314 | 286 | 106 |
| All providers | claude-opus-5 | deepseek-4.1-flash | fact_f1 | 5.4% | 1.7% to 9.2% | 600 | 419 | 181 | 119 |
| All providers | claude-opus-5 | deepseek-4.1-flash | exact_world | 14.1% | 7.1% to 21.2% | 600 | 419 | 181 | 119 |
| All providers | claude-opus-5 | deepseek-4.1-flash | task_success | 14.1% | 7.1% to 21.2% | 600 | 419 | 181 | 119 |
| All providers | claude-opus-5 | deepseek-4.1-flash | decision_correct | -8.4% | -11.1% to -5.7% | 600 | 419 | 181 | 119 |
| All providers | claude-opus-5 | deepseek-v4-pro | fact_f1 | -9.0% | -12.4% to -5.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | deepseek-v4-pro | exact_world | 22.2% | 15.3% to 28.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | deepseek-v4-pro | task_success | 22.2% | 15.3% to 28.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | deepseek-v4-pro | decision_correct | -3.5% | -6.7% to -0.2% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.6-flash | fact_f1 | 6.3% | 2.2% to 10.6% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.6-flash | exact_world | 21.0% | 13.5% to 28.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.6-flash | task_success | 21.0% | 13.5% to 28.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.6-flash | decision_correct | -7.3% | -10.3% to -4.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.7-flash | fact_f1 | -8.8% | -12.2% to -5.2% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.7-flash | exact_world | -9.0% | -15.9% to -2.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.7-flash | task_success | -9.0% | -15.9% to -2.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.7-flash | decision_correct | -7.8% | -10.7% to -4.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gemini-3.8-flash | fact_f1 | -3.6% | -6.9% to -0.2% | 600 | 422 | 178 | 119 |
| All providers | claude-opus-5 | gemini-3.8-flash | exact_world | 1.4% | -5.8% to 8.7% | 600 | 422 | 178 | 119 |
| All providers | claude-opus-5 | gemini-3.8-flash | task_success | 1.4% | -5.8% to 8.7% | 600 | 422 | 178 | 119 |
| All providers | claude-opus-5 | gemini-3.8-flash | decision_correct | -8.1% | -11.0% to -5.2% | 600 | 422 | 178 | 119 |
| All providers | claude-opus-5 | glm-4.7-flash | fact_f1 | 19.6% | 15.2% to 24.2% | 600 | 393 | 207 | 118 |
| All providers | claude-opus-5 | glm-4.7-flash | exact_world | 47.3% | 41.4% to 53.3% | 600 | 393 | 207 | 118 |
| All providers | claude-opus-5 | glm-4.7-flash | task_success | 47.3% | 41.4% to 53.3% | 600 | 393 | 207 | 118 |
| All providers | claude-opus-5 | glm-4.7-flash | decision_correct | 11.2% | 6.0% to 16.4% | 600 | 393 | 207 | 118 |
| All providers | claude-opus-5 | glm-5.3 | fact_f1 | -1.2% | -5.5% to 3.0% | 600 | 390 | 210 | 118 |
| All providers | claude-opus-5 | glm-5.3 | exact_world | 6.4% | -1.3% to 13.9% | 600 | 390 | 210 | 118 |
| All providers | claude-opus-5 | glm-5.3 | task_success | 6.7% | -1.0% to 14.0% | 600 | 390 | 210 | 118 |
| All providers | claude-opus-5 | glm-5.3 | decision_correct | -4.1% | -7.6% to -0.5% | 600 | 390 | 210 | 118 |
| All providers | claude-opus-5 | glm-5.3-flash | fact_f1 | 20.9% | 16.0% to 25.9% | 600 | 420 | 180 | 119 |
| All providers | claude-opus-5 | glm-5.3-flash | exact_world | 40.0% | 33.5% to 46.5% | 600 | 420 | 180 | 119 |
| All providers | claude-opus-5 | glm-5.3-flash | task_success | 40.2% | 33.6% to 46.7% | 600 | 420 | 180 | 119 |
| All providers | claude-opus-5 | glm-5.3-flash | decision_correct | -6.4% | -9.6% to -3.3% | 600 | 420 | 180 | 119 |
| All providers | claude-opus-5 | gpt-5.6-luna | fact_f1 | 2.4% | -1.0% to 5.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-luna | exact_world | 22.6% | 16.1% to 29.0% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-luna | task_success | 23.6% | 17.2% to 30.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-luna | decision_correct | -5.7% | -8.9% to -2.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-sol | fact_f1 | -8.4% | -11.6% to -5.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-sol | exact_world | -8.7% | -15.8% to -1.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-sol | task_success | -8.7% | -15.8% to -1.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-sol | decision_correct | -8.5% | -11.4% to -5.9% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-terra | fact_f1 | 2.1% | -1.5% to 5.9% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-terra | exact_world | 12.7% | 6.5% to 19.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-terra | task_success | 12.7% | 6.5% to 19.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-5.6-terra | decision_correct | -8.3% | -11.1% to -5.5% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-astra | fact_f1 | -16.7% | -20.1% to -13.5% | 600 | 386 | 214 | 111 |
| All providers | claude-opus-5 | gpt-6-astra | exact_world | -32.1% | -39.0% to -25.1% | 600 | 386 | 214 | 111 |
| All providers | claude-opus-5 | gpt-6-astra | task_success | -32.1% | -39.0% to -25.1% | 600 | 386 | 214 | 111 |
| All providers | claude-opus-5 | gpt-6-astra | decision_correct | -7.8% | -10.7% to -5.2% | 600 | 386 | 214 | 111 |
| All providers | claude-opus-5 | gpt-6-luna | fact_f1 | 3.6% | -0.4% to 7.5% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-luna | exact_world | 24.8% | 18.1% to 31.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-luna | task_success | 24.8% | 18.1% to 31.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-luna | decision_correct | -6.4% | -9.6% to -3.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-sol | fact_f1 | -15.1% | -18.5% to -11.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-sol | exact_world | -16.0% | -22.9% to -8.9% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-sol | task_success | -15.8% | -22.7% to -8.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | gpt-6-sol | decision_correct | -6.8% | -9.9% to -3.9% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | grok-4.6 | fact_f1 | -6.2% | -9.7% to -2.4% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | grok-4.6 | exact_world | 30.7% | 24.4% to 36.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | grok-4.6 | task_success | 30.7% | 24.4% to 36.8% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | grok-4.6 | decision_correct | -4.7% | -8.4% to -0.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | kimi-k2.6 | fact_f1 | -7.5% | -10.8% to -4.4% | 600 | 423 | 177 | 119 |
| All providers | claude-opus-5 | kimi-k2.6 | exact_world | 30.7% | 24.0% to 37.7% | 600 | 423 | 177 | 119 |
| All providers | claude-opus-5 | kimi-k2.6 | task_success | 30.7% | 24.0% to 37.7% | 600 | 423 | 177 | 119 |
| All providers | claude-opus-5 | kimi-k2.6 | decision_correct | -6.4% | -9.5% to -3.5% | 600 | 423 | 177 | 119 |
| All providers | claude-opus-5 | kimi-k3 | fact_f1 | -0.1% | -4.1% to 4.0% | 600 | 310 | 290 | 94 |
| All providers | claude-opus-5 | kimi-k3 | exact_world | 24.2% | 16.2% to 32.6% | 600 | 310 | 290 | 94 |
| All providers | claude-opus-5 | kimi-k3 | task_success | 24.2% | 16.2% to 32.6% | 600 | 310 | 290 | 94 |
| All providers | claude-opus-5 | kimi-k3 | decision_correct | -7.1% | -10.5% to -4.1% | 600 | 310 | 290 | 94 |
| All providers | claude-opus-5 | mimo-v2.6-flash | fact_f1 | 4.3% | 0.4% to 8.1% | 600 | 418 | 182 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-flash | exact_world | 23.4% | 17.3% to 29.8% | 600 | 418 | 182 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-flash | task_success | 24.9% | 18.4% to 31.3% | 600 | 418 | 182 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-flash | decision_correct | 1.9% | -2.0% to 6.0% | 600 | 418 | 182 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-pro | fact_f1 | 7.9% | 3.8% to 11.9% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-pro | exact_world | 25.5% | 19.2% to 32.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-pro | task_success | 26.7% | 20.2% to 33.4% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | mimo-v2.6-pro | decision_correct | 5.9% | 1.7% to 10.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | minimax-m3 | fact_f1 | 19.3% | 14.4% to 24.0% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | minimax-m3 | exact_world | 34.0% | 27.4% to 40.5% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | minimax-m3 | task_success | 34.0% | 27.4% to 40.5% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | minimax-m3 | decision_correct | -4.7% | -8.3% to -1.1% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | qwen3.8-27b | fact_f1 | -0.3% | -4.3% to 3.7% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | qwen3.8-27b | exact_world | 32.5% | 26.0% to 39.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | qwen3.8-27b | task_success | 32.5% | 26.0% to 39.3% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5 | qwen3.8-27b | decision_correct | -5.7% | -8.9% to -2.2% | 600 | 424 | 176 | 119 |
| All providers | claude-opus-5.5 | claude-sonnet-5 | fact_f1 | 3.9% | 1.6% to 6.2% | 600 | 441 | 159 | 120 |
| All providers | claude-opus-5.5 | claude-sonnet-5 | exact_world | -12.2% | -17.8% to -5.9% | 600 | 441 | 159 | 120 |
| All providers | claude-opus-5.5 | claude-sonnet-5 | task_success | -12.5% | -17.9% to -6.3% | 600 | 441 | 159 | 120 |
| All providers | claude-opus-5.5 | claude-sonnet-5 | decision_correct | -0.2% | -2.9% to 2.3% | 600 | 441 | 159 | 120 |
| All providers | claude-opus-5.5 | deepseek-4.1-flash | fact_f1 | 13.8% | 11.4% to 16.1% | 600 | 586 | 14 | 120 |
| All providers | claude-opus-5.5 | deepseek-4.1-flash | exact_world | -8.4% | -13.6% to -3.1% | 600 | 586 | 14 | 120 |
| All providers | claude-opus-5.5 | deepseek-4.1-flash | task_success | -8.5% | -13.8% to -3.4% | 600 | 586 | 14 | 120 |
| All providers | claude-opus-5.5 | deepseek-4.1-flash | decision_correct | -4.4% | -6.5% to -2.7% | 600 | 586 | 14 | 120 |
| All providers | claude-opus-5.5 | deepseek-v4-pro | fact_f1 | 1.8% | 0.3% to 3.3% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | deepseek-v4-pro | exact_world | -2.2% | -7.8% to 3.2% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | deepseek-v4-pro | task_success | -2.4% | -8.0% to 3.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | deepseek-v4-pro | decision_correct | -0.3% | -2.5% to 1.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.6-flash | fact_f1 | 13.8% | 11.0% to 16.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.6-flash | exact_world | -2.9% | -8.9% to 3.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.6-flash | task_success | -3.0% | -9.1% to 2.9% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.6-flash | decision_correct | -3.7% | -5.9% to -1.7% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.7-flash | fact_f1 | 1.9% | 0.1% to 3.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.7-flash | exact_world | -28.3% | -33.3% to -23.3% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.7-flash | task_success | -28.4% | -33.4% to -23.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.7-flash | decision_correct | -4.2% | -6.3% to -2.4% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gemini-3.8-flash | fact_f1 | 6.6% | 4.4% to 8.7% | 600 | 589 | 11 | 120 |
| All providers | claude-opus-5.5 | gemini-3.8-flash | exact_world | -18.8% | -24.0% to -13.7% | 600 | 589 | 11 | 120 |
| All providers | claude-opus-5.5 | gemini-3.8-flash | task_success | -19.0% | -24.1% to -13.9% | 600 | 589 | 11 | 120 |
| All providers | claude-opus-5.5 | gemini-3.8-flash | decision_correct | -4.4% | -6.5% to -2.6% | 600 | 589 | 11 | 120 |
| All providers | claude-opus-5.5 | glm-4.7-flash | fact_f1 | 26.1% | 23.2% to 28.9% | 600 | 557 | 43 | 120 |
| All providers | claude-opus-5.5 | glm-4.7-flash | exact_world | 20.5% | 15.8% to 25.2% | 600 | 557 | 43 | 120 |
| All providers | claude-opus-5.5 | glm-4.7-flash | task_success | 20.3% | 15.6% to 25.0% | 600 | 557 | 43 | 120 |
| All providers | claude-opus-5.5 | glm-4.7-flash | decision_correct | 13.3% | 9.5% to 17.0% | 600 | 557 | 43 | 120 |
| All providers | claude-opus-5.5 | glm-5.3 | fact_f1 | 7.9% | 4.9% to 11.1% | 600 | 556 | 44 | 120 |
| All providers | claude-opus-5.5 | glm-5.3 | exact_world | -15.3% | -21.8% to -9.2% | 600 | 556 | 44 | 120 |
| All providers | claude-opus-5.5 | glm-5.3 | task_success | -15.3% | -21.9% to -9.2% | 600 | 556 | 44 | 120 |
| All providers | claude-opus-5.5 | glm-5.3 | decision_correct | -1.3% | -4.0% to 1.4% | 600 | 556 | 44 | 120 |
| All providers | claude-opus-5.5 | glm-5.3-flash | fact_f1 | 26.0% | 22.4% to 29.4% | 600 | 588 | 12 | 120 |
| All providers | claude-opus-5.5 | glm-5.3-flash | exact_world | 14.5% | 9.5% to 19.3% | 600 | 588 | 12 | 120 |
| All providers | claude-opus-5.5 | glm-5.3-flash | task_success | 14.5% | 9.6% to 19.2% | 600 | 588 | 12 | 120 |
| All providers | claude-opus-5.5 | glm-5.3-flash | decision_correct | -3.1% | -5.4% to -0.8% | 600 | 588 | 12 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-luna | fact_f1 | 11.7% | 9.6% to 13.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-luna | exact_world | -2.2% | -7.1% to 2.9% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-luna | task_success | -1.4% | -6.1% to 3.9% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-luna | decision_correct | -2.0% | -4.3% to 0.2% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-sol | fact_f1 | 2.1% | 0.2% to 4.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-sol | exact_world | -27.9% | -33.6% to -22.5% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-sol | task_success | -28.1% | -33.7% to -22.7% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-sol | decision_correct | -4.7% | -6.9% to -2.9% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-terra | fact_f1 | 11.6% | 9.3% to 13.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-terra | exact_world | -10.0% | -14.9% to -5.4% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-terra | task_success | -10.2% | -15.0% to -5.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-5.6-terra | decision_correct | -4.6% | -6.6% to -2.7% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-astra | fact_f1 | -5.3% | -6.1% to -4.5% | 600 | 542 | 58 | 115 |
| All providers | claude-opus-5.5 | gpt-6-astra | exact_world | -46.7% | -51.1% to -41.8% | 600 | 542 | 58 | 115 |
| All providers | claude-opus-5.5 | gpt-6-astra | task_success | -46.9% | -51.3% to -42.1% | 600 | 542 | 58 | 115 |
| All providers | claude-opus-5.5 | gpt-6-astra | decision_correct | -4.8% | -7.0% to -2.9% | 600 | 542 | 58 | 115 |
| All providers | claude-opus-5.5 | gpt-6-luna | fact_f1 | 11.7% | 9.4% to 14.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-luna | exact_world | -2.5% | -7.5% to 2.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-luna | task_success | -2.5% | -7.6% to 2.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-luna | decision_correct | -3.2% | -5.4% to -1.2% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-sol | fact_f1 | -3.4% | -4.5% to -2.3% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-sol | exact_world | -33.8% | -38.7% to -29.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-sol | task_success | -33.8% | -38.7% to -29.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | gpt-6-sol | decision_correct | -3.2% | -5.2% to -1.4% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | grok-4.6 | fact_f1 | 3.8% | 2.6% to 5.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | grok-4.6 | exact_world | 3.7% | -0.3% to 7.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | grok-4.6 | task_success | 3.6% | -0.5% to 7.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | grok-4.6 | decision_correct | -1.4% | -3.7% to 1.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | kimi-k2.6 | fact_f1 | 2.7% | 1.4% to 4.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | kimi-k2.6 | exact_world | 3.7% | -1.4% to 8.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | kimi-k2.6 | task_success | 3.6% | -1.7% to 8.4% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | kimi-k2.6 | decision_correct | -2.7% | -5.1% to -0.5% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | kimi-k3 | fact_f1 | 8.3% | 5.3% to 11.2% | 600 | 438 | 162 | 96 |
| All providers | claude-opus-5.5 | kimi-k3 | exact_world | -2.1% | -8.1% to 4.0% | 600 | 438 | 162 | 96 |
| All providers | claude-opus-5.5 | kimi-k3 | task_success | -2.3% | -8.3% to 3.6% | 600 | 438 | 162 | 96 |
| All providers | claude-opus-5.5 | kimi-k3 | decision_correct | -4.6% | -6.9% to -2.5% | 600 | 438 | 162 | 96 |
| All providers | claude-opus-5.5 | mimo-v2.6-flash | fact_f1 | 14.1% | 11.6% to 16.6% | 600 | 585 | 15 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-flash | exact_world | -0.7% | -5.6% to 4.0% | 600 | 585 | 15 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-flash | task_success | 0.9% | -4.1% to 5.3% | 600 | 585 | 15 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-flash | decision_correct | 5.6% | 2.2% to 9.0% | 600 | 585 | 15 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-pro | fact_f1 | 16.5% | 14.0% to 19.2% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-pro | exact_world | 1.2% | -3.7% to 6.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-pro | task_success | 2.2% | -2.9% to 7.1% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | mimo-v2.6-pro | decision_correct | 6.9% | 3.4% to 10.6% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | minimax-m3 | fact_f1 | 24.7% | 21.4% to 28.0% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | minimax-m3 | exact_world | 6.6% | 1.7% to 11.7% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | minimax-m3 | task_success | 6.4% | 1.5% to 11.5% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | minimax-m3 | decision_correct | -1.4% | -3.7% to 0.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | qwen3.8-27b | fact_f1 | 9.5% | 7.4% to 11.8% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | qwen3.8-27b | exact_world | 6.8% | 1.3% to 12.3% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | qwen3.8-27b | task_success | 6.6% | 1.0% to 12.2% | 600 | 591 | 9 | 120 |
| All providers | claude-opus-5.5 | qwen3.8-27b | decision_correct | -1.7% | -3.9% to 0.5% | 600 | 591 | 9 | 120 |
| All providers | claude-sonnet-5 | deepseek-4.1-flash | fact_f1 | 5.0% | 2.1% to 7.7% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | deepseek-4.1-flash | exact_world | -0.7% | -5.8% to 4.5% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | deepseek-4.1-flash | task_success | -0.7% | -5.8% to 4.5% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | deepseek-4.1-flash | decision_correct | -4.1% | -6.0% to -2.3% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | deepseek-v4-pro | fact_f1 | -1.9% | -4.4% to 0.5% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | deepseek-v4-pro | exact_world | 7.9% | 2.4% to 13.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | deepseek-v4-pro | task_success | 7.9% | 2.4% to 13.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | deepseek-v4-pro | decision_correct | 0.7% | -1.8% to 3.4% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.6-flash | fact_f1 | 7.9% | 5.0% to 10.8% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.6-flash | exact_world | 4.3% | -1.6% to 9.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.6-flash | task_success | 4.3% | -1.6% to 9.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.6-flash | decision_correct | -3.1% | -4.9% to -1.6% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.7-flash | fact_f1 | -3.4% | -5.6% to -1.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.7-flash | exact_world | -16.9% | -22.0% to -12.1% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.7-flash | task_success | -16.9% | -22.0% to -12.1% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.7-flash | decision_correct | -3.4% | -5.0% to -1.9% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gemini-3.8-flash | fact_f1 | 0.7% | -1.7% to 3.0% | 600 | 443 | 157 | 120 |
| All providers | claude-sonnet-5 | gemini-3.8-flash | exact_world | -9.3% | -14.4% to -4.6% | 600 | 443 | 157 | 120 |
| All providers | claude-sonnet-5 | gemini-3.8-flash | task_success | -9.3% | -14.4% to -4.6% | 600 | 443 | 157 | 120 |
| All providers | claude-sonnet-5 | gemini-3.8-flash | decision_correct | -3.6% | -5.3% to -2.0% | 600 | 443 | 157 | 120 |
| All providers | claude-sonnet-5 | glm-4.7-flash | fact_f1 | 22.8% | 19.2% to 26.2% | 600 | 415 | 185 | 120 |
| All providers | claude-sonnet-5 | glm-4.7-flash | exact_world | 31.6% | 26.1% to 36.6% | 600 | 415 | 185 | 120 |
| All providers | claude-sonnet-5 | glm-4.7-flash | task_success | 31.6% | 26.1% to 36.6% | 600 | 415 | 185 | 120 |
| All providers | claude-sonnet-5 | glm-4.7-flash | decision_correct | 13.3% | 9.3% to 17.6% | 600 | 415 | 185 | 120 |
| All providers | claude-sonnet-5 | glm-5.3 | fact_f1 | 3.7% | 0.4% to 7.2% | 600 | 421 | 179 | 120 |
| All providers | claude-sonnet-5 | glm-5.3 | exact_world | -3.8% | -9.5% to 1.0% | 600 | 421 | 179 | 120 |
| All providers | claude-sonnet-5 | glm-5.3 | task_success | -3.6% | -9.3% to 1.4% | 600 | 421 | 179 | 120 |
| All providers | claude-sonnet-5 | glm-5.3 | decision_correct | -1.0% | -3.2% to 1.2% | 600 | 421 | 179 | 120 |
| All providers | claude-sonnet-5 | glm-5.3-flash | fact_f1 | 18.5% | 14.9% to 21.9% | 600 | 441 | 159 | 120 |
| All providers | claude-sonnet-5 | glm-5.3-flash | exact_world | 23.1% | 17.6% to 28.3% | 600 | 441 | 159 | 120 |
| All providers | claude-sonnet-5 | glm-5.3-flash | task_success | 23.4% | 18.0% to 28.5% | 600 | 441 | 159 | 120 |
| All providers | claude-sonnet-5 | glm-5.3-flash | decision_correct | -2.5% | -4.6% to -0.4% | 600 | 441 | 159 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-luna | fact_f1 | 3.9% | 0.9% to 6.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-luna | exact_world | 5.6% | 0.0% to 10.8% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-luna | task_success | 7.0% | 1.4% to 12.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-luna | decision_correct | -1.1% | -3.3% to 0.9% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-sol | fact_f1 | -3.6% | -6.2% to -1.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-sol | exact_world | -16.0% | -22.6% to -9.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-sol | task_success | -16.0% | -22.6% to -9.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-sol | decision_correct | -4.0% | -5.9% to -2.3% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-terra | fact_f1 | 4.0% | 1.5% to 6.5% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-terra | exact_world | 0.0% | -5.6% to 5.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-terra | task_success | 0.0% | -5.6% to 5.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-5.6-terra | decision_correct | -3.8% | -5.6% to -2.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-astra | fact_f1 | -8.1% | -10.4% to -5.9% | 600 | 407 | 193 | 114 |
| All providers | claude-sonnet-5 | gpt-6-astra | exact_world | -32.7% | -38.9% to -26.0% | 600 | 407 | 193 | 114 |
| All providers | claude-sonnet-5 | gpt-6-astra | task_success | -32.7% | -38.9% to -26.0% | 600 | 407 | 193 | 114 |
| All providers | claude-sonnet-5 | gpt-6-astra | decision_correct | -4.4% | -6.4% to -2.6% | 600 | 407 | 193 | 114 |
| All providers | claude-sonnet-5 | gpt-6-luna | fact_f1 | 5.5% | 3.0% to 8.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-luna | exact_world | 9.9% | 4.7% to 14.8% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-luna | task_success | 10.1% | 4.9% to 15.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-luna | decision_correct | -2.2% | -4.4% to 0.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-sol | fact_f1 | -6.8% | -9.2% to -4.5% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-sol | exact_world | -20.7% | -27.7% to -13.8% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-sol | task_success | -20.4% | -27.5% to -13.5% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | gpt-6-sol | decision_correct | -2.2% | -4.3% to -0.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | grok-4.6 | fact_f1 | -1.0% | -3.4% to 1.3% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | grok-4.6 | exact_world | 11.7% | 5.6% to 17.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | grok-4.6 | task_success | 11.7% | 5.6% to 17.2% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | grok-4.6 | decision_correct | -0.7% | -2.9% to 1.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | kimi-k2.6 | fact_f1 | -1.7% | -4.0% to 0.5% | 600 | 444 | 156 | 120 |
| All providers | claude-sonnet-5 | kimi-k2.6 | exact_world | 13.7% | 8.1% to 18.6% | 600 | 444 | 156 | 120 |
| All providers | claude-sonnet-5 | kimi-k2.6 | task_success | 13.7% | 8.1% to 18.6% | 600 | 444 | 156 | 120 |
| All providers | claude-sonnet-5 | kimi-k2.6 | decision_correct | -1.8% | -4.2% to 0.7% | 600 | 444 | 156 | 120 |
| All providers | claude-sonnet-5 | kimi-k3 | fact_f1 | 2.6% | -0.3% to 5.6% | 600 | 329 | 271 | 95 |
| All providers | claude-sonnet-5 | kimi-k3 | exact_world | 9.1% | 3.2% to 15.0% | 600 | 329 | 271 | 95 |
| All providers | claude-sonnet-5 | kimi-k3 | task_success | 9.1% | 3.2% to 15.0% | 600 | 329 | 271 | 95 |
| All providers | claude-sonnet-5 | kimi-k3 | decision_correct | -4.3% | -6.3% to -2.4% | 600 | 329 | 271 | 95 |
| All providers | claude-sonnet-5 | mimo-v2.6-flash | fact_f1 | 7.8% | 5.0% to 10.6% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-flash | exact_world | 7.7% | 2.3% to 13.0% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-flash | task_success | 9.5% | 4.2% to 14.4% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-flash | decision_correct | 6.6% | 3.4% to 10.0% | 600 | 440 | 160 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-pro | fact_f1 | 9.9% | 6.7% to 13.0% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-pro | exact_world | 11.0% | 5.3% to 16.4% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-pro | task_success | 12.1% | 6.2% to 17.6% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | mimo-v2.6-pro | decision_correct | 7.6% | 3.9% to 11.6% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | minimax-m3 | fact_f1 | 16.4% | 13.0% to 19.8% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | minimax-m3 | exact_world | 14.6% | 9.8% to 19.3% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | minimax-m3 | task_success | 14.6% | 9.8% to 19.3% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | minimax-m3 | decision_correct | -1.1% | -3.3% to 0.9% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | qwen3.8-27b | fact_f1 | 4.6% | 1.6% to 7.4% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | qwen3.8-27b | exact_world | 14.8% | 9.6% to 19.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | qwen3.8-27b | task_success | 14.8% | 9.6% to 19.7% | 600 | 445 | 155 | 120 |
| All providers | claude-sonnet-5 | qwen3.8-27b | decision_correct | -0.2% | -2.5% to 2.2% | 600 | 445 | 155 | 120 |
| All providers | deepseek-4.1-flash | deepseek-v4-pro | fact_f1 | -12.1% | -14.6% to -9.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | deepseek-v4-pro | exact_world | 6.4% | 2.0% to 10.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | deepseek-v4-pro | task_success | 6.4% | 2.0% to 10.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | deepseek-v4-pro | decision_correct | 4.7% | 2.9% to 6.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.6-flash | fact_f1 | -0.2% | -2.6% to 2.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.6-flash | exact_world | 5.4% | 0.7% to 10.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.6-flash | task_success | 5.4% | 0.7% to 10.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.6-flash | decision_correct | 0.8% | 0.2% to 1.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.7-flash | fact_f1 | -12.3% | -14.7% to -10.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.7-flash | exact_world | -20.0% | -25.0% to -15.1% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.7-flash | task_success | -20.0% | -25.0% to -15.1% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.7-flash | decision_correct | 0.3% | 0.0% to 0.8% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.8-flash | fact_f1 | -7.2% | -9.2% to -5.2% | 600 | 593 | 7 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.8-flash | exact_world | -10.3% | -15.1% to -5.6% | 600 | 593 | 7 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.8-flash | task_success | -10.3% | -15.1% to -5.6% | 600 | 593 | 7 | 120 |
| All providers | deepseek-4.1-flash | gemini-3.8-flash | decision_correct | 0.2% | 0.0% to 0.5% | 600 | 593 | 7 | 120 |
| All providers | deepseek-4.1-flash | glm-4.7-flash | fact_f1 | 12.3% | 9.1% to 15.6% | 600 | 563 | 37 | 120 |
| All providers | deepseek-4.1-flash | glm-4.7-flash | exact_world | 29.7% | 25.0% to 34.2% | 600 | 563 | 37 | 120 |
| All providers | deepseek-4.1-flash | glm-4.7-flash | task_success | 29.7% | 25.0% to 34.2% | 600 | 563 | 37 | 120 |
| All providers | deepseek-4.1-flash | glm-4.7-flash | decision_correct | 17.6% | 14.2% to 21.0% | 600 | 563 | 37 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3 | fact_f1 | -6.0% | -8.9% to -3.2% | 600 | 559 | 41 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3 | exact_world | -7.5% | -12.5% to -2.8% | 600 | 559 | 41 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3 | task_success | -7.3% | -12.3% to -2.6% | 600 | 559 | 41 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3 | decision_correct | 3.8% | 2.3% to 5.4% | 600 | 559 | 41 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3-flash | fact_f1 | 12.0% | 9.2% to 14.7% | 600 | 591 | 9 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3-flash | exact_world | 22.5% | 18.3% to 26.8% | 600 | 591 | 9 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3-flash | task_success | 22.7% | 18.4% to 26.9% | 600 | 591 | 9 | 120 |
| All providers | deepseek-4.1-flash | glm-5.3-flash | decision_correct | 1.5% | 0.7% to 2.5% | 600 | 591 | 9 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-luna | fact_f1 | -1.9% | -4.7% to 0.9% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-luna | exact_world | 6.7% | 1.9% to 11.6% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-luna | task_success | 7.7% | 2.9% to 12.6% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-luna | decision_correct | 2.9% | 1.5% to 4.3% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-sol | fact_f1 | -11.6% | -14.0% to -9.3% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-sol | exact_world | -19.2% | -25.0% to -13.2% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-sol | task_success | -19.2% | -25.0% to -13.2% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-sol | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-terra | fact_f1 | -2.4% | -4.8% to 0.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-terra | exact_world | -1.5% | -6.2% to 3.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-terra | task_success | -1.5% | -6.2% to 3.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-5.6-terra | decision_correct | 0.2% | 0.0% to 0.5% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-astra | fact_f1 | -17.9% | -20.4% to -15.5% | 600 | 545 | 55 | 115 |
| All providers | deepseek-4.1-flash | gpt-6-astra | exact_world | -36.0% | -41.5% to -30.1% | 600 | 545 | 55 | 115 |
| All providers | deepseek-4.1-flash | gpt-6-astra | task_success | -36.0% | -41.5% to -30.1% | 600 | 545 | 55 | 115 |
| All providers | deepseek-4.1-flash | gpt-6-astra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 545 | 55 | 115 |
| All providers | deepseek-4.1-flash | gpt-6-luna | fact_f1 | -2.1% | -5.0% to 0.6% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-luna | exact_world | 6.6% | 1.3% to 11.4% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-luna | task_success | 6.7% | 1.5% to 11.5% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-luna | decision_correct | 1.7% | 0.7% to 2.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-sol | fact_f1 | -17.3% | -19.8% to -14.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-sol | exact_world | -25.4% | -30.7% to -19.9% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-sol | task_success | -25.2% | -30.6% to -19.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | gpt-6-sol | decision_correct | 1.3% | 0.5% to 2.3% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | grok-4.6 | fact_f1 | -10.2% | -12.7% to -7.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | grok-4.6 | exact_world | 11.9% | 7.1% to 16.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | grok-4.6 | task_success | 11.9% | 7.1% to 16.7% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | grok-4.6 | decision_correct | 3.2% | 1.3% to 5.2% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | kimi-k2.6 | fact_f1 | -11.2% | -13.4% to -9.0% | 600 | 594 | 6 | 120 |
| All providers | deepseek-4.1-flash | kimi-k2.6 | exact_world | 12.3% | 7.9% to 16.5% | 600 | 594 | 6 | 120 |
| All providers | deepseek-4.1-flash | kimi-k2.6 | task_success | 12.3% | 7.9% to 16.5% | 600 | 594 | 6 | 120 |
| All providers | deepseek-4.1-flash | kimi-k2.6 | decision_correct | 2.2% | 1.0% to 3.6% | 600 | 594 | 6 | 120 |
| All providers | deepseek-4.1-flash | kimi-k3 | fact_f1 | -2.8% | -5.9% to 0.2% | 600 | 440 | 160 | 96 |
| All providers | deepseek-4.1-flash | kimi-k3 | exact_world | 9.5% | 4.0% to 14.7% | 600 | 440 | 160 | 96 |
| All providers | deepseek-4.1-flash | kimi-k3 | task_success | 9.5% | 4.0% to 14.7% | 600 | 440 | 160 | 96 |
| All providers | deepseek-4.1-flash | kimi-k3 | decision_correct | 0.2% | 0.0% to 0.7% | 600 | 440 | 160 | 96 |
| All providers | deepseek-4.1-flash | mimo-v2.6-flash | fact_f1 | -0.1% | -2.7% to 2.3% | 600 | 590 | 10 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-flash | exact_world | 7.6% | 2.9% to 12.2% | 600 | 590 | 10 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-flash | task_success | 9.3% | 4.6% to 13.8% | 600 | 590 | 10 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-flash | decision_correct | 10.2% | 7.5% to 13.0% | 600 | 590 | 10 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-pro | fact_f1 | 2.8% | 0.4% to 5.2% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-pro | exact_world | 10.1% | 6.2% to 14.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-pro | task_success | 11.3% | 7.2% to 15.3% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | mimo-v2.6-pro | decision_correct | 11.4% | 8.7% to 14.3% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | minimax-m3 | fact_f1 | 11.1% | 8.3% to 13.8% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | minimax-m3 | exact_world | 15.0% | 10.3% to 19.4% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | minimax-m3 | task_success | 15.0% | 10.3% to 19.4% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | minimax-m3 | decision_correct | 3.4% | 1.8% to 5.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | qwen3.8-27b | fact_f1 | -4.5% | -7.1% to -2.0% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | qwen3.8-27b | exact_world | 15.1% | 10.4% to 19.9% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | qwen3.8-27b | task_success | 15.1% | 10.4% to 19.9% | 600 | 595 | 5 | 120 |
| All providers | deepseek-4.1-flash | qwen3.8-27b | decision_correct | 2.7% | 1.2% to 4.4% | 600 | 595 | 5 | 120 |
| All providers | deepseek-v4-pro | gemini-3.6-flash | fact_f1 | 12.0% | 9.4% to 14.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.6-flash | exact_world | -1.0% | -5.7% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.6-flash | task_success | -1.0% | -5.7% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.6-flash | decision_correct | -3.7% | -5.7% to -1.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.7-flash | fact_f1 | 0.2% | -1.8% to 2.2% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.7-flash | exact_world | -26.0% | -31.0% to -21.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.7-flash | task_success | -26.0% | -31.0% to -21.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.7-flash | decision_correct | -4.2% | -6.2% to -2.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gemini-3.8-flash | fact_f1 | 5.0% | 2.8% to 7.2% | 600 | 598 | 2 | 120 |
| All providers | deepseek-v4-pro | gemini-3.8-flash | exact_world | -16.4% | -21.3% to -11.7% | 600 | 598 | 2 | 120 |
| All providers | deepseek-v4-pro | gemini-3.8-flash | task_success | -16.4% | -21.3% to -11.7% | 600 | 598 | 2 | 120 |
| All providers | deepseek-v4-pro | gemini-3.8-flash | decision_correct | -4.3% | -6.2% to -2.7% | 600 | 598 | 2 | 120 |
| All providers | deepseek-v4-pro | glm-4.7-flash | fact_f1 | 24.5% | 21.6% to 27.4% | 600 | 566 | 34 | 120 |
| All providers | deepseek-v4-pro | glm-4.7-flash | exact_world | 22.6% | 18.4% to 27.1% | 600 | 566 | 34 | 120 |
| All providers | deepseek-v4-pro | glm-4.7-flash | task_success | 22.6% | 18.4% to 27.1% | 600 | 566 | 34 | 120 |
| All providers | deepseek-v4-pro | glm-4.7-flash | decision_correct | 13.1% | 9.3% to 16.8% | 600 | 566 | 34 | 120 |
| All providers | deepseek-v4-pro | glm-5.3 | fact_f1 | 6.4% | 3.3% to 9.4% | 600 | 564 | 36 | 120 |
| All providers | deepseek-v4-pro | glm-5.3 | exact_world | -13.5% | -18.2% to -9.0% | 600 | 564 | 36 | 120 |
| All providers | deepseek-v4-pro | glm-5.3 | task_success | -13.3% | -18.0% to -8.8% | 600 | 564 | 36 | 120 |
| All providers | deepseek-v4-pro | glm-5.3 | decision_correct | -1.2% | -3.6% to 1.1% | 600 | 564 | 36 | 120 |
| All providers | deepseek-v4-pro | glm-5.3-flash | fact_f1 | 24.3% | 20.8% to 27.9% | 600 | 596 | 4 | 120 |
| All providers | deepseek-v4-pro | glm-5.3-flash | exact_world | 16.6% | 12.6% to 20.7% | 600 | 596 | 4 | 120 |
| All providers | deepseek-v4-pro | glm-5.3-flash | task_success | 16.8% | 12.8% to 20.8% | 600 | 596 | 4 | 120 |
| All providers | deepseek-v4-pro | glm-5.3-flash | decision_correct | -3.0% | -5.4% to -0.8% | 600 | 596 | 4 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-luna | fact_f1 | 10.1% | 8.0% to 12.2% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-luna | exact_world | 0.0% | -4.0% to 4.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-luna | task_success | 1.0% | -3.2% to 5.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-luna | decision_correct | -1.8% | -4.2% to 0.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-sol | fact_f1 | 0.5% | -1.7% to 2.6% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-sol | exact_world | -25.8% | -31.3% to -20.2% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-sol | task_success | -25.8% | -31.3% to -20.2% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-sol | decision_correct | -4.7% | -6.7% to -2.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-terra | fact_f1 | 9.7% | 7.4% to 12.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-terra | exact_world | -8.0% | -12.5% to -3.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-terra | task_success | -8.0% | -12.5% to -3.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-5.6-terra | decision_correct | -4.5% | -6.5% to -2.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-astra | fact_f1 | -6.9% | -8.3% to -5.6% | 600 | 550 | 50 | 115 |
| All providers | deepseek-v4-pro | gpt-6-astra | exact_world | -43.5% | -48.7% to -37.9% | 600 | 550 | 50 | 115 |
| All providers | deepseek-v4-pro | gpt-6-astra | task_success | -43.5% | -48.7% to -37.9% | 600 | 550 | 50 | 115 |
| All providers | deepseek-v4-pro | gpt-6-astra | decision_correct | -4.4% | -6.3% to -2.7% | 600 | 550 | 50 | 115 |
| All providers | deepseek-v4-pro | gpt-6-luna | fact_f1 | 10.0% | 7.5% to 12.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-luna | exact_world | 0.2% | -4.5% to 4.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-luna | task_success | 0.3% | -4.3% to 5.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-luna | decision_correct | -3.0% | -5.0% to -1.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-sol | fact_f1 | -5.2% | -6.7% to -3.6% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-sol | exact_world | -31.7% | -36.8% to -26.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-sol | task_success | -31.5% | -36.7% to -26.2% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | gpt-6-sol | decision_correct | -3.2% | -5.2% to -1.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | grok-4.6 | fact_f1 | 2.0% | 0.3% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | grok-4.6 | exact_world | 5.7% | 0.5% to 10.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | grok-4.6 | task_success | 5.7% | 0.5% to 10.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | grok-4.6 | decision_correct | -1.3% | -3.7% to 0.8% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | kimi-k2.6 | fact_f1 | 0.9% | -0.6% to 2.5% | 600 | 599 | 1 | 120 |
| All providers | deepseek-v4-pro | kimi-k2.6 | exact_world | 6.0% | 2.0% to 10.2% | 600 | 599 | 1 | 120 |
| All providers | deepseek-v4-pro | kimi-k2.6 | task_success | 6.0% | 2.0% to 10.2% | 600 | 599 | 1 | 120 |
| All providers | deepseek-v4-pro | kimi-k2.6 | decision_correct | -2.5% | -4.2% to -1.0% | 600 | 599 | 1 | 120 |
| All providers | deepseek-v4-pro | kimi-k3 | fact_f1 | 6.9% | 3.9% to 10.0% | 600 | 443 | 157 | 96 |
| All providers | deepseek-v4-pro | kimi-k3 | exact_world | 2.7% | -2.6% to 8.0% | 600 | 443 | 157 | 96 |
| All providers | deepseek-v4-pro | kimi-k3 | task_success | 2.7% | -2.6% to 8.0% | 600 | 443 | 157 | 96 |
| All providers | deepseek-v4-pro | kimi-k3 | decision_correct | -4.3% | -6.5% to -2.3% | 600 | 443 | 157 | 96 |
| All providers | deepseek-v4-pro | mimo-v2.6-flash | fact_f1 | 12.1% | 9.7% to 14.6% | 600 | 594 | 6 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-flash | exact_world | 1.9% | -2.2% to 6.0% | 600 | 594 | 6 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-flash | task_success | 3.5% | -0.5% to 7.8% | 600 | 594 | 6 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-flash | decision_correct | 5.4% | 2.2% to 8.7% | 600 | 594 | 6 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-pro | fact_f1 | 14.9% | 12.3% to 17.6% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-pro | exact_world | 3.5% | -0.2% to 7.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-pro | task_success | 4.7% | 0.7% to 9.0% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | mimo-v2.6-pro | decision_correct | 6.8% | 3.3% to 10.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | minimax-m3 | fact_f1 | 23.0% | 19.8% to 26.3% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | minimax-m3 | exact_world | 8.8% | 5.0% to 12.7% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | minimax-m3 | task_success | 8.8% | 5.0% to 12.7% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | minimax-m3 | decision_correct | -1.3% | -3.5% to 0.7% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | qwen3.8-27b | fact_f1 | 7.8% | 5.4% to 10.4% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | qwen3.8-27b | exact_world | 9.0% | 4.7% to 13.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | qwen3.8-27b | task_success | 9.0% | 4.7% to 13.5% | 600 | 600 | 0 | 120 |
| All providers | deepseek-v4-pro | qwen3.8-27b | decision_correct | -1.7% | -4.0% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gemini-3.7-flash | fact_f1 | -11.8% | -14.1% to -9.6% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gemini-3.7-flash | exact_world | -25.0% | -29.8% to -20.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gemini-3.7-flash | task_success | -25.0% | -29.8% to -20.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gemini-3.7-flash | decision_correct | -0.5% | -1.2% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gemini-3.8-flash | fact_f1 | -7.0% | -9.0% to -5.1% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.6-flash | gemini-3.8-flash | exact_world | -15.6% | -20.0% to -10.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.6-flash | gemini-3.8-flash | task_success | -15.6% | -20.0% to -10.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.6-flash | gemini-3.8-flash | decision_correct | -0.7% | -1.5% to 0.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.6-flash | glm-4.7-flash | fact_f1 | 13.3% | 10.2% to 16.4% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.6-flash | glm-4.7-flash | exact_world | 24.7% | 20.7% to 29.0% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.6-flash | glm-4.7-flash | task_success | 24.7% | 20.7% to 29.0% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.6-flash | glm-4.7-flash | decision_correct | 17.0% | 13.6% to 20.3% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.6-flash | glm-5.3 | fact_f1 | -5.3% | -8.3% to -2.3% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.6-flash | glm-5.3 | exact_world | -11.9% | -17.2% to -6.9% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.6-flash | glm-5.3 | task_success | -11.7% | -17.0% to -6.8% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.6-flash | glm-5.3 | decision_correct | 2.7% | 0.7% to 4.5% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.6-flash | glm-5.3-flash | fact_f1 | 12.2% | 10.0% to 14.5% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.6-flash | glm-5.3-flash | exact_world | 17.4% | 13.1% to 22.0% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.6-flash | glm-5.3-flash | task_success | 17.6% | 13.3% to 22.1% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.6-flash | glm-5.3-flash | decision_correct | 0.7% | -0.8% to 2.0% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-luna | fact_f1 | -1.9% | -4.9% to 1.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-luna | exact_world | 1.0% | -4.0% to 6.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-luna | task_success | 2.0% | -3.0% to 7.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-luna | decision_correct | 1.8% | 0.3% to 3.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-sol | fact_f1 | -11.5% | -14.1% to -8.9% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-sol | exact_world | -24.8% | -30.8% to -18.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-sol | task_success | -24.8% | -30.8% to -18.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-sol | decision_correct | -1.0% | -2.2% to -0.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-terra | fact_f1 | -2.3% | -4.8% to 0.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-terra | exact_world | -7.0% | -12.0% to -2.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-terra | task_success | -7.0% | -12.0% to -2.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-5.6-terra | decision_correct | -0.8% | -1.7% to -0.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-astra | fact_f1 | -17.1% | -19.8% to -14.3% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.6-flash | gpt-6-astra | exact_world | -41.3% | -47.5% to -34.4% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.6-flash | gpt-6-astra | task_success | -41.3% | -47.5% to -34.4% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.6-flash | gpt-6-astra | decision_correct | -0.9% | -1.9% to -0.2% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.6-flash | gpt-6-luna | fact_f1 | -2.0% | -4.8% to 0.8% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-luna | exact_world | 1.2% | -3.8% to 6.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-luna | task_success | 1.3% | -3.7% to 6.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-luna | decision_correct | 0.7% | -0.8% to 2.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-sol | fact_f1 | -17.2% | -20.0% to -14.4% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-sol | exact_world | -30.7% | -36.7% to -24.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-sol | task_success | -30.5% | -36.5% to -24.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | gpt-6-sol | decision_correct | 0.5% | -0.7% to 1.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | grok-4.6 | fact_f1 | -10.0% | -12.8% to -7.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | grok-4.6 | exact_world | 6.7% | 1.2% to 12.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | grok-4.6 | task_success | 6.7% | 1.2% to 12.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | grok-4.6 | decision_correct | 2.3% | 1.0% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | kimi-k2.6 | fact_f1 | -11.1% | -13.7% to -8.6% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.6-flash | kimi-k2.6 | exact_world | 7.0% | 3.0% to 11.3% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.6-flash | kimi-k2.6 | task_success | 7.0% | 3.0% to 11.3% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.6-flash | kimi-k2.6 | decision_correct | 1.2% | -0.3% to 2.8% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.6-flash | kimi-k3 | fact_f1 | -1.1% | -3.6% to 1.2% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.6-flash | kimi-k3 | exact_world | 5.0% | -0.2% to 10.2% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.6-flash | kimi-k3 | task_success | 5.0% | -0.2% to 10.2% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.6-flash | kimi-k3 | decision_correct | -0.9% | -2.1% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.6-flash | mimo-v2.6-flash | fact_f1 | 0.0% | -2.4% to 2.3% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-flash | exact_world | 2.7% | -1.3% to 6.7% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-flash | task_success | 4.4% | 0.5% to 8.4% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-flash | decision_correct | 9.1% | 6.4% to 12.1% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-pro | fact_f1 | 2.9% | 0.4% to 5.6% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-pro | exact_world | 4.5% | 0.0% to 9.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-pro | task_success | 5.7% | 1.3% to 10.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | mimo-v2.6-pro | decision_correct | 10.5% | 7.5% to 13.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | minimax-m3 | fact_f1 | 11.0% | 7.9% to 14.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | minimax-m3 | exact_world | 9.8% | 5.0% to 14.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | minimax-m3 | task_success | 9.8% | 5.0% to 14.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | minimax-m3 | decision_correct | 2.3% | 0.8% to 3.8% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | qwen3.8-27b | fact_f1 | -4.1% | -6.6% to -1.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | qwen3.8-27b | exact_world | 10.0% | 5.7% to 14.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | qwen3.8-27b | task_success | 10.0% | 5.7% to 14.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.6-flash | qwen3.8-27b | decision_correct | 2.0% | 0.7% to 3.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gemini-3.8-flash | fact_f1 | 4.8% | 3.3% to 6.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.7-flash | gemini-3.8-flash | exact_world | 9.7% | 5.7% to 13.5% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.7-flash | gemini-3.8-flash | task_success | 9.7% | 5.7% to 13.5% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.7-flash | gemini-3.8-flash | decision_correct | -0.2% | -0.5% to 0.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.7-flash | glm-4.7-flash | fact_f1 | 24.9% | 22.3% to 27.6% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.7-flash | glm-4.7-flash | exact_world | 49.5% | 45.1% to 53.9% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.7-flash | glm-4.7-flash | task_success | 49.5% | 45.1% to 53.9% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.7-flash | glm-4.7-flash | decision_correct | 17.3% | 14.0% to 20.6% | 600 | 566 | 34 | 120 |
| All providers | gemini-3.7-flash | glm-5.3 | fact_f1 | 6.1% | 3.4% to 8.9% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.7-flash | glm-5.3 | exact_world | 11.7% | 6.0% to 16.9% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.7-flash | glm-5.3 | task_success | 11.9% | 6.2% to 17.1% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.7-flash | glm-5.3 | decision_correct | 3.2% | 1.6% to 5.0% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.7-flash | glm-5.3-flash | fact_f1 | 24.1% | 21.2% to 27.1% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.7-flash | glm-5.3-flash | exact_world | 42.6% | 38.3% to 47.2% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.7-flash | glm-5.3-flash | task_success | 42.8% | 38.4% to 47.3% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.7-flash | glm-5.3-flash | decision_correct | 1.2% | -0.2% to 2.5% | 600 | 596 | 4 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-luna | fact_f1 | 9.9% | 7.4% to 12.6% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-luna | exact_world | 26.0% | 20.8% to 31.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-luna | task_success | 27.0% | 21.8% to 32.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-luna | decision_correct | 2.3% | 1.2% to 3.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-sol | fact_f1 | 0.3% | -1.7% to 2.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-sol | exact_world | 0.2% | -5.7% to 6.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-sol | task_success | 0.2% | -5.7% to 6.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-sol | decision_correct | -0.5% | -1.3% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-terra | fact_f1 | 9.6% | 7.5% to 11.8% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-terra | exact_world | 18.0% | 13.3% to 22.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-terra | task_success | 18.0% | 13.3% to 22.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-5.6-terra | decision_correct | -0.3% | -0.8% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-astra | fact_f1 | -6.6% | -8.3% to -4.8% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.7-flash | gpt-6-astra | exact_world | -18.0% | -23.7% to -12.0% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.7-flash | gpt-6-astra | task_success | -18.0% | -23.7% to -12.0% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.7-flash | gpt-6-astra | decision_correct | -0.4% | -0.9% to 0.0% | 600 | 550 | 50 | 115 |
| All providers | gemini-3.7-flash | gpt-6-luna | fact_f1 | 9.8% | 7.5% to 12.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-luna | exact_world | 26.2% | 21.2% to 31.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-luna | task_success | 26.3% | 21.3% to 31.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-luna | decision_correct | 1.2% | 0.0% to 2.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-sol | fact_f1 | -5.3% | -7.3% to -3.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-sol | exact_world | -5.7% | -11.5% to 0.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-sol | task_success | -5.5% | -11.3% to 0.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | gpt-6-sol | decision_correct | 1.0% | 0.2% to 2.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | grok-4.6 | fact_f1 | 1.8% | 0.0% to 3.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | grok-4.6 | exact_world | 31.7% | 27.0% to 36.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | grok-4.6 | task_success | 31.7% | 27.0% to 36.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | grok-4.6 | decision_correct | 2.8% | 1.2% to 4.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | kimi-k2.6 | fact_f1 | 0.7% | -1.1% to 2.6% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.7-flash | kimi-k2.6 | exact_world | 32.1% | 27.7% to 36.6% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.7-flash | kimi-k2.6 | task_success | 32.1% | 27.7% to 36.6% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.7-flash | kimi-k2.6 | decision_correct | 1.7% | 0.3% to 3.2% | 600 | 599 | 1 | 120 |
| All providers | gemini-3.7-flash | kimi-k3 | fact_f1 | 8.3% | 5.9% to 10.7% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.7-flash | kimi-k3 | exact_world | 29.1% | 23.7% to 34.5% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.7-flash | kimi-k3 | task_success | 29.1% | 23.7% to 34.5% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.7-flash | kimi-k3 | decision_correct | -0.5% | -1.1% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | gemini-3.7-flash | mimo-v2.6-flash | fact_f1 | 11.9% | 9.8% to 14.0% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-flash | exact_world | 27.6% | 22.7% to 32.5% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-flash | task_success | 29.3% | 24.6% to 34.0% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-flash | decision_correct | 9.6% | 7.0% to 12.5% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-pro | fact_f1 | 14.7% | 12.4% to 17.1% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-pro | exact_world | 29.5% | 24.8% to 34.5% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-pro | task_success | 30.7% | 25.8% to 35.8% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | mimo-v2.6-pro | decision_correct | 11.0% | 8.0% to 14.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | minimax-m3 | fact_f1 | 22.9% | 19.9% to 25.8% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | minimax-m3 | exact_world | 34.8% | 30.7% to 39.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | minimax-m3 | task_success | 34.8% | 30.7% to 39.2% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | minimax-m3 | decision_correct | 2.8% | 1.5% to 4.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | qwen3.8-27b | fact_f1 | 7.7% | 5.8% to 9.7% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | qwen3.8-27b | exact_world | 35.0% | 30.7% to 39.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | qwen3.8-27b | task_success | 35.0% | 30.7% to 39.3% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.7-flash | qwen3.8-27b | decision_correct | 2.5% | 1.2% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | gemini-3.8-flash | glm-4.7-flash | fact_f1 | 20.5% | 17.7% to 23.5% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.8-flash | glm-4.7-flash | exact_world | 40.2% | 35.6% to 45.1% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.8-flash | glm-4.7-flash | task_success | 40.2% | 35.6% to 45.1% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.8-flash | glm-4.7-flash | decision_correct | 17.4% | 14.1% to 20.7% | 600 | 564 | 36 | 120 |
| All providers | gemini-3.8-flash | glm-5.3 | fact_f1 | 1.3% | -1.4% to 3.9% | 600 | 562 | 38 | 120 |
| All providers | gemini-3.8-flash | glm-5.3 | exact_world | 2.7% | -3.0% to 8.3% | 600 | 562 | 38 | 120 |
| All providers | gemini-3.8-flash | glm-5.3 | task_success | 2.8% | -2.9% to 8.6% | 600 | 562 | 38 | 120 |
| All providers | gemini-3.8-flash | glm-5.3 | decision_correct | 3.4% | 1.8% to 5.0% | 600 | 562 | 38 | 120 |
| All providers | gemini-3.8-flash | glm-5.3-flash | fact_f1 | 19.2% | 16.7% to 21.8% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.8-flash | glm-5.3-flash | exact_world | 33.0% | 28.3% to 37.8% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.8-flash | glm-5.3-flash | task_success | 33.2% | 28.6% to 37.9% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.8-flash | glm-5.3-flash | decision_correct | 1.3% | 0.3% to 2.5% | 600 | 594 | 6 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-luna | fact_f1 | 5.1% | 2.7% to 7.6% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-luna | exact_world | 16.6% | 11.4% to 21.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-luna | task_success | 17.6% | 12.5% to 22.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-luna | decision_correct | 2.5% | 1.3% to 3.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-sol | fact_f1 | -4.6% | -6.5% to -2.5% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-sol | exact_world | -9.4% | -15.2% to -2.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-sol | task_success | -9.4% | -15.2% to -2.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-sol | decision_correct | -0.3% | -0.8% to 0.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-terra | fact_f1 | 4.7% | 2.5% to 6.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-terra | exact_world | 8.5% | 3.7% to 13.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-terra | task_success | 8.5% | 3.7% to 13.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-5.6-terra | decision_correct | -0.2% | -0.5% to 0.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-astra | fact_f1 | -11.2% | -13.3% to -9.0% | 600 | 548 | 52 | 115 |
| All providers | gemini-3.8-flash | gpt-6-astra | exact_world | -26.5% | -32.2% to -20.6% | 600 | 548 | 52 | 115 |
| All providers | gemini-3.8-flash | gpt-6-astra | task_success | -26.5% | -32.2% to -20.6% | 600 | 548 | 52 | 115 |
| All providers | gemini-3.8-flash | gpt-6-astra | decision_correct | -0.4% | -0.9% to 0.0% | 600 | 548 | 52 | 115 |
| All providers | gemini-3.8-flash | gpt-6-luna | fact_f1 | 5.0% | 2.6% to 7.3% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-luna | exact_world | 16.6% | 11.5% to 21.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-luna | task_success | 16.7% | 11.7% to 21.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-luna | decision_correct | 1.3% | 0.3% to 2.3% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-sol | fact_f1 | -10.1% | -12.4% to -7.8% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-sol | exact_world | -15.2% | -21.0% to -9.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-sol | task_success | -15.1% | -20.7% to -9.2% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | gpt-6-sol | decision_correct | 1.2% | 0.3% to 2.2% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | grok-4.6 | fact_f1 | -2.9% | -5.2% to -0.7% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | grok-4.6 | exact_world | 22.1% | 16.7% to 27.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | grok-4.6 | task_success | 22.1% | 16.7% to 27.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | grok-4.6 | decision_correct | 3.0% | 1.3% to 5.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | kimi-k2.6 | fact_f1 | -4.1% | -6.1% to -2.1% | 600 | 597 | 3 | 120 |
| All providers | gemini-3.8-flash | kimi-k2.6 | exact_world | 22.6% | 18.2% to 27.1% | 600 | 597 | 3 | 120 |
| All providers | gemini-3.8-flash | kimi-k2.6 | task_success | 22.6% | 18.2% to 27.1% | 600 | 597 | 3 | 120 |
| All providers | gemini-3.8-flash | kimi-k2.6 | decision_correct | 1.8% | 0.7% to 3.3% | 600 | 597 | 3 | 120 |
| All providers | gemini-3.8-flash | kimi-k3 | fact_f1 | 4.3% | 2.2% to 6.7% | 600 | 441 | 159 | 96 |
| All providers | gemini-3.8-flash | kimi-k3 | exact_world | 19.7% | 14.3% to 25.1% | 600 | 441 | 159 | 96 |
| All providers | gemini-3.8-flash | kimi-k3 | task_success | 19.7% | 14.3% to 25.1% | 600 | 441 | 159 | 96 |
| All providers | gemini-3.8-flash | kimi-k3 | decision_correct | -0.2% | -0.7% to 0.0% | 600 | 441 | 159 | 96 |
| All providers | gemini-3.8-flash | mimo-v2.6-flash | fact_f1 | 7.0% | 5.2% to 8.8% | 600 | 592 | 8 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-flash | exact_world | 18.2% | 13.8% to 22.7% | 600 | 592 | 8 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-flash | task_success | 19.9% | 15.5% to 24.4% | 600 | 592 | 8 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-flash | decision_correct | 9.8% | 7.2% to 12.6% | 600 | 592 | 8 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-pro | fact_f1 | 10.0% | 7.6% to 12.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-pro | exact_world | 20.2% | 15.7% to 24.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-pro | task_success | 21.4% | 16.8% to 26.1% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | mimo-v2.6-pro | decision_correct | 11.2% | 8.4% to 14.2% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | minimax-m3 | fact_f1 | 18.1% | 15.3% to 20.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | minimax-m3 | exact_world | 25.4% | 21.2% to 29.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | minimax-m3 | task_success | 25.4% | 21.2% to 29.9% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | minimax-m3 | decision_correct | 3.0% | 1.7% to 4.5% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | qwen3.8-27b | fact_f1 | 2.9% | 1.0% to 5.0% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | qwen3.8-27b | exact_world | 25.4% | 20.7% to 30.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | qwen3.8-27b | task_success | 25.4% | 20.7% to 30.4% | 600 | 598 | 2 | 120 |
| All providers | gemini-3.8-flash | qwen3.8-27b | decision_correct | 2.7% | 1.3% to 4.3% | 600 | 598 | 2 | 120 |
| All providers | glm-4.7-flash | glm-5.3 | fact_f1 | -17.8% | -20.9% to -14.4% | 600 | 531 | 69 | 120 |
| All providers | glm-4.7-flash | glm-5.3 | exact_world | -36.0% | -40.5% to -31.6% | 600 | 531 | 69 | 120 |
| All providers | glm-4.7-flash | glm-5.3 | task_success | -35.8% | -40.3% to -31.4% | 600 | 531 | 69 | 120 |
| All providers | glm-4.7-flash | glm-5.3 | decision_correct | -13.4% | -17.3% to -9.4% | 600 | 531 | 69 | 120 |
| All providers | glm-4.7-flash | glm-5.3-flash | fact_f1 | -0.5% | -4.1% to 2.9% | 600 | 562 | 38 | 120 |
| All providers | glm-4.7-flash | glm-5.3-flash | exact_world | -6.8% | -10.1% to -3.6% | 600 | 562 | 38 | 120 |
| All providers | glm-4.7-flash | glm-5.3-flash | task_success | -6.6% | -9.9% to -3.4% | 600 | 562 | 38 | 120 |
| All providers | glm-4.7-flash | glm-5.3-flash | decision_correct | -15.7% | -19.1% to -12.1% | 600 | 562 | 38 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-luna | fact_f1 | -14.3% | -17.6% to -10.9% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-luna | exact_world | -23.3% | -27.3% to -19.4% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-luna | task_success | -22.3% | -26.2% to -18.4% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-luna | decision_correct | -15.2% | -18.7% to -11.9% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-sol | fact_f1 | -24.1% | -27.2% to -21.1% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-sol | exact_world | -47.9% | -52.8% to -42.6% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-sol | task_success | -47.9% | -52.8% to -42.6% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-sol | decision_correct | -17.5% | -20.9% to -14.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-terra | fact_f1 | -15.1% | -18.4% to -11.9% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-terra | exact_world | -31.6% | -36.1% to -27.3% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-terra | task_success | -31.6% | -36.1% to -27.3% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-5.6-terra | decision_correct | -17.5% | -20.9% to -14.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-astra | fact_f1 | -30.2% | -33.0% to -27.4% | 600 | 520 | 80 | 115 |
| All providers | glm-4.7-flash | gpt-6-astra | exact_world | -65.6% | -70.4% to -60.3% | 600 | 520 | 80 | 115 |
| All providers | glm-4.7-flash | gpt-6-astra | task_success | -65.6% | -70.4% to -60.3% | 600 | 520 | 80 | 115 |
| All providers | glm-4.7-flash | gpt-6-astra | decision_correct | -17.9% | -21.6% to -14.5% | 600 | 520 | 80 | 115 |
| All providers | glm-4.7-flash | gpt-6-luna | fact_f1 | -15.2% | -18.5% to -11.9% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-luna | exact_world | -23.1% | -27.2% to -19.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-luna | task_success | -23.0% | -27.0% to -19.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-luna | decision_correct | -15.9% | -19.4% to -12.5% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-sol | fact_f1 | -29.5% | -32.4% to -26.5% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-sol | exact_world | -53.5% | -58.6% to -48.3% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-sol | task_success | -53.5% | -58.6% to -48.3% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | gpt-6-sol | decision_correct | -16.3% | -19.5% to -13.0% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | grok-4.6 | fact_f1 | -22.6% | -25.4% to -19.7% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | grok-4.6 | exact_world | -17.5% | -22.1% to -13.0% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | grok-4.6 | task_success | -17.5% | -22.1% to -13.0% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | grok-4.6 | decision_correct | -14.5% | -18.0% to -10.7% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | kimi-k2.6 | fact_f1 | -23.8% | -26.6% to -20.9% | 600 | 565 | 35 | 120 |
| All providers | glm-4.7-flash | kimi-k2.6 | exact_world | -17.2% | -21.1% to -13.4% | 600 | 565 | 35 | 120 |
| All providers | glm-4.7-flash | kimi-k2.6 | task_success | -17.2% | -21.1% to -13.4% | 600 | 565 | 35 | 120 |
| All providers | glm-4.7-flash | kimi-k2.6 | decision_correct | -15.4% | -18.9% to -12.1% | 600 | 565 | 35 | 120 |
| All providers | glm-4.7-flash | kimi-k3 | fact_f1 | -16.0% | -20.3% to -12.0% | 600 | 417 | 183 | 96 |
| All providers | glm-4.7-flash | kimi-k3 | exact_world | -21.6% | -26.5% to -16.5% | 600 | 417 | 183 | 96 |
| All providers | glm-4.7-flash | kimi-k3 | task_success | -21.6% | -26.5% to -16.5% | 600 | 417 | 183 | 96 |
| All providers | glm-4.7-flash | kimi-k3 | decision_correct | -18.0% | -22.2% to -14.0% | 600 | 417 | 183 | 96 |
| All providers | glm-4.7-flash | mimo-v2.6-flash | fact_f1 | -12.9% | -16.0% to -9.8% | 600 | 561 | 39 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-flash | exact_world | -21.2% | -25.4% to -17.5% | 600 | 561 | 39 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-flash | task_success | -19.4% | -23.5% to -15.8% | 600 | 561 | 39 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-flash | decision_correct | -8.4% | -12.4% to -4.5% | 600 | 561 | 39 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-pro | fact_f1 | -10.0% | -13.3% to -6.8% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-pro | exact_world | -19.6% | -23.7% to -15.7% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-pro | task_success | -18.4% | -22.6% to -14.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | mimo-v2.6-pro | decision_correct | -6.4% | -11.1% to -1.7% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | minimax-m3 | fact_f1 | -2.2% | -6.1% to 1.5% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | minimax-m3 | exact_world | -14.3% | -18.1% to -10.6% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | minimax-m3 | task_success | -14.3% | -18.1% to -10.6% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | minimax-m3 | decision_correct | -14.8% | -18.3% to -11.4% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | qwen3.8-27b | fact_f1 | -16.8% | -19.6% to -13.9% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | qwen3.8-27b | exact_world | -14.1% | -18.3% to -10.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | qwen3.8-27b | task_success | -14.1% | -18.3% to -10.2% | 600 | 566 | 34 | 120 |
| All providers | glm-4.7-flash | qwen3.8-27b | decision_correct | -15.0% | -18.5% to -11.5% | 600 | 566 | 34 | 120 |
| All providers | glm-5.3 | glm-5.3-flash | fact_f1 | 17.3% | 14.4% to 20.3% | 600 | 560 | 40 | 120 |
| All providers | glm-5.3 | glm-5.3-flash | exact_world | 29.1% | 24.6% to 34.2% | 600 | 560 | 40 | 120 |
| All providers | glm-5.3 | glm-5.3-flash | task_success | 29.1% | 24.6% to 34.1% | 600 | 560 | 40 | 120 |
| All providers | glm-5.3 | glm-5.3-flash | decision_correct | -2.0% | -4.1% to 0.0% | 600 | 560 | 40 | 120 |
| All providers | glm-5.3 | gpt-5.6-luna | fact_f1 | 3.5% | 0.3% to 7.0% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-luna | exact_world | 13.1% | 7.8% to 18.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-luna | task_success | 14.0% | 8.8% to 19.4% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-luna | decision_correct | -0.7% | -2.7% to 1.2% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-sol | fact_f1 | -5.9% | -8.7% to -3.2% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-sol | exact_world | -11.7% | -17.4% to -5.2% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-sol | task_success | -11.9% | -17.6% to -5.3% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-sol | decision_correct | -3.7% | -5.4% to -2.3% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-terra | fact_f1 | 3.4% | 0.3% to 6.4% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-terra | exact_world | 5.3% | 0.4% to 10.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-terra | task_success | 5.1% | 0.2% to 10.4% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-5.6-terra | decision_correct | -3.5% | -5.2% to -2.0% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-astra | fact_f1 | -12.0% | -15.0% to -9.0% | 600 | 517 | 83 | 115 |
| All providers | glm-5.3 | gpt-6-astra | exact_world | -28.8% | -35.1% to -21.9% | 600 | 517 | 83 | 115 |
| All providers | glm-5.3 | gpt-6-astra | task_success | -29.0% | -35.3% to -22.1% | 600 | 517 | 83 | 115 |
| All providers | glm-5.3 | gpt-6-astra | decision_correct | -3.7% | -5.4% to -2.1% | 600 | 517 | 83 | 115 |
| All providers | glm-5.3 | gpt-6-luna | fact_f1 | 3.5% | 0.1% to 6.7% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-luna | exact_world | 14.0% | 8.7% to 19.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-luna | task_success | 14.0% | 8.7% to 19.7% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-luna | decision_correct | -2.3% | -4.0% to -0.7% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-sol | fact_f1 | -11.5% | -14.6% to -8.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-sol | exact_world | -18.4% | -24.0% to -12.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-sol | task_success | -18.4% | -23.9% to -12.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | gpt-6-sol | decision_correct | -2.3% | -4.0% to -0.7% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | grok-4.6 | fact_f1 | -4.4% | -7.5% to -1.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | grok-4.6 | exact_world | 18.8% | 12.9% to 24.9% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | grok-4.6 | task_success | 18.6% | 12.8% to 24.6% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | grok-4.6 | decision_correct | -0.4% | -2.7% to 2.1% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | kimi-k2.6 | fact_f1 | -5.5% | -8.3% to -2.8% | 600 | 563 | 37 | 120 |
| All providers | glm-5.3 | kimi-k2.6 | exact_world | 18.7% | 13.4% to 24.2% | 600 | 563 | 37 | 120 |
| All providers | glm-5.3 | kimi-k2.6 | task_success | 18.5% | 13.1% to 24.1% | 600 | 563 | 37 | 120 |
| All providers | glm-5.3 | kimi-k2.6 | decision_correct | -1.6% | -3.6% to 0.2% | 600 | 563 | 37 | 120 |
| All providers | glm-5.3 | kimi-k3 | fact_f1 | 4.0% | 1.6% to 6.7% | 600 | 419 | 181 | 94 |
| All providers | glm-5.3 | kimi-k3 | exact_world | 17.2% | 12.0% to 23.0% | 600 | 419 | 181 | 94 |
| All providers | glm-5.3 | kimi-k3 | task_success | 16.9% | 11.7% to 22.7% | 600 | 419 | 181 | 94 |
| All providers | glm-5.3 | kimi-k3 | decision_correct | -3.3% | -5.4% to -1.7% | 600 | 419 | 181 | 94 |
| All providers | glm-5.3 | mimo-v2.6-flash | fact_f1 | 5.8% | 2.7% to 8.9% | 600 | 559 | 41 | 120 |
| All providers | glm-5.3 | mimo-v2.6-flash | exact_world | 15.0% | 10.1% to 20.0% | 600 | 559 | 41 | 120 |
| All providers | glm-5.3 | mimo-v2.6-flash | task_success | 16.6% | 11.7% to 21.6% | 600 | 559 | 41 | 120 |
| All providers | glm-5.3 | mimo-v2.6-flash | decision_correct | 6.4% | 3.5% to 9.6% | 600 | 559 | 41 | 120 |
| All providers | glm-5.3 | mimo-v2.6-pro | fact_f1 | 7.9% | 5.1% to 10.9% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | mimo-v2.6-pro | exact_world | 16.3% | 11.7% to 21.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | mimo-v2.6-pro | task_success | 17.0% | 12.2% to 22.3% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | mimo-v2.6-pro | decision_correct | 7.4% | 4.4% to 10.6% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | minimax-m3 | fact_f1 | 16.6% | 12.9% to 20.6% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | minimax-m3 | exact_world | 22.7% | 18.1% to 27.7% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | minimax-m3 | task_success | 22.5% | 17.9% to 27.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | minimax-m3 | decision_correct | -0.5% | -2.5% to 1.4% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | qwen3.8-27b | fact_f1 | 1.2% | -1.9% to 4.3% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | qwen3.8-27b | exact_world | 21.5% | 16.6% to 26.6% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | qwen3.8-27b | task_success | 21.3% | 16.4% to 26.5% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3 | qwen3.8-27b | decision_correct | -0.5% | -2.9% to 1.9% | 600 | 564 | 36 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-luna | fact_f1 | -14.2% | -17.6% to -10.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-luna | exact_world | -16.6% | -20.9% to -12.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-luna | task_success | -15.8% | -20.1% to -11.5% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-luna | decision_correct | 1.2% | -0.5% to 2.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-sol | fact_f1 | -23.9% | -27.0% to -20.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-sol | exact_world | -42.6% | -47.8% to -37.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-sol | task_success | -42.8% | -48.0% to -37.6% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-sol | decision_correct | -1.7% | -2.7% to -0.8% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-terra | fact_f1 | -14.5% | -17.7% to -11.5% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-terra | exact_world | -24.3% | -28.7% to -19.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-terra | task_success | -24.5% | -28.9% to -20.1% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-5.6-terra | decision_correct | -1.5% | -2.7% to -0.5% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-astra | fact_f1 | -29.3% | -32.7% to -25.8% | 600 | 546 | 54 | 115 |
| All providers | glm-5.3-flash | gpt-6-astra | exact_world | -60.3% | -65.2% to -55.0% | 600 | 546 | 54 | 115 |
| All providers | glm-5.3-flash | gpt-6-astra | task_success | -60.3% | -65.2% to -55.0% | 600 | 546 | 54 | 115 |
| All providers | glm-5.3-flash | gpt-6-astra | decision_correct | -1.6% | -2.8% to -0.7% | 600 | 546 | 54 | 115 |
| All providers | glm-5.3-flash | gpt-6-luna | fact_f1 | -14.2% | -17.5% to -10.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-luna | exact_world | -16.4% | -20.8% to -12.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-luna | task_success | -16.4% | -20.6% to -12.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-luna | decision_correct | 0.0% | -1.3% to 1.3% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-sol | fact_f1 | -29.5% | -33.0% to -25.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-sol | exact_world | -48.3% | -53.4% to -43.0% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-sol | task_success | -48.3% | -53.4% to -43.0% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | gpt-6-sol | decision_correct | -0.2% | -1.7% to 1.3% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | grok-4.6 | fact_f1 | -22.3% | -25.7% to -19.0% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | grok-4.6 | exact_world | -10.9% | -15.5% to -6.7% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | grok-4.6 | task_success | -11.1% | -15.7% to -6.9% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | grok-4.6 | decision_correct | 1.7% | -0.7% to 4.2% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | kimi-k2.6 | fact_f1 | -23.3% | -26.5% to -20.2% | 600 | 595 | 5 | 120 |
| All providers | glm-5.3-flash | kimi-k2.6 | exact_world | -10.6% | -14.8% to -6.4% | 600 | 595 | 5 | 120 |
| All providers | glm-5.3-flash | kimi-k2.6 | task_success | -10.8% | -15.0% to -6.5% | 600 | 595 | 5 | 120 |
| All providers | glm-5.3-flash | kimi-k2.6 | decision_correct | 0.5% | -1.2% to 2.2% | 600 | 595 | 5 | 120 |
| All providers | glm-5.3-flash | kimi-k3 | fact_f1 | -11.5% | -14.5% to -8.6% | 600 | 439 | 161 | 96 |
| All providers | glm-5.3-flash | kimi-k3 | exact_world | -13.7% | -18.0% to -9.2% | 600 | 439 | 161 | 96 |
| All providers | glm-5.3-flash | kimi-k3 | task_success | -13.7% | -18.0% to -9.2% | 600 | 439 | 161 | 96 |
| All providers | glm-5.3-flash | kimi-k3 | decision_correct | -1.1% | -2.4% to 0.0% | 600 | 439 | 161 | 96 |
| All providers | glm-5.3-flash | mimo-v2.6-flash | fact_f1 | -12.3% | -15.2% to -9.5% | 600 | 590 | 10 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-flash | exact_world | -15.1% | -18.8% to -11.7% | 600 | 590 | 10 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-flash | task_success | -13.6% | -17.2% to -10.3% | 600 | 590 | 10 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-flash | decision_correct | 8.3% | 5.6% to 11.2% | 600 | 590 | 10 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-pro | fact_f1 | -9.3% | -11.8% to -6.8% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-pro | exact_world | -12.8% | -16.1% to -9.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-pro | task_success | -11.7% | -15.2% to -8.1% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | mimo-v2.6-pro | decision_correct | 9.9% | 7.0% to 13.1% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | minimax-m3 | fact_f1 | -1.2% | -4.3% to 2.1% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | minimax-m3 | exact_world | -7.9% | -11.4% to -4.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | minimax-m3 | task_success | -8.1% | -11.7% to -4.5% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | minimax-m3 | decision_correct | 1.7% | -0.2% to 3.7% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | qwen3.8-27b | fact_f1 | -16.5% | -19.5% to -13.4% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | qwen3.8-27b | exact_world | -7.7% | -11.6% to -4.0% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | qwen3.8-27b | task_success | -7.9% | -11.9% to -4.2% | 600 | 596 | 4 | 120 |
| All providers | glm-5.3-flash | qwen3.8-27b | decision_correct | 1.3% | -0.7% to 3.5% | 600 | 596 | 4 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-sol | fact_f1 | -9.6% | -12.0% to -7.4% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-sol | exact_world | -25.8% | -30.3% to -21.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-sol | task_success | -26.8% | -31.3% to -22.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-sol | decision_correct | -2.8% | -4.3% to -1.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-terra | fact_f1 | -0.4% | -3.1% to 2.1% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-terra | exact_world | -8.0% | -12.5% to -3.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-terra | task_success | -9.0% | -13.3% to -5.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-5.6-terra | decision_correct | -2.7% | -4.2% to -1.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-astra | fact_f1 | -17.0% | -19.1% to -14.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-luna | gpt-6-astra | exact_world | -44.0% | -48.8% to -39.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-luna | gpt-6-astra | task_success | -44.7% | -49.6% to -39.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-luna | gpt-6-astra | decision_correct | -2.4% | -3.7% to -1.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-luna | gpt-6-luna | fact_f1 | -0.1% | -2.8% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-luna | exact_world | 0.2% | -4.0% to 4.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-luna | task_success | -0.7% | -4.8% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-luna | decision_correct | -1.2% | -2.7% to 0.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-sol | fact_f1 | -15.3% | -17.5% to -13.1% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-sol | exact_world | -31.7% | -36.5% to -26.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-sol | task_success | -32.5% | -37.5% to -27.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | gpt-6-sol | decision_correct | -1.3% | -2.7% to -0.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | grok-4.6 | fact_f1 | -8.1% | -10.4% to -5.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | grok-4.6 | exact_world | 5.7% | 0.7% to 10.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | grok-4.6 | task_success | 4.7% | -0.5% to 9.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | grok-4.6 | decision_correct | 0.5% | -1.5% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | kimi-k2.6 | fact_f1 | -9.2% | -11.3% to -7.1% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-luna | kimi-k2.6 | exact_world | 6.2% | 1.5% to 10.7% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-luna | kimi-k2.6 | task_success | 5.2% | 0.7% to 9.7% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-luna | kimi-k2.6 | decision_correct | -0.7% | -2.5% to 1.0% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-luna | kimi-k3 | fact_f1 | -2.3% | -6.1% to 1.4% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-luna | kimi-k3 | exact_world | 2.0% | -3.9% to 7.9% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-luna | kimi-k3 | task_success | 1.4% | -4.6% to 7.3% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-luna | kimi-k3 | decision_correct | -2.5% | -4.1% to -1.1% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-luna | mimo-v2.6-flash | fact_f1 | 2.0% | -0.9% to 4.7% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-flash | exact_world | 2.0% | -2.7% to 6.4% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-flash | task_success | 2.7% | -2.0% to 7.0% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-flash | decision_correct | 7.2% | 4.7% to 9.9% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-pro | fact_f1 | 4.8% | 1.9% to 7.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-pro | exact_world | 3.5% | -1.2% to 8.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-pro | task_success | 3.7% | -1.0% to 8.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | mimo-v2.6-pro | decision_correct | 8.7% | 5.8% to 11.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | minimax-m3 | fact_f1 | 12.9% | 9.5% to 16.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | minimax-m3 | exact_world | 8.8% | 4.5% to 13.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | minimax-m3 | task_success | 7.8% | 3.7% to 12.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | minimax-m3 | decision_correct | 0.5% | -1.5% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | qwen3.8-27b | fact_f1 | -2.3% | -5.1% to 0.6% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | qwen3.8-27b | exact_world | 9.0% | 4.3% to 13.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | qwen3.8-27b | task_success | 8.0% | 3.2% to 12.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-luna | qwen3.8-27b | decision_correct | 0.2% | -1.7% to 2.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-5.6-terra | fact_f1 | 9.3% | 7.2% to 11.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-5.6-terra | exact_world | 17.8% | 13.3% to 22.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-5.6-terra | task_success | 17.8% | 13.3% to 22.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-5.6-terra | decision_correct | 0.2% | 0.0% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-astra | fact_f1 | -6.9% | -8.7% to -5.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-sol | gpt-6-astra | exact_world | -18.0% | -23.1% to -12.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-sol | gpt-6-astra | task_success | -18.0% | -23.1% to -12.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-sol | gpt-6-astra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-sol | gpt-6-luna | fact_f1 | 9.5% | 7.0% to 11.9% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-luna | exact_world | 26.0% | 21.0% to 30.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-luna | task_success | 26.2% | 21.3% to 31.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-luna | decision_correct | 1.7% | 0.7% to 2.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-sol | fact_f1 | -5.6% | -7.5% to -3.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-sol | exact_world | -5.8% | -10.3% to -1.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-sol | task_success | -5.7% | -10.3% to -1.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | gpt-6-sol | decision_correct | 1.5% | 0.7% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | grok-4.6 | fact_f1 | 1.5% | -0.4% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | grok-4.6 | exact_world | 31.5% | 25.7% to 36.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | grok-4.6 | task_success | 31.5% | 25.7% to 36.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | grok-4.6 | decision_correct | 3.3% | 1.3% to 5.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | kimi-k2.6 | fact_f1 | 0.4% | -1.3% to 2.1% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-sol | kimi-k2.6 | exact_world | 31.9% | 25.5% to 37.7% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-sol | kimi-k2.6 | task_success | 31.9% | 25.5% to 37.7% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-sol | kimi-k2.6 | decision_correct | 2.2% | 1.0% to 3.5% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-sol | kimi-k3 | fact_f1 | 7.6% | 4.6% to 10.4% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-sol | kimi-k3 | exact_world | 25.5% | 18.7% to 32.0% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-sol | kimi-k3 | task_success | 25.5% | 18.7% to 32.0% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-sol | kimi-k3 | decision_correct | 0.2% | 0.0% to 0.7% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-sol | mimo-v2.6-flash | fact_f1 | 11.6% | 9.3% to 14.0% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-flash | exact_world | 27.3% | 21.4% to 32.4% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-flash | task_success | 29.0% | 22.9% to 34.2% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-flash | decision_correct | 10.1% | 7.5% to 12.9% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-pro | fact_f1 | 14.4% | 11.8% to 17.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-pro | exact_world | 29.3% | 24.2% to 34.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-pro | task_success | 30.5% | 25.3% to 35.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | mimo-v2.6-pro | decision_correct | 11.5% | 8.8% to 14.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | minimax-m3 | fact_f1 | 22.6% | 19.5% to 25.6% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | minimax-m3 | exact_world | 34.7% | 28.8% to 40.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | minimax-m3 | task_success | 34.7% | 28.8% to 40.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | minimax-m3 | decision_correct | 3.3% | 1.8% to 5.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | qwen3.8-27b | fact_f1 | 7.4% | 5.1% to 9.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | qwen3.8-27b | exact_world | 34.8% | 28.3% to 40.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | qwen3.8-27b | task_success | 34.8% | 28.3% to 40.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-sol | qwen3.8-27b | decision_correct | 3.0% | 1.5% to 4.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-astra | fact_f1 | -15.6% | -17.9% to -13.4% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-terra | gpt-6-astra | exact_world | -35.1% | -40.0% to -29.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-terra | gpt-6-astra | task_success | -35.1% | -40.0% to -29.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-terra | gpt-6-astra | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 550 | 50 | 115 |
| All providers | gpt-5.6-terra | gpt-6-luna | fact_f1 | 0.3% | -2.1% to 2.6% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-luna | exact_world | 8.2% | 3.7% to 12.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-luna | task_success | 8.3% | 3.8% to 12.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-luna | decision_correct | 1.5% | 0.5% to 2.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-sol | fact_f1 | -14.9% | -17.2% to -12.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-sol | exact_world | -23.7% | -28.5% to -18.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-sol | task_success | -23.5% | -28.3% to -18.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | gpt-6-sol | decision_correct | 1.3% | 0.5% to 2.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | grok-4.6 | fact_f1 | -7.7% | -9.9% to -5.4% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | grok-4.6 | exact_world | 13.7% | 8.7% to 18.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | grok-4.6 | task_success | 13.7% | 8.7% to 18.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | grok-4.6 | decision_correct | 3.2% | 1.3% to 5.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | kimi-k2.6 | fact_f1 | -8.8% | -10.9% to -6.8% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-terra | kimi-k2.6 | exact_world | 14.0% | 9.2% to 19.0% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-terra | kimi-k2.6 | task_success | 14.0% | 9.2% to 19.0% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-terra | kimi-k2.6 | decision_correct | 2.0% | 0.7% to 3.5% | 600 | 599 | 1 | 120 |
| All providers | gpt-5.6-terra | kimi-k3 | fact_f1 | -0.9% | -3.6% to 2.0% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-terra | kimi-k3 | exact_world | 9.3% | 4.0% to 14.8% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-terra | kimi-k3 | task_success | 9.3% | 4.0% to 14.8% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-terra | kimi-k3 | decision_correct | 0.0% | 0.0% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | gpt-5.6-terra | mimo-v2.6-flash | fact_f1 | 2.3% | -0.2% to 4.7% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-flash | exact_world | 9.6% | 5.4% to 13.5% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-flash | task_success | 11.3% | 7.1% to 15.2% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-flash | decision_correct | 9.9% | 7.4% to 12.8% | 600 | 594 | 6 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-pro | fact_f1 | 5.2% | 2.6% to 7.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-pro | exact_world | 11.5% | 7.3% to 15.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-pro | task_success | 12.7% | 8.3% to 17.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | mimo-v2.6-pro | decision_correct | 11.3% | 8.5% to 14.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | minimax-m3 | fact_f1 | 13.3% | 10.3% to 16.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | minimax-m3 | exact_world | 16.8% | 12.0% to 21.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | minimax-m3 | task_success | 16.8% | 12.0% to 21.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | minimax-m3 | decision_correct | 3.2% | 1.8% to 4.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | qwen3.8-27b | fact_f1 | -1.9% | -4.2% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | qwen3.8-27b | exact_world | 17.0% | 12.3% to 21.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | qwen3.8-27b | task_success | 17.0% | 12.3% to 21.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-5.6-terra | qwen3.8-27b | decision_correct | 2.8% | 1.3% to 4.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-astra | gpt-6-luna | fact_f1 | 16.3% | 14.0% to 18.7% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-luna | exact_world | 44.0% | 38.7% to 48.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-luna | task_success | 44.2% | 39.0% to 49.0% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-luna | decision_correct | 1.5% | 0.5% to 2.5% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-sol | fact_f1 | 2.0% | 1.3% to 2.7% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-sol | exact_world | 13.8% | 9.5% to 18.3% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-sol | task_success | 14.0% | 9.6% to 18.5% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | gpt-6-sol | decision_correct | 1.5% | 0.6% to 2.5% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | grok-4.6 | fact_f1 | 8.8% | 7.6% to 10.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | grok-4.6 | exact_world | 49.8% | 44.6% to 54.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | grok-4.6 | task_success | 49.8% | 44.6% to 54.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | grok-4.6 | decision_correct | 3.5% | 1.5% to 5.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | kimi-k2.6 | fact_f1 | 7.9% | 6.7% to 9.2% | 600 | 549 | 51 | 115 |
| All providers | gpt-6-astra | kimi-k2.6 | exact_world | 49.5% | 43.8% to 55.2% | 600 | 549 | 51 | 115 |
| All providers | gpt-6-astra | kimi-k2.6 | task_success | 49.5% | 43.8% to 55.2% | 600 | 549 | 51 | 115 |
| All providers | gpt-6-astra | kimi-k2.6 | decision_correct | 2.2% | 0.9% to 3.7% | 600 | 549 | 51 | 115 |
| All providers | gpt-6-astra | kimi-k3 | fact_f1 | 13.4% | 10.7% to 16.3% | 600 | 436 | 164 | 95 |
| All providers | gpt-6-astra | kimi-k3 | exact_world | 42.9% | 36.0% to 49.2% | 600 | 436 | 164 | 95 |
| All providers | gpt-6-astra | kimi-k3 | task_success | 42.9% | 36.0% to 49.2% | 600 | 436 | 164 | 95 |
| All providers | gpt-6-astra | kimi-k3 | decision_correct | 0.2% | 0.0% to 0.7% | 600 | 436 | 164 | 95 |
| All providers | gpt-6-astra | mimo-v2.6-flash | fact_f1 | 18.2% | 15.7% to 20.6% | 600 | 544 | 56 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-flash | exact_world | 45.0% | 39.4% to 50.0% | 600 | 544 | 56 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-flash | task_success | 46.9% | 41.2% to 52.0% | 600 | 544 | 56 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-flash | decision_correct | 9.7% | 7.1% to 12.7% | 600 | 544 | 56 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-pro | fact_f1 | 20.5% | 17.9% to 23.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-pro | exact_world | 46.2% | 40.4% to 51.2% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-pro | task_success | 47.3% | 41.2% to 52.5% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | mimo-v2.6-pro | decision_correct | 10.7% | 7.8% to 13.8% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | minimax-m3 | fact_f1 | 28.2% | 24.9% to 31.7% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | minimax-m3 | exact_world | 52.4% | 46.6% to 57.7% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | minimax-m3 | task_success | 52.4% | 46.6% to 57.7% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | minimax-m3 | decision_correct | 3.3% | 1.8% to 4.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | qwen3.8-27b | fact_f1 | 13.8% | 11.6% to 16.1% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | qwen3.8-27b | exact_world | 52.7% | 46.3% to 58.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | qwen3.8-27b | task_success | 52.7% | 46.3% to 58.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-astra | qwen3.8-27b | decision_correct | 3.1% | 1.5% to 4.9% | 600 | 550 | 50 | 115 |
| All providers | gpt-6-luna | gpt-6-sol | fact_f1 | -15.2% | -17.5% to -12.9% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | gpt-6-sol | exact_world | -31.8% | -36.5% to -27.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | gpt-6-sol | task_success | -31.8% | -36.5% to -27.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | gpt-6-sol | decision_correct | -0.2% | -1.5% to 1.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | grok-4.6 | fact_f1 | -8.0% | -10.3% to -5.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | grok-4.6 | exact_world | 5.5% | 0.7% to 10.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | grok-4.6 | task_success | 5.3% | 0.5% to 10.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | grok-4.6 | decision_correct | 1.7% | -0.5% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | kimi-k2.6 | fact_f1 | -9.1% | -11.5% to -6.7% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-luna | kimi-k2.6 | exact_world | 6.0% | 1.2% to 10.5% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-luna | kimi-k2.6 | task_success | 5.8% | 1.0% to 10.4% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-luna | kimi-k2.6 | decision_correct | 0.5% | -1.2% to 2.2% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-luna | kimi-k3 | fact_f1 | -1.1% | -4.5% to 2.1% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-luna | kimi-k3 | exact_world | 2.0% | -4.2% to 8.3% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-luna | kimi-k3 | task_success | 2.0% | -4.2% to 8.3% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-luna | kimi-k3 | decision_correct | -0.5% | -1.4% to 0.4% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-luna | mimo-v2.6-flash | fact_f1 | 2.0% | -0.8% to 4.7% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-flash | exact_world | 1.5% | -3.0% to 5.7% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-flash | task_success | 3.0% | -1.4% to 7.3% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-flash | decision_correct | 8.4% | 5.7% to 11.4% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-pro | fact_f1 | 4.9% | 2.0% to 7.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-pro | exact_world | 3.3% | -1.0% to 7.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-pro | task_success | 4.3% | -0.2% to 9.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | mimo-v2.6-pro | decision_correct | 9.8% | 7.2% to 12.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | minimax-m3 | fact_f1 | 13.0% | 9.8% to 16.4% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | minimax-m3 | exact_world | 8.7% | 4.2% to 13.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | minimax-m3 | task_success | 8.5% | 3.8% to 13.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | minimax-m3 | decision_correct | 1.7% | -0.2% to 3.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | qwen3.8-27b | fact_f1 | -2.2% | -4.8% to 0.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | qwen3.8-27b | exact_world | 8.8% | 4.0% to 13.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | qwen3.8-27b | task_success | 8.7% | 3.8% to 13.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-luna | qwen3.8-27b | decision_correct | 1.3% | -0.5% to 3.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | grok-4.6 | fact_f1 | 7.2% | 5.9% to 8.4% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | grok-4.6 | exact_world | 37.3% | 32.7% to 41.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | grok-4.6 | task_success | 37.2% | 32.3% to 41.7% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | grok-4.6 | decision_correct | 1.8% | 0.0% to 3.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | kimi-k2.6 | fact_f1 | 6.1% | 4.8% to 7.4% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-sol | kimi-k2.6 | exact_world | 37.7% | 32.4% to 43.1% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-sol | kimi-k2.6 | task_success | 37.6% | 32.2% to 42.9% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-sol | kimi-k2.6 | decision_correct | 0.7% | -0.8% to 2.3% | 600 | 599 | 1 | 120 |
| All providers | gpt-6-sol | kimi-k3 | fact_f1 | 11.3% | 8.4% to 14.2% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-sol | kimi-k3 | exact_world | 28.7% | 22.6% to 34.9% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-sol | kimi-k3 | task_success | 28.4% | 22.4% to 34.7% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-sol | kimi-k3 | decision_correct | -1.6% | -2.8% to -0.7% | 600 | 443 | 157 | 96 |
| All providers | gpt-6-sol | mimo-v2.6-flash | fact_f1 | 17.3% | 14.8% to 19.9% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-flash | exact_world | 33.2% | 27.7% to 38.3% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-flash | task_success | 34.7% | 29.0% to 40.1% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-flash | decision_correct | 8.6% | 6.0% to 11.4% | 600 | 594 | 6 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-pro | fact_f1 | 20.0% | 17.3% to 22.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-pro | exact_world | 35.2% | 30.0% to 39.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-pro | task_success | 36.2% | 31.0% to 41.0% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | mimo-v2.6-pro | decision_correct | 10.0% | 7.0% to 13.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | minimax-m3 | fact_f1 | 28.2% | 24.6% to 31.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | minimax-m3 | exact_world | 40.5% | 34.8% to 45.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | minimax-m3 | task_success | 40.3% | 34.7% to 45.8% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | minimax-m3 | decision_correct | 1.8% | 0.3% to 3.3% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | qwen3.8-27b | fact_f1 | 13.0% | 10.8% to 15.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | qwen3.8-27b | exact_world | 40.7% | 34.7% to 46.5% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | qwen3.8-27b | task_success | 40.5% | 34.3% to 46.2% | 600 | 600 | 0 | 120 |
| All providers | gpt-6-sol | qwen3.8-27b | decision_correct | 1.5% | 0.0% to 3.3% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | kimi-k2.6 | fact_f1 | -1.1% | -2.5% to 0.3% | 600 | 599 | 1 | 120 |
| All providers | grok-4.6 | kimi-k2.6 | exact_world | 0.3% | -3.7% to 4.3% | 600 | 599 | 1 | 120 |
| All providers | grok-4.6 | kimi-k2.6 | task_success | 0.3% | -3.7% to 4.3% | 600 | 599 | 1 | 120 |
| All providers | grok-4.6 | kimi-k2.6 | decision_correct | -1.2% | -3.5% to 1.0% | 600 | 599 | 1 | 120 |
| All providers | grok-4.6 | kimi-k3 | fact_f1 | 5.1% | 2.1% to 8.2% | 600 | 443 | 157 | 96 |
| All providers | grok-4.6 | kimi-k3 | exact_world | -5.6% | -11.3% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | grok-4.6 | kimi-k3 | task_success | -5.6% | -11.3% to 0.0% | 600 | 443 | 157 | 96 |
| All providers | grok-4.6 | kimi-k3 | decision_correct | -3.8% | -6.7% to -1.4% | 600 | 443 | 157 | 96 |
| All providers | grok-4.6 | mimo-v2.6-flash | fact_f1 | 10.3% | 7.8% to 12.7% | 600 | 594 | 6 | 120 |
| All providers | grok-4.6 | mimo-v2.6-flash | exact_world | -3.7% | -8.8% to 1.2% | 600 | 594 | 6 | 120 |
| All providers | grok-4.6 | mimo-v2.6-flash | task_success | -2.0% | -6.9% to 3.0% | 600 | 594 | 6 | 120 |
| All providers | grok-4.6 | mimo-v2.6-flash | decision_correct | 6.9% | 3.7% to 10.4% | 600 | 594 | 6 | 120 |
| All providers | grok-4.6 | mimo-v2.6-pro | fact_f1 | 12.9% | 10.4% to 15.5% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | mimo-v2.6-pro | exact_world | -2.2% | -6.5% to 2.7% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | mimo-v2.6-pro | task_success | -1.0% | -5.5% to 3.8% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | mimo-v2.6-pro | decision_correct | 8.2% | 4.7% to 11.8% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | minimax-m3 | fact_f1 | 21.0% | 17.8% to 24.2% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | minimax-m3 | exact_world | 3.2% | -1.3% to 7.7% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | minimax-m3 | task_success | 3.2% | -1.3% to 7.7% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | minimax-m3 | decision_correct | 0.0% | -2.3% to 2.2% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | qwen3.8-27b | fact_f1 | 5.8% | 3.6% to 8.4% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | qwen3.8-27b | exact_world | 3.3% | -1.3% to 8.0% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | qwen3.8-27b | task_success | 3.3% | -1.3% to 8.0% | 600 | 600 | 0 | 120 |
| All providers | grok-4.6 | qwen3.8-27b | decision_correct | -0.3% | -2.5% to 1.5% | 600 | 600 | 0 | 120 |
| All providers | kimi-k2.6 | kimi-k3 | fact_f1 | 6.2% | 3.9% to 8.6% | 600 | 443 | 157 | 96 |
| All providers | kimi-k2.6 | kimi-k3 | exact_world | -2.7% | -7.8% to 2.6% | 600 | 443 | 157 | 96 |
| All providers | kimi-k2.6 | kimi-k3 | task_success | -2.7% | -7.8% to 2.6% | 600 | 443 | 157 | 96 |
| All providers | kimi-k2.6 | kimi-k3 | decision_correct | -2.5% | -4.4% to -0.9% | 600 | 443 | 157 | 96 |
| All providers | kimi-k2.6 | mimo-v2.6-flash | fact_f1 | 11.3% | 9.0% to 13.4% | 600 | 593 | 7 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-flash | exact_world | -3.9% | -7.7% to 0.0% | 600 | 593 | 7 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-flash | task_success | -2.2% | -6.1% to 1.8% | 600 | 593 | 7 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-flash | decision_correct | 7.9% | 4.9% to 11.1% | 600 | 593 | 7 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-pro | fact_f1 | 14.0% | 11.6% to 16.6% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-pro | exact_world | -2.5% | -6.2% to 1.3% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-pro | task_success | -1.3% | -5.2% to 2.7% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | mimo-v2.6-pro | decision_correct | 9.3% | 6.2% to 12.8% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | minimax-m3 | fact_f1 | 22.1% | 19.0% to 25.2% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | minimax-m3 | exact_world | 2.7% | -1.3% to 6.5% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | minimax-m3 | task_success | 2.7% | -1.3% to 6.5% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | minimax-m3 | decision_correct | 1.2% | -0.8% to 3.2% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | qwen3.8-27b | fact_f1 | 7.0% | 4.9% to 9.1% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | qwen3.8-27b | exact_world | 3.0% | -1.0% to 7.2% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | qwen3.8-27b | task_success | 3.0% | -1.0% to 7.2% | 600 | 599 | 1 | 120 |
| All providers | kimi-k2.6 | qwen3.8-27b | decision_correct | 0.8% | -1.2% to 3.2% | 600 | 599 | 1 | 120 |
| All providers | kimi-k3 | mimo-v2.6-flash | fact_f1 | 2.6% | -0.3% to 5.3% | 600 | 437 | 163 | 96 |
| All providers | kimi-k3 | mimo-v2.6-flash | exact_world | -1.8% | -6.7% to 2.8% | 600 | 437 | 163 | 96 |
| All providers | kimi-k3 | mimo-v2.6-flash | task_success | -0.2% | -5.1% to 4.4% | 600 | 437 | 163 | 96 |
| All providers | kimi-k3 | mimo-v2.6-flash | decision_correct | 8.5% | 6.0% to 11.2% | 600 | 437 | 163 | 96 |
| All providers | kimi-k3 | mimo-v2.6-pro | fact_f1 | 3.9% | 1.2% to 6.8% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | mimo-v2.6-pro | exact_world | -1.6% | -6.2% to 2.9% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | mimo-v2.6-pro | task_success | -0.7% | -5.3% to 3.9% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | mimo-v2.6-pro | decision_correct | 7.9% | 5.1% to 10.9% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | minimax-m3 | fact_f1 | 12.6% | 8.8% to 16.7% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | minimax-m3 | exact_world | 5.6% | 0.9% to 10.2% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | minimax-m3 | task_success | 5.6% | 0.9% to 10.2% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | minimax-m3 | decision_correct | 3.6% | 1.9% to 5.5% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | qwen3.8-27b | fact_f1 | -1.0% | -4.0% to 2.1% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | qwen3.8-27b | exact_world | 6.1% | 1.4% to 11.0% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | qwen3.8-27b | task_success | 6.1% | 1.4% to 11.0% | 600 | 443 | 157 | 96 |
| All providers | kimi-k3 | qwen3.8-27b | decision_correct | 3.4% | 1.6% to 5.6% | 600 | 443 | 157 | 96 |
| All providers | mimo-v2.6-flash | mimo-v2.6-pro | fact_f1 | 2.9% | 0.5% to 5.5% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | mimo-v2.6-pro | exact_world | 2.0% | -1.7% to 5.9% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | mimo-v2.6-pro | task_success | 1.5% | -2.2% to 5.4% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | mimo-v2.6-pro | decision_correct | 1.5% | -1.8% to 5.0% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | minimax-m3 | fact_f1 | 10.9% | 7.9% to 14.0% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | minimax-m3 | exact_world | 7.1% | 3.4% to 11.0% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | minimax-m3 | task_success | 5.4% | 1.5% to 9.4% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | minimax-m3 | decision_correct | -6.9% | -9.8% to -4.2% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | qwen3.8-27b | fact_f1 | -4.2% | -6.7% to -1.6% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | qwen3.8-27b | exact_world | 7.2% | 3.4% to 11.3% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | qwen3.8-27b | task_success | 5.6% | 1.7% to 9.8% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-flash | qwen3.8-27b | decision_correct | -7.1% | -10.1% to -4.0% | 600 | 594 | 6 | 120 |
| All providers | mimo-v2.6-pro | minimax-m3 | fact_f1 | 8.1% | 5.0% to 11.4% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | minimax-m3 | exact_world | 5.3% | 1.2% to 9.2% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | minimax-m3 | task_success | 4.2% | -0.2% to 8.2% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | minimax-m3 | decision_correct | -8.2% | -11.7% to -4.8% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | qwen3.8-27b | fact_f1 | -7.0% | -9.9% to -4.2% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | qwen3.8-27b | exact_world | 5.5% | 1.2% to 9.8% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | qwen3.8-27b | task_success | 4.3% | -0.2% to 8.8% | 600 | 600 | 0 | 120 |
| All providers | mimo-v2.6-pro | qwen3.8-27b | decision_correct | -8.5% | -12.0% to -5.0% | 600 | 600 | 0 | 120 |
| All providers | minimax-m3 | qwen3.8-27b | fact_f1 | -15.2% | -18.3% to -12.1% | 600 | 600 | 0 | 120 |
| All providers | minimax-m3 | qwen3.8-27b | exact_world | 0.2% | -3.7% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | minimax-m3 | qwen3.8-27b | task_success | 0.2% | -3.7% to 4.0% | 600 | 600 | 0 | 120 |
| All providers | minimax-m3 | qwen3.8-27b | decision_correct | -0.3% | -2.0% to 1.3% | 600 | 600 | 0 | 120 |
| Provider | Left | Right | Metric | Difference (pp) | 95% CI (pp) | Expected pairs | Complete pairs | Missing pairs | Families |
|---|---|---|---|---|---|---|---|---|---|
| baseline:constraint-solver | explicit | formal | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | formal | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | formal | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | formal | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | formal | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | formal | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:constraint-solver | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | formal | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | formal | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | formal | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | formal | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | formal | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | formal | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:direct-facts | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | formal | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | formal | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | formal | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | formal | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | formal | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | formal | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:forward-chain | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | formal | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | formal | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | formal | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | formal | implicit | fact_f1 | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | formal | implicit | exact_world | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | formal | implicit | task_success | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| baseline:story-copy | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| claude-fable-5 | explicit | formal | fact_f1 | -7.5% | -10.5% to -4.9% | 120 | 44 | 76 | 44 |
| claude-fable-5 | explicit | formal | exact_world | -52.3% | -63.6% to -40.9% | 120 | 44 | 76 | 44 |
| claude-fable-5 | explicit | formal | task_success | -52.3% | -63.6% to -40.9% | 120 | 44 | 76 | 44 |
| claude-fable-5 | explicit | formal | decision_correct | -3.4% | -8.0% to 0.0% | 120 | 44 | 76 | 44 |
| claude-fable-5 | explicit | implicit | fact_f1 | -0.9% | -3.1% to 1.2% | 120 | 32 | 88 | 32 |
| claude-fable-5 | explicit | implicit | exact_world | 0.0% | -10.9% to 10.9% | 120 | 32 | 88 | 32 |
| claude-fable-5 | explicit | implicit | task_success | 0.0% | -10.9% to 10.9% | 120 | 32 | 88 | 32 |
| claude-fable-5 | explicit | implicit | decision_correct | -1.6% | -4.7% to 0.0% | 120 | 32 | 88 | 32 |
| claude-fable-5 | formal | implicit | fact_f1 | 6.7% | 4.6% to 8.9% | 120 | 32 | 88 | 32 |
| claude-fable-5 | formal | implicit | exact_world | 56.2% | 43.8% to 68.8% | 120 | 32 | 88 | 32 |
| claude-fable-5 | formal | implicit | task_success | 56.2% | 43.8% to 68.8% | 120 | 32 | 88 | 32 |
| claude-fable-5 | formal | implicit | decision_correct | 3.1% | 0.0% to 7.8% | 120 | 32 | 88 | 32 |
| claude-fable-5.1 | explicit | formal | fact_f1 | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | formal | exact_world | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | formal | task_success | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | formal | decision_correct | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | implicit | fact_f1 | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | implicit | exact_world | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | implicit | task_success | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | explicit | implicit | decision_correct | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | formal | implicit | fact_f1 | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | formal | implicit | exact_world | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | formal | implicit | task_success | — | — | 120 | 0 | 120 | 0 |
| claude-fable-5.1 | formal | implicit | decision_correct | — | — | 120 | 0 | 120 | 0 |
| claude-haiku-4.5 | explicit | formal | fact_f1 | -30.4% | -34.8% to -26.0% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | formal | exact_world | -92.5% | -96.2% to -88.3% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | formal | task_success | -92.5% | -96.2% to -88.3% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | formal | decision_correct | -2.9% | -6.7% to 0.8% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | implicit | fact_f1 | 15.0% | 11.3% to 18.6% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | implicit | exact_world | 1.7% | -2.1% to 5.4% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | implicit | task_success | 1.7% | -2.1% to 5.4% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | explicit | implicit | decision_correct | 1.2% | -2.5% to 5.0% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | formal | implicit | fact_f1 | 45.4% | 40.3% to 50.1% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | formal | implicit | exact_world | 94.2% | 90.4% to 97.1% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | formal | implicit | task_success | 94.2% | 90.4% to 97.1% | 120 | 120 | 0 | 120 |
| claude-haiku-4.5 | formal | implicit | decision_correct | 4.2% | 0.4% to 7.9% | 120 | 120 | 0 | 120 |
| claude-opus-5 | explicit | formal | fact_f1 | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | explicit | formal | exact_world | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | explicit | formal | task_success | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | explicit | formal | decision_correct | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | explicit | implicit | fact_f1 | 10.1% | 3.9% to 16.8% | 120 | 85 | 35 | 85 |
| claude-opus-5 | explicit | implicit | exact_world | 18.2% | 8.8% to 27.1% | 120 | 85 | 35 | 85 |
| claude-opus-5 | explicit | implicit | task_success | 18.2% | 8.8% to 27.1% | 120 | 85 | 35 | 85 |
| claude-opus-5 | explicit | implicit | decision_correct | 0.0% | -5.9% to 6.5% | 120 | 85 | 35 | 85 |
| claude-opus-5 | formal | implicit | fact_f1 | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | formal | implicit | exact_world | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | formal | implicit | task_success | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5 | formal | implicit | decision_correct | — | — | 120 | 0 | 120 | 0 |
| claude-opus-5.5 | explicit | formal | fact_f1 | -7.2% | -8.3% to -6.1% | 120 | 115 | 5 | 115 |
| claude-opus-5.5 | explicit | formal | exact_world | -71.3% | -77.0% to -65.6% | 120 | 115 | 5 | 115 |
| claude-opus-5.5 | explicit | formal | task_success | -71.7% | -77.4% to -65.7% | 120 | 115 | 5 | 115 |
| claude-opus-5.5 | explicit | formal | decision_correct | -4.8% | -7.8% to -2.2% | 120 | 115 | 5 | 115 |
| claude-opus-5.5 | explicit | implicit | fact_f1 | 2.5% | 0.8% to 4.3% | 120 | 113 | 7 | 113 |
| claude-opus-5.5 | explicit | implicit | exact_world | 7.5% | 0.9% to 14.2% | 120 | 113 | 7 | 113 |
| claude-opus-5.5 | explicit | implicit | task_success | 7.1% | 0.0% to 13.7% | 120 | 113 | 7 | 113 |
| claude-opus-5.5 | explicit | implicit | decision_correct | 2.2% | -0.9% to 5.3% | 120 | 113 | 7 | 113 |
| claude-opus-5.5 | formal | implicit | fact_f1 | 9.6% | 8.1% to 11.2% | 120 | 116 | 4 | 116 |
| claude-opus-5.5 | formal | implicit | exact_world | 78.0% | 72.4% to 83.6% | 120 | 116 | 4 | 116 |
| claude-opus-5.5 | formal | implicit | task_success | 78.0% | 72.4% to 83.6% | 120 | 116 | 4 | 116 |
| claude-opus-5.5 | formal | implicit | decision_correct | 6.5% | 3.9% to 9.5% | 120 | 116 | 4 | 116 |
| claude-sonnet-5 | explicit | formal | fact_f1 | -0.4% | -6.2% to 6.0% | 120 | 86 | 34 | 86 |
| claude-sonnet-5 | explicit | formal | exact_world | -51.7% | -59.9% to -42.4% | 120 | 86 | 34 | 86 |
| claude-sonnet-5 | explicit | formal | task_success | -51.7% | -59.9% to -42.4% | 120 | 86 | 34 | 86 |
| claude-sonnet-5 | explicit | formal | decision_correct | 4.1% | -1.7% to 10.5% | 120 | 86 | 34 | 86 |
| claude-sonnet-5 | explicit | implicit | fact_f1 | 9.4% | 3.1% to 15.5% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | explicit | implicit | exact_world | -1.0% | -11.5% to 9.6% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | explicit | implicit | task_success | -1.0% | -11.5% to 9.6% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | explicit | implicit | decision_correct | -1.0% | -5.8% to 3.8% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | formal | implicit | fact_f1 | 7.6% | -1.8% to 16.5% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | formal | implicit | exact_world | 50.0% | 38.5% to 61.5% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | formal | implicit | task_success | 50.0% | 38.5% to 61.5% | 120 | 52 | 68 | 52 |
| claude-sonnet-5 | formal | implicit | decision_correct | -5.8% | -15.4% to 2.9% | 120 | 52 | 68 | 52 |
| deepseek-4.1-flash | explicit | formal | fact_f1 | -4.5% | -5.9% to -3.3% | 120 | 116 | 4 | 116 |
| deepseek-4.1-flash | explicit | formal | exact_world | -44.8% | -52.2% to -37.5% | 120 | 116 | 4 | 116 |
| deepseek-4.1-flash | explicit | formal | task_success | -44.8% | -52.2% to -37.5% | 120 | 116 | 4 | 116 |
| deepseek-4.1-flash | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 116 | 4 | 116 |
| deepseek-4.1-flash | explicit | implicit | fact_f1 | 41.9% | 36.4% to 47.2% | 120 | 115 | 5 | 115 |
| deepseek-4.1-flash | explicit | implicit | exact_world | 37.0% | 29.6% to 45.2% | 120 | 115 | 5 | 115 |
| deepseek-4.1-flash | explicit | implicit | task_success | 37.0% | 29.6% to 45.2% | 120 | 115 | 5 | 115 |
| deepseek-4.1-flash | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 115 | 5 | 115 |
| deepseek-4.1-flash | formal | implicit | fact_f1 | 46.6% | 41.4% to 52.1% | 120 | 119 | 1 | 119 |
| deepseek-4.1-flash | formal | implicit | exact_world | 81.9% | 76.5% to 87.8% | 120 | 119 | 1 | 119 |
| deepseek-4.1-flash | formal | implicit | task_success | 81.9% | 76.5% to 87.8% | 120 | 119 | 1 | 119 |
| deepseek-4.1-flash | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 119 | 1 | 119 |
| deepseek-v4-pro | explicit | formal | fact_f1 | -4.8% | -7.9% to -1.3% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | formal | exact_world | -58.8% | -65.4% to -52.1% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | formal | task_success | -58.8% | -65.4% to -52.1% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | formal | decision_correct | -2.9% | -6.7% to 1.2% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | implicit | fact_f1 | 5.4% | 2.3% to 8.4% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | implicit | exact_world | 19.6% | 12.1% to 26.7% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | implicit | task_success | 19.6% | 12.1% to 26.7% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | explicit | implicit | decision_correct | -0.4% | -3.3% to 2.5% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | formal | implicit | fact_f1 | 10.2% | 6.5% to 13.4% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | formal | implicit | exact_world | 78.3% | 71.7% to 84.2% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | formal | implicit | task_success | 78.3% | 71.7% to 84.2% | 120 | 120 | 0 | 120 |
| deepseek-v4-pro | formal | implicit | decision_correct | 2.5% | -1.7% to 6.2% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | formal | fact_f1 | -16.3% | -19.7% to -12.9% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | formal | exact_world | -57.1% | -63.7% to -50.0% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | formal | task_success | -57.1% | -63.7% to -50.0% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | formal | decision_correct | -1.7% | -3.3% to -0.4% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | implicit | fact_f1 | 18.8% | 14.9% to 22.4% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | implicit | exact_world | 26.7% | 20.4% to 32.9% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | implicit | task_success | 26.7% | 20.4% to 32.9% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | explicit | implicit | decision_correct | -0.8% | -2.5% to 0.8% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | formal | implicit | fact_f1 | 35.0% | 30.9% to 39.2% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | formal | implicit | exact_world | 83.8% | 77.9% to 88.8% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | formal | implicit | task_success | 83.8% | 77.9% to 88.8% | 120 | 120 | 0 | 120 |
| gemini-3.6-flash | formal | implicit | decision_correct | 0.8% | 0.0% to 2.5% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | formal | fact_f1 | -3.1% | -4.5% to -2.0% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | formal | exact_world | -31.2% | -37.5% to -25.4% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | formal | task_success | -31.2% | -37.5% to -25.4% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | formal | decision_correct | -0.8% | -2.1% to 0.0% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | implicit | fact_f1 | 15.5% | 11.6% to 19.2% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | implicit | exact_world | 15.8% | 7.5% to 23.3% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | implicit | task_success | 15.8% | 7.5% to 23.3% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | explicit | implicit | decision_correct | -0.4% | -1.2% to 0.0% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | formal | implicit | fact_f1 | 18.6% | 14.8% to 22.6% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | formal | implicit | exact_world | 47.1% | 40.4% to 53.3% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | formal | implicit | task_success | 47.1% | 40.4% to 53.3% | 120 | 120 | 0 | 120 |
| gemini-3.7-flash | formal | implicit | decision_correct | 0.4% | 0.0% to 1.2% | 120 | 120 | 0 | 120 |
| gemini-3.8-flash | explicit | formal | fact_f1 | -4.6% | -6.4% to -3.3% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | explicit | formal | exact_world | -41.2% | -48.3% to -34.0% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | explicit | formal | task_success | -41.2% | -48.3% to -34.0% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | explicit | formal | decision_correct | -0.8% | -2.1% to 0.0% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | explicit | implicit | fact_f1 | 24.5% | 19.5% to 29.4% | 120 | 118 | 2 | 118 |
| gemini-3.8-flash | explicit | implicit | exact_world | 19.1% | 8.9% to 28.0% | 120 | 118 | 2 | 118 |
| gemini-3.8-flash | explicit | implicit | task_success | 19.1% | 8.9% to 28.0% | 120 | 118 | 2 | 118 |
| gemini-3.8-flash | explicit | implicit | decision_correct | -0.8% | -2.1% to 0.0% | 120 | 118 | 2 | 118 |
| gemini-3.8-flash | formal | implicit | fact_f1 | 29.2% | 24.6% to 33.8% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | formal | implicit | exact_world | 60.9% | 53.4% to 68.1% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | formal | implicit | task_success | 60.9% | 53.4% to 68.1% | 120 | 119 | 1 | 119 |
| gemini-3.8-flash | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 119 | 1 | 119 |
| glm-4.7-flash | explicit | formal | fact_f1 | -26.3% | -33.2% to -19.0% | 120 | 104 | 16 | 104 |
| glm-4.7-flash | explicit | formal | exact_world | -74.0% | -81.7% to -65.9% | 120 | 104 | 16 | 104 |
| glm-4.7-flash | explicit | formal | task_success | -74.0% | -81.7% to -65.9% | 120 | 104 | 16 | 104 |
| glm-4.7-flash | explicit | formal | decision_correct | -9.6% | -16.8% to -2.9% | 120 | 104 | 16 | 104 |
| glm-4.7-flash | explicit | implicit | fact_f1 | 6.3% | 0.9% to 11.9% | 120 | 97 | 23 | 97 |
| glm-4.7-flash | explicit | implicit | exact_world | 4.6% | 0.5% to 8.8% | 120 | 97 | 23 | 97 |
| glm-4.7-flash | explicit | implicit | task_success | 4.6% | 0.5% to 8.8% | 120 | 97 | 23 | 97 |
| glm-4.7-flash | explicit | implicit | decision_correct | 3.6% | -2.6% to 10.8% | 120 | 97 | 23 | 97 |
| glm-4.7-flash | formal | implicit | fact_f1 | 33.9% | 27.4% to 39.8% | 120 | 107 | 13 | 107 |
| glm-4.7-flash | formal | implicit | exact_world | 78.0% | 70.1% to 85.5% | 120 | 107 | 13 | 107 |
| glm-4.7-flash | formal | implicit | task_success | 78.0% | 70.1% to 85.5% | 120 | 107 | 13 | 107 |
| glm-4.7-flash | formal | implicit | decision_correct | 14.5% | 6.5% to 22.4% | 120 | 107 | 13 | 107 |
| glm-5.3 | explicit | formal | fact_f1 | -14.0% | -18.8% to -9.5% | 120 | 97 | 23 | 97 |
| glm-5.3 | explicit | formal | exact_world | -46.4% | -54.6% to -38.1% | 120 | 97 | 23 | 97 |
| glm-5.3 | explicit | formal | task_success | -46.4% | -54.6% to -38.1% | 120 | 97 | 23 | 97 |
| glm-5.3 | explicit | formal | decision_correct | -2.6% | -5.7% to 0.0% | 120 | 97 | 23 | 97 |
| glm-5.3 | explicit | implicit | fact_f1 | 6.2% | 1.4% to 11.1% | 120 | 91 | 29 | 91 |
| glm-5.3 | explicit | implicit | exact_world | 16.5% | 8.2% to 24.7% | 120 | 91 | 29 | 91 |
| glm-5.3 | explicit | implicit | task_success | 16.5% | 8.2% to 24.7% | 120 | 91 | 29 | 91 |
| glm-5.3 | explicit | implicit | decision_correct | 1.1% | -3.3% to 5.5% | 120 | 91 | 29 | 91 |
| glm-5.3 | formal | implicit | fact_f1 | 19.9% | 14.4% to 25.1% | 120 | 109 | 11 | 109 |
| glm-5.3 | formal | implicit | exact_world | 60.1% | 52.3% to 67.0% | 120 | 109 | 11 | 109 |
| glm-5.3 | formal | implicit | task_success | 60.1% | 52.3% to 67.0% | 120 | 109 | 11 | 109 |
| glm-5.3 | formal | implicit | decision_correct | 2.8% | -0.9% to 6.4% | 120 | 109 | 11 | 109 |
| glm-5.3-flash | explicit | formal | fact_f1 | -34.3% | -39.3% to -29.4% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | formal | exact_world | -66.4% | -74.6% to -58.2% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | formal | task_success | -66.8% | -75.0% to -58.6% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | formal | decision_correct | -2.2% | -4.3% to -0.4% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | implicit | fact_f1 | 12.7% | 7.9% to 17.6% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | implicit | exact_world | 14.2% | 9.1% to 19.8% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | implicit | task_success | 13.8% | 8.6% to 19.4% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | explicit | implicit | decision_correct | 0.0% | -2.6% to 2.6% | 120 | 116 | 4 | 116 |
| glm-5.3-flash | formal | implicit | fact_f1 | 45.8% | 41.1% to 50.6% | 120 | 120 | 0 | 120 |
| glm-5.3-flash | formal | implicit | exact_world | 78.3% | 71.2% to 85.0% | 120 | 120 | 0 | 120 |
| glm-5.3-flash | formal | implicit | task_success | 78.3% | 71.2% to 85.0% | 120 | 120 | 0 | 120 |
| glm-5.3-flash | formal | implicit | decision_correct | 2.1% | 0.4% to 4.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | formal | fact_f1 | -5.5% | -6.5% to -4.6% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | formal | exact_world | -60.0% | -66.2% to -53.8% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | formal | task_success | -60.0% | -66.3% to -53.3% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | formal | decision_correct | -1.7% | -4.6% to 2.1% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | implicit | fact_f1 | 35.5% | 30.3% to 40.8% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | implicit | exact_world | 23.3% | 16.7% to 30.4% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | implicit | task_success | 21.7% | 15.0% to 28.3% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | explicit | implicit | decision_correct | -0.4% | -3.3% to 2.5% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | formal | implicit | fact_f1 | 41.1% | 36.0% to 46.1% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | formal | implicit | exact_world | 83.3% | 78.8% to 87.5% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | formal | implicit | task_success | 81.7% | 75.8% to 86.7% | 120 | 120 | 0 | 120 |
| gpt-5.6-luna | formal | implicit | decision_correct | 1.2% | -2.1% to 4.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | formal | fact_f1 | -2.8% | -3.6% to -2.1% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | formal | exact_world | -32.9% | -40.0% to -26.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | formal | task_success | -32.9% | -40.0% to -26.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | implicit | fact_f1 | 16.8% | 12.6% to 21.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | implicit | exact_world | 12.9% | 5.0% to 20.8% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | implicit | task_success | 12.9% | 5.0% to 20.8% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | formal | implicit | fact_f1 | 19.6% | 15.3% to 23.8% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | formal | implicit | exact_world | 45.8% | 38.8% to 52.9% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | formal | implicit | task_success | 45.8% | 38.8% to 52.9% | 120 | 120 | 0 | 120 |
| gpt-5.6-sol | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | formal | fact_f1 | -3.7% | -4.5% to -3.0% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | formal | exact_world | -47.5% | -54.2% to -41.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | formal | task_success | -47.5% | -54.2% to -41.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | implicit | fact_f1 | 38.1% | 32.7% to 43.5% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | implicit | exact_world | 28.3% | 20.0% to 36.7% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | implicit | task_success | 28.3% | 20.0% to 36.7% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | explicit | implicit | decision_correct | 0.4% | 0.0% to 1.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | formal | implicit | fact_f1 | 41.9% | 36.6% to 47.1% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | formal | implicit | exact_world | 75.8% | 70.0% to 81.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | formal | implicit | task_success | 75.8% | 70.0% to 81.2% | 120 | 120 | 0 | 120 |
| gpt-5.6-terra | formal | implicit | decision_correct | 0.4% | 0.0% to 1.2% | 120 | 120 | 0 | 120 |
| gpt-6-astra | explicit | formal | fact_f1 | -0.9% | -1.4% to -0.5% | 120 | 108 | 12 | 108 |
| gpt-6-astra | explicit | formal | exact_world | -11.6% | -16.7% to -7.4% | 120 | 108 | 12 | 108 |
| gpt-6-astra | explicit | formal | task_success | -11.6% | -16.7% to -7.4% | 120 | 108 | 12 | 108 |
| gpt-6-astra | explicit | formal | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 108 | 12 | 108 |
| gpt-6-astra | explicit | implicit | fact_f1 | 1.3% | 0.6% to 2.3% | 120 | 101 | 19 | 101 |
| gpt-6-astra | explicit | implicit | exact_world | 8.9% | 4.5% to 13.9% | 120 | 101 | 19 | 101 |
| gpt-6-astra | explicit | implicit | task_success | 8.9% | 4.5% to 13.9% | 120 | 101 | 19 | 101 |
| gpt-6-astra | explicit | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 101 | 19 | 101 |
| gpt-6-astra | formal | implicit | fact_f1 | 2.3% | 1.4% to 3.3% | 120 | 104 | 16 | 104 |
| gpt-6-astra | formal | implicit | exact_world | 20.7% | 14.4% to 26.9% | 120 | 104 | 16 | 104 |
| gpt-6-astra | formal | implicit | task_success | 20.7% | 14.4% to 26.9% | 120 | 104 | 16 | 104 |
| gpt-6-astra | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 104 | 16 | 104 |
| gpt-6-luna | explicit | formal | fact_f1 | -7.8% | -9.5% to -6.4% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | formal | exact_world | -66.7% | -72.1% to -60.8% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | formal | task_success | -65.8% | -71.7% to -59.6% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | formal | decision_correct | -2.1% | -4.6% to 0.8% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | implicit | fact_f1 | 30.7% | 25.1% to 36.4% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | implicit | exact_world | 10.4% | 2.9% to 17.5% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | implicit | task_success | 10.4% | 2.9% to 17.5% | 120 | 120 | 0 | 120 |
| gpt-6-luna | explicit | implicit | decision_correct | -2.1% | -4.6% to 0.0% | 120 | 120 | 0 | 120 |
| gpt-6-luna | formal | implicit | fact_f1 | 38.5% | 33.1% to 43.9% | 120 | 120 | 0 | 120 |
| gpt-6-luna | formal | implicit | exact_world | 77.1% | 71.2% to 82.5% | 120 | 120 | 0 | 120 |
| gpt-6-luna | formal | implicit | task_success | 76.2% | 69.6% to 82.5% | 120 | 120 | 0 | 120 |
| gpt-6-luna | formal | implicit | decision_correct | 0.0% | -2.1% to 1.7% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | formal | fact_f1 | -4.2% | -6.2% to -1.8% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | formal | exact_world | -37.9% | -45.0% to -31.2% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | formal | task_success | -38.3% | -45.4% to -31.7% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | formal | decision_correct | -2.5% | -5.4% to 0.4% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | implicit | fact_f1 | -2.1% | -3.5% to -0.7% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | implicit | exact_world | -13.8% | -20.0% to -7.1% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | implicit | task_success | -14.2% | -20.4% to -7.5% | 120 | 120 | 0 | 120 |
| gpt-6-sol | explicit | implicit | decision_correct | -3.3% | -5.8% to -1.2% | 120 | 120 | 0 | 120 |
| gpt-6-sol | formal | implicit | fact_f1 | 2.1% | -0.2% to 3.9% | 120 | 120 | 0 | 120 |
| gpt-6-sol | formal | implicit | exact_world | 24.2% | 17.9% to 30.4% | 120 | 120 | 0 | 120 |
| gpt-6-sol | formal | implicit | task_success | 24.2% | 17.9% to 30.4% | 120 | 120 | 0 | 120 |
| gpt-6-sol | formal | implicit | decision_correct | -0.8% | -2.5% to 0.0% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | formal | fact_f1 | -12.0% | -13.6% to -10.3% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | formal | exact_world | -78.8% | -84.2% to -72.9% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | formal | task_success | -78.8% | -84.2% to -72.9% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | formal | decision_correct | -4.2% | -7.5% to -1.2% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | implicit | fact_f1 | 2.4% | 0.1% to 4.6% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | implicit | exact_world | 0.0% | -6.2% to 5.8% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | implicit | task_success | 0.0% | -6.2% to 5.8% | 120 | 120 | 0 | 120 |
| grok-4.6 | explicit | implicit | decision_correct | 0.0% | -2.1% to 2.1% | 120 | 120 | 0 | 120 |
| grok-4.6 | formal | implicit | fact_f1 | 14.3% | 12.2% to 16.4% | 120 | 120 | 0 | 120 |
| grok-4.6 | formal | implicit | exact_world | 78.8% | 72.9% to 84.2% | 120 | 120 | 0 | 120 |
| grok-4.6 | formal | implicit | task_success | 78.8% | 72.9% to 84.2% | 120 | 120 | 0 | 120 |
| grok-4.6 | formal | implicit | decision_correct | 4.2% | 1.7% to 7.1% | 120 | 120 | 0 | 120 |
| kimi-k2.6 | explicit | formal | fact_f1 | -7.6% | -8.6% to -6.5% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | formal | exact_world | -76.1% | -81.5% to -70.2% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | formal | task_success | -76.1% | -81.5% to -70.2% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | formal | decision_correct | -2.1% | -4.2% to -0.4% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | implicit | fact_f1 | 8.2% | 5.7% to 10.8% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | implicit | exact_world | 6.3% | 0.0% to 12.6% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | implicit | task_success | 6.3% | 0.0% to 12.6% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | explicit | implicit | decision_correct | 1.3% | -1.3% to 3.8% | 120 | 119 | 1 | 119 |
| kimi-k2.6 | formal | implicit | fact_f1 | 16.0% | 13.5% to 18.6% | 120 | 120 | 0 | 120 |
| kimi-k2.6 | formal | implicit | exact_world | 82.5% | 77.1% to 87.5% | 120 | 120 | 0 | 120 |
| kimi-k2.6 | formal | implicit | task_success | 82.5% | 77.1% to 87.5% | 120 | 120 | 0 | 120 |
| kimi-k2.6 | formal | implicit | decision_correct | 3.3% | 1.2% to 5.8% | 120 | 120 | 0 | 120 |
| kimi-k3 | explicit | formal | fact_f1 | -15.2% | -18.7% to -11.6% | 120 | 81 | 39 | 81 |
| kimi-k3 | explicit | formal | exact_world | -73.5% | -80.2% to -66.7% | 120 | 81 | 39 | 81 |
| kimi-k3 | explicit | formal | task_success | -73.5% | -80.2% to -66.7% | 120 | 81 | 39 | 81 |
| kimi-k3 | explicit | formal | decision_correct | -0.6% | -1.9% to 0.0% | 120 | 81 | 39 | 81 |
| kimi-k3 | explicit | implicit | fact_f1 | 9.1% | 4.9% to 13.4% | 120 | 79 | 41 | 79 |
| kimi-k3 | explicit | implicit | exact_world | 2.5% | -5.7% to 11.4% | 120 | 79 | 41 | 79 |
| kimi-k3 | explicit | implicit | task_success | 2.5% | -5.7% to 11.4% | 120 | 79 | 41 | 79 |
| kimi-k3 | explicit | implicit | decision_correct | -0.6% | -1.9% to 0.0% | 120 | 79 | 41 | 79 |
| kimi-k3 | formal | implicit | fact_f1 | 22.7% | 17.8% to 27.6% | 120 | 86 | 34 | 86 |
| kimi-k3 | formal | implicit | exact_world | 72.7% | 64.5% to 80.2% | 120 | 86 | 34 | 86 |
| kimi-k3 | formal | implicit | task_success | 72.7% | 64.5% to 80.2% | 120 | 86 | 34 | 86 |
| kimi-k3 | formal | implicit | decision_correct | 0.0% | 0.0% to 0.0% | 120 | 86 | 34 | 86 |
| mimo-v2.6-flash | explicit | formal | fact_f1 | -6.0% | -10.0% to -1.6% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | explicit | formal | exact_world | -54.7% | -62.0% to -47.0% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | explicit | formal | task_success | -53.0% | -60.7% to -44.4% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | explicit | formal | decision_correct | -2.1% | -7.7% to 3.8% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | explicit | implicit | fact_f1 | 30.6% | 24.8% to 36.1% | 120 | 115 | 5 | 115 |
| mimo-v2.6-flash | explicit | implicit | exact_world | 26.5% | 19.6% to 33.0% | 120 | 115 | 5 | 115 |
| mimo-v2.6-flash | explicit | implicit | task_success | 25.7% | 18.7% to 32.2% | 120 | 115 | 5 | 115 |
| mimo-v2.6-flash | explicit | implicit | decision_correct | 4.3% | -0.9% to 10.0% | 120 | 115 | 5 | 115 |
| mimo-v2.6-flash | formal | implicit | fact_f1 | 36.6% | 30.8% to 42.0% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | formal | implicit | exact_world | 81.2% | 74.8% to 86.8% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | formal | implicit | task_success | 78.6% | 71.8% to 85.0% | 120 | 117 | 3 | 117 |
| mimo-v2.6-flash | formal | implicit | decision_correct | 6.4% | 0.0% to 12.8% | 120 | 117 | 3 | 117 |
| mimo-v2.6-pro | explicit | formal | fact_f1 | -12.8% | -17.1% to -8.0% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | formal | exact_world | -55.4% | -62.5% to -47.9% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | formal | task_success | -55.8% | -63.3% to -47.9% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | formal | decision_correct | -9.2% | -14.6% to -3.3% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | implicit | fact_f1 | 26.3% | 20.9% to 31.1% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | implicit | exact_world | 26.7% | 19.6% to 33.8% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | implicit | task_success | 24.6% | 17.5% to 32.1% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | explicit | implicit | decision_correct | 0.0% | -5.8% to 6.2% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | formal | implicit | fact_f1 | 39.1% | 34.4% to 43.6% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | formal | implicit | exact_world | 82.1% | 75.8% to 87.9% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | formal | implicit | task_success | 80.4% | 73.8% to 86.7% | 120 | 120 | 0 | 120 |
| mimo-v2.6-pro | formal | implicit | decision_correct | 9.2% | 3.3% to 15.0% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | formal | fact_f1 | -28.0% | -33.7% to -22.7% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | formal | exact_world | -72.9% | -79.6% to -66.2% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | formal | task_success | -72.9% | -79.6% to -66.2% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | formal | decision_correct | -0.8% | -4.6% to 2.9% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | implicit | fact_f1 | 16.6% | 9.9% to 23.2% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | implicit | exact_world | 13.3% | 6.2% to 20.8% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | implicit | task_success | 13.3% | 6.2% to 20.8% | 120 | 120 | 0 | 120 |
| minimax-m3 | explicit | implicit | decision_correct | 0.4% | -2.5% to 3.3% | 120 | 120 | 0 | 120 |
| minimax-m3 | formal | implicit | fact_f1 | 44.6% | 38.5% to 50.6% | 120 | 120 | 0 | 120 |
| minimax-m3 | formal | implicit | exact_world | 86.2% | 80.8% to 91.2% | 120 | 120 | 0 | 120 |
| minimax-m3 | formal | implicit | task_success | 86.2% | 80.8% to 91.2% | 120 | 120 | 0 | 120 |
| minimax-m3 | formal | implicit | decision_correct | 1.2% | -2.5% to 4.6% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | formal | fact_f1 | -9.3% | -13.8% to -4.3% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | formal | exact_world | -71.7% | -78.8% to -64.6% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | formal | task_success | -71.7% | -78.8% to -64.6% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | formal | decision_correct | 1.2% | -2.9% to 5.8% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | implicit | fact_f1 | 11.9% | 7.9% to 15.9% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | implicit | exact_world | 12.1% | 5.8% to 18.3% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | implicit | task_success | 12.1% | 5.8% to 18.3% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | explicit | implicit | decision_correct | -0.4% | -2.5% to 1.2% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | formal | implicit | fact_f1 | 21.2% | 15.7% to 26.2% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | formal | implicit | exact_world | 83.8% | 77.9% to 89.2% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | formal | implicit | task_success | 83.8% | 77.9% to 89.2% | 120 | 120 | 0 | 120 |
| qwen3.8-27b | formal | implicit | decision_correct | -1.7% | -6.2% to 2.5% | 120 | 120 | 0 | 120 |
Structural depth by configured level
{
"by_family": {
"inst-strata-L00-000-theater": {
"constraint_kinds": {
"eq": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.875,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-001-maritime": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-002-hotel": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-003-manor": {
"constraint_kinds": {
"eq": 5,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-004-manor": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-005-theater": {
"constraint_kinds": {
"eq": 4,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-006-theater": {
"constraint_kinds": {
"eq": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.875,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-007-hotel": {
"constraint_kinds": {
"eq": 7,
"implies": 1
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.875,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-008-maritime": {
"constraint_kinds": {
"eq": 5,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-009-maritime": {
"constraint_kinds": {
"eq": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 1.0,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 0,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-010-theater": {
"constraint_kinds": {
"eq": 7,
"implies": 1
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-011-hotel": {
"constraint_kinds": {
"eq": 4,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-012-manor": {
"constraint_kinds": {
"eq": 5,
"implies": 3
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 2,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 2,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-013-hotel": {
"constraint_kinds": {
"eq": 5,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-014-theater": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 1,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 2
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 2,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-015-maritime": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-016-maritime": {
"constraint_kinds": {
"eq": 7,
"implies": 1
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 2,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-017-manor": {
"constraint_kinds": {
"eq": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-018-theater": {
"constraint_kinds": {
"eq": 6,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L00-019-theater": {
"constraint_kinds": {
"eq": 7,
"implies": 1
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0
},
"direct_assignment_fraction": 0.875,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 8,
"forward_chain_rounds": 1,
"n_constraints": 8,
"n_variables": 8,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-000-hotel": {
"constraint_kinds": {
"eq": 10,
"implies": 1,
"neq": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V11": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V8": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 9,
"forward_chain_complete": false,
"forward_chain_fraction": 0.75,
"forward_chain_recovered": 9,
"forward_chain_rounds": 0,
"n_constraints": 15,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": [
"V7",
"V9",
"V10"
]
},
"inst-strata-L01-001-manor": {
"constraint_kinds": {
"eq": 10,
"implies": 2
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 2,
"V11": 3,
"V2": 1,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 3,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-002-observatory": {
"constraint_kinds": {
"eq": 8,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 2,
"V2": 1,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-003-observatory": {
"constraint_kinds": {
"eq": 6,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 1,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-004-observatory": {
"constraint_kinds": {
"eq": 9,
"implies": 3
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 1,
"V11": 1,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 2,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 6,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-005-maritime": {
"constraint_kinds": {
"eq": 10,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 0,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 1,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-006-theater": {
"constraint_kinds": {
"eq": 7,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-007-hotel": {
"constraint_kinds": {
"eq": 8,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 0,
"V2": 1,
"V3": 0,
"V4": 1,
"V5": 2,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-008-manor": {
"constraint_kinds": {
"eq": 10,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 1,
"V2": 0,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 1,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-009-theater": {
"constraint_kinds": {
"eq": 6,
"implies": 6
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 2,
"V11": 3,
"V2": 3,
"V3": 2,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 2,
"V8": 3,
"V9": 0
},
"direct_assignment_fraction": 0.25,
"direct_assignments": 3,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 3,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-010-manor": {
"constraint_kinds": {
"eq": 6,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 2,
"V9": 1
},
"direct_assignment_fraction": 0.3333333333333333,
"direct_assignments": 4,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-011-theater": {
"constraint_kinds": {
"eq": 8,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-012-manor": {
"constraint_kinds": {
"eq": 10,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.8333333333333334,
"direct_assignments": 10,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 1,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-013-observatory": {
"constraint_kinds": {
"eq": 9,
"implies": 3
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 2,
"V11": 3,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 3,
"V6": 0,
"V7": 0,
"V8": 3,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 3,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-014-theater": {
"constraint_kinds": {
"eq": 9,
"implies": 3
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 1,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-015-manor": {
"constraint_kinds": {
"eq": 7,
"implies": 5
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 1,
"V11": 0,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 2,
"V6": 1,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.4166666666666667,
"direct_assignments": 5,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-016-maritime": {
"constraint_kinds": {
"eq": 7,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V2": 1,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 1,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-017-observatory": {
"constraint_kinds": {
"eq": 9,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-018-manor": {
"constraint_kinds": {
"eq": 8,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L01-019-theater": {
"constraint_kinds": {
"eq": 10,
"implies": 2
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 1,
"V11": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 12,
"forward_chain_rounds": 2,
"n_constraints": 12,
"n_variables": 12,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-000-observatory": {
"constraint_kinds": {
"eq": 13,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 0,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-001-hotel": {
"constraint_kinds": {
"eq": 9,
"implies": 7
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 0,
"V2": 3,
"V3": 0,
"V4": 1,
"V5": 2,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 3,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-002-maritime": {
"constraint_kinds": {
"eq": 13,
"implies": 3
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-003-hotel": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 10,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-004-hotel": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-005-theater": {
"constraint_kinds": {
"eq": 9,
"implies": 7
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 2,
"V6": 0,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.4375,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-006-observatory": {
"constraint_kinds": {
"eq": 13,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.8125,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-007-observatory": {
"constraint_kinds": {
"eq": 10,
"implies": 6
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V2": 2,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-008-hotel": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5625,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-009-observatory": {
"constraint_kinds": {
"eq": 12,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 2,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-010-observatory": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 1,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-011-observatory": {
"constraint_kinds": {
"eq": 14,
"implies": 2
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 2,
"V11": 1,
"V12": 1,
"V13": 0,
"V14": 0,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 12,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-012-theater": {
"constraint_kinds": {
"eq": 9,
"implies": 7
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-013-manor": {
"constraint_kinds": {
"eq": 7,
"implies": 9
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 2,
"V6": 0,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.4375,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-014-hotel": {
"constraint_kinds": {
"eq": 13,
"implies": 3
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-015-theater": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-016-hotel": {
"constraint_kinds": {
"eq": 12,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 12,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-017-theater": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6875,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 2,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-018-manor": {
"constraint_kinds": {
"eq": 15,
"implies": 1
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.8125,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 1,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L02-019-maritime": {
"constraint_kinds": {
"eq": 11,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 2,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 1,
"V2": 2,
"V3": 0,
"V4": 2,
"V5": 3,
"V6": 0,
"V7": 0,
"V8": 3,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 16,
"forward_chain_rounds": 3,
"n_constraints": 16,
"n_variables": 16,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-000-maritime": {
"constraint_kinds": {
"eq": 13,
"implies": 7
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 0,
"V2": 1,
"V3": 3,
"V4": 0,
"V5": 0,
"V6": 2,
"V7": 1,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.55,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-001-theater": {
"constraint_kinds": {
"eq": 9,
"implies": 11
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 2,
"V11": 3,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 3,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 3,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.4,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-002-hotel": {
"constraint_kinds": {
"eq": 18,
"implies": 2
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 1,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 1,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-003-theater": {
"constraint_kinds": {
"eq": 12,
"implies": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 2,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 0,
"V2": 2,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 3,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6,
"direct_assignments": 12,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-004-maritime": {
"constraint_kinds": {
"eq": 14,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 4,
"V12": 0,
"V13": 0,
"V14": 3,
"V15": 0,
"V16": 0,
"V17": 2,
"V18": 2,
"V19": 0,
"V2": 2,
"V3": 0,
"V4": 1,
"V5": 1,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.65,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 4,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-005-hotel": {
"constraint_kinds": {
"eq": 11,
"implies": 9
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 1,
"V15": 2,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 2,
"V2": 1,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 2,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.45,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 2,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-006-observatory": {
"constraint_kinds": {
"eq": 14,
"implies": 6
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 0,
"V11": 4,
"V12": 1,
"V13": 2,
"V14": 3,
"V15": 1,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 2,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6,
"direct_assignments": 12,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 4,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-007-observatory": {
"constraint_kinds": {
"eq": 12,
"implies": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 1,
"V11": 3,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 3,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 3,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.45,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-008-theater": {
"constraint_kinds": {
"eq": 16,
"implies": 4
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 1,
"V13": 0,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 2,
"V18": 1,
"V19": 1,
"V2": 0,
"V3": 2,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.65,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 2,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-009-observatory": {
"constraint_kinds": {
"eq": 13,
"implies": 7
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 4,
"V12": 0,
"V13": 2,
"V14": 3,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 0,
"V2": 3,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.55,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 4,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-010-observatory": {
"constraint_kinds": {
"eq": 12,
"implies": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 3,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 0,
"V2": 1,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 2,
"V7": 3,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.45,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-011-manor": {
"constraint_kinds": {
"eq": 15,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 1,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 1,
"V2": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 1,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-012-theater": {
"constraint_kinds": {
"eq": 14,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 1,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 1,
"V2": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.7,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 1,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-013-hotel": {
"constraint_kinds": {
"eq": 15,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 2,
"V16": 0,
"V17": 2,
"V18": 0,
"V19": 4,
"V2": 0,
"V3": 1,
"V4": 1,
"V5": 0,
"V6": 3,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.55,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 4,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-014-hotel": {
"constraint_kinds": {
"eq": 13,
"implies": 7
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 0,
"V2": 0,
"V3": 3,
"V4": 1,
"V5": 2,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.55,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-015-manor": {
"constraint_kinds": {
"eq": 10,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 3,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 2,
"V2": 1,
"V3": 3,
"V4": 0,
"V5": 0,
"V6": 2,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 10,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-016-theater": {
"constraint_kinds": {
"eq": 12,
"implies": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 1,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 0,
"V2": 1,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 1
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 10,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 1,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-017-theater": {
"constraint_kinds": {
"eq": 13,
"implies": 7
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 0,
"V2": 2,
"V3": 3,
"V4": 0,
"V5": 1,
"V6": 1,
"V7": 3,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.55,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-018-theater": {
"constraint_kinds": {
"eq": 11,
"implies": 9
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 2,
"V11": 3,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 3,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 0,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 2,
"V7": 1,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.45,
"direct_assignments": 9,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 3,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L03-019-observatory": {
"constraint_kinds": {
"eq": 11,
"implies": 9
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V10": 1,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 2,
"V2": 2,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 10,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 20,
"forward_chain_rounds": 2,
"n_constraints": 20,
"n_variables": 20,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-000-observatory": {
"constraint_kinds": {
"eq": 19,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 2,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 1,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 2,
"V7": 0,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-001-theater": {
"constraint_kinds": {
"eq": 9,
"implies": 15
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 2,
"V11": 3,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 3,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 3,
"V3": 3,
"V4": 0,
"V5": 1,
"V6": 2,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.2916666666666667,
"direct_assignments": 7,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-002-manor": {
"constraint_kinds": {
"eq": 16,
"implies": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 1,
"V13": 0,
"V14": 0,
"V15": 2,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 0,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 1,
"V3": 3,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-003-hotel": {
"constraint_kinds": {
"eq": 16,
"implies": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 3,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 1,
"V2": 0,
"V20": 0,
"V21": 0,
"V22": 2,
"V23": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5416666666666666,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-004-maritime": {
"constraint_kinds": {
"eq": 18,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 1,
"V3": 1,
"V4": 1,
"V5": 1,
"V6": 1,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-005-hotel": {
"constraint_kinds": {
"eq": 13,
"implies": 10,
"neq": 4
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 1,
"V11": 0,
"V13": 1,
"V14": 1,
"V15": 2,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 1,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 1,
"V3": 2,
"V4": 0,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.4583333333333333,
"direct_assignments": 11,
"forward_chain_complete": false,
"forward_chain_fraction": 0.9166666666666666,
"forward_chain_recovered": 22,
"forward_chain_rounds": 2,
"n_constraints": 27,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": [
"V5",
"V12"
]
},
"inst-strata-L04-006-hotel": {
"constraint_kinds": {
"eq": 14,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 2,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 2,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 2,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 1,
"V23": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 1,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-007-manor": {
"constraint_kinds": {
"eq": 19,
"implies": 5
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 1,
"V23": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.75,
"direct_assignments": 18,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-008-observatory": {
"constraint_kinds": {
"eq": 12,
"implies": 12
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 1,
"V12": 1,
"V13": 2,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 2,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 1,
"V3": 3,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 3,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.4583333333333333,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-009-maritime": {
"constraint_kinds": {
"eq": 16,
"implies": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 2,
"V15": 3,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 0,
"V22": 2,
"V23": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 3,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-010-observatory": {
"constraint_kinds": {
"eq": 16,
"implies": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 2,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 1,
"V2": 2,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 1,
"V3": 1,
"V4": 1,
"V5": 0,
"V6": 2,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5416666666666666,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-011-observatory": {
"constraint_kinds": {
"eq": 17,
"implies": 7
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 2,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 1,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 2,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 0,
"V3": 3,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-012-theater": {
"constraint_kinds": {
"eq": 17,
"implies": 7
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 2,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.625,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-013-theater": {
"constraint_kinds": {
"eq": 18,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 2,
"V22": 1,
"V23": 0,
"V3": 2,
"V4": 1,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-014-manor": {
"constraint_kinds": {
"eq": 14,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 1,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 0,
"V3": 3,
"V4": 0,
"V5": 0,
"V6": 2,
"V7": 3,
"V8": 1,
"V9": 1
},
"direct_assignment_fraction": 0.4583333333333333,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-015-observatory": {
"constraint_kinds": {
"eq": 15,
"implies": 9
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 0,
"V15": 3,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 0,
"V2": 2,
"V20": 0,
"V21": 0,
"V22": 2,
"V23": 3,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 2,
"V8": 1,
"V9": 1
},
"direct_assignment_fraction": 0.5833333333333334,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-016-observatory": {
"constraint_kinds": {
"eq": 13,
"implies": 11
},
"derivation_rounds": {
"V0": 1,
"V1": 2,
"V10": 1,
"V11": 0,
"V12": 0,
"V13": 1,
"V14": 3,
"V15": 0,
"V16": 1,
"V17": 0,
"V18": 3,
"V19": 1,
"V2": 0,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 1,
"V3": 2,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 4,
"V8": 0,
"V9": 2
},
"direct_assignment_fraction": 0.4583333333333333,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 4,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-017-observatory": {
"constraint_kinds": {
"eq": 20,
"implies": 4
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 3,
"V11": 2,
"V12": 0,
"V13": 2,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6666666666666666,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 3,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-018-hotel": {
"constraint_kinds": {
"eq": 12,
"implies": 12
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 1,
"V14": 0,
"V15": 1,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 1,
"V3": 1,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 2,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.4583333333333333,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L04-019-maritime": {
"constraint_kinds": {
"eq": 18,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 2,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 0,
"V3": 1,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 2,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.7083333333333334,
"direct_assignments": 17,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 24,
"forward_chain_rounds": 2,
"n_constraints": 24,
"n_variables": 24,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-000-theater": {
"constraint_kinds": {
"eq": 14,
"implies": 14
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 2,
"V12": 2,
"V13": 0,
"V14": 1,
"V15": 3,
"V16": 1,
"V17": 2,
"V18": 0,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 2,
"V23": 0,
"V24": 1,
"V25": 0,
"V26": 1,
"V27": 1,
"V3": 3,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 1,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.39285714285714285,
"direct_assignments": 11,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-001-maritime": {
"constraint_kinds": {
"eq": 17,
"implies": 11
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 2,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 1,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 2,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 0,
"V3": 0,
"V4": 3,
"V5": 0,
"V6": 1,
"V7": 2,
"V8": 1,
"V9": 2
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-002-maritime": {
"constraint_kinds": {
"eq": 16,
"implies": 12
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 2,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 1,
"V19": 3,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 2,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-003-manor": {
"constraint_kinds": {
"eq": 18,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 2,
"V13": 0,
"V14": 2,
"V15": 0,
"V16": 1,
"V17": 0,
"V18": 0,
"V19": 1,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 1,
"V24": 1,
"V25": 0,
"V26": 1,
"V27": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 2,
"V8": 3,
"V9": 0
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-004-maritime": {
"constraint_kinds": {
"eq": 18,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 1,
"V17": 2,
"V18": 0,
"V19": 1,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 3,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 1,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-005-theater": {
"constraint_kinds": {
"eq": 19,
"implies": 9
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 1,
"V11": 2,
"V12": 1,
"V13": 4,
"V14": 5,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 1,
"V19": 0,
"V2": 3,
"V20": 0,
"V21": 0,
"V22": 3,
"V23": 4,
"V24": 2,
"V25": 0,
"V26": 0,
"V27": 0,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 1,
"V8": 4,
"V9": 0
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 5,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-006-hotel": {
"constraint_kinds": {
"eq": 18,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 1,
"V12": 2,
"V13": 1,
"V14": 0,
"V15": 0,
"V16": 1,
"V17": 0,
"V18": 0,
"V19": 2,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 1,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.6071428571428571,
"direct_assignments": 17,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 2,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-007-manor": {
"constraint_kinds": {
"eq": 18,
"implies": 10
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 1,
"V13": 0,
"V14": 2,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 0,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 2,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.6428571428571429,
"direct_assignments": 18,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 2,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-008-theater": {
"constraint_kinds": {
"eq": 20,
"implies": 8
},
"derivation_rounds": {
"V0": 1,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 1,
"V13": 4,
"V14": 0,
"V15": 0,
"V16": 2,
"V17": 0,
"V18": 0,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 2,
"V22": 0,
"V23": 1,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 3,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 0,
"V9": 0
},
"direct_assignment_fraction": 0.6071428571428571,
"direct_assignments": 17,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 4,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-009-theater": {
"constraint_kinds": {
"eq": 15,
"implies": 13
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 2,
"V12": 1,
"V13": 0,
"V14": 3,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 2,
"V19": 0,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 3,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 3,
"V3": 2,
"V4": 1,
"V5": 1,
"V6": 1,
"V7": 1,
"V8": 2,
"V9": 0
},
"direct_assignment_fraction": 0.42857142857142855,
"direct_assignments": 12,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-010-theater": {
"constraint_kinds": {
"eq": 19,
"implies": 9
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 1,
"V12": 1,
"V13": 1,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 3,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 2,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 1,
"V3": 0,
"V4": 0,
"V5": 2,
"V6": 1,
"V7": 2,
"V8": 0,
"V9": 3
},
"direct_assignment_fraction": 0.5357142857142857,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-011-hotel": {
"constraint_kinds": {
"eq": 17,
"implies": 11
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 1,
"V12": 1,
"V13": 1,
"V14": 3,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 1,
"V23": 2,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 0,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 1,
"V8": 1,
"V9": 2
},
"direct_assignment_fraction": 0.5357142857142857,
"direct_assignments": 15,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-012-observatory": {
"constraint_kinds": {
"eq": 16,
"implies": 12
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 0,
"V14": 3,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 1,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 1,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 1,
"V9": 2
},
"direct_assignment_fraction": 0.5,
"direct_assignments": 14,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-013-observatory": {
"constraint_kinds": {
"eq": 22,
"implies": 6
},
"derivation_rounds": {
"V0": 0,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 0,
"V13": 2,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 1,
"V18": 0,
"V19": 2,
"V2": 0,
"V20": 0,
"V21": 1,
"V22": 0,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 1,
"V3": 1,
"V4": 0,
"V5": 0,
"V6": 1,
"V7": 0,
"V8": 0,
"V9": 1
},
"direct_assignment_fraction": 0.7142857142857143,
"direct_assignments": 20,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 2,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-014-theater": {
"constraint_kinds": {
"eq": 20,
"implies": 8
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 1,
"V12": 1,
"V13": 2,
"V14": 0,
"V15": 0,
"V16": 2,
"V17": 1,
"V18": 1,
"V19": 0,
"V2": 0,
"V20": 0,
"V21": 0,
"V22": 1,
"V23": 0,
"V24": 1,
"V25": 0,
"V26": 0,
"V27": 0,
"V3": 0,
"V4": 0,
"V5": 0,
"V6": 0,
"V7": 1,
"V8": 1,
"V9": 0
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 2,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-015-maritime": {
"constraint_kinds": {
"eq": 16,
"implies": 12
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 0,
"V12": 2,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 0,
"V17": 2,
"V18": 2,
"V19": 3,
"V2": 1,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 2,
"V24": 1,
"V25": 0,
"V26": 1,
"V27": 0,
"V3": 0,
"V4": 0,
"V5": 1,
"V6": 0,
"V7": 1,
"V8": 0,
"V9": 3
},
"direct_assignment_fraction": 0.4642857142857143,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-016-manor": {
"constraint_kinds": {
"eq": 18,
"implies": 10
},
"derivation_rounds": {
"V0": 1,
"V1": 0,
"V10": 0,
"V11": 0,
"V12": 2,
"V13": 0,
"V14": 1,
"V15": 0,
"V16": 2,
"V17": 0,
"V18": 2,
"V19": 3,
"V2": 1,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 2,
"V3": 0,
"V4": 1,
"V5": 0,
"V6": 0,
"V7": 0,
"V8": 3,
"V9": 1
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 3,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-017-theater": {
"constraint_kinds": {
"eq": 17,
"implies": 11
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 2,
"V13": 1,
"V14": 0,
"V15": 0,
"V16": 1,
"V17": 0,
"V18": 0,
"V19": 2,
"V2": 2,
"V20": 0,
"V21": 0,
"V22": 0,
"V23": 0,
"V24": 0,
"V25": 0,
"V26": 0,
"V27": 0,
"V3": 0,
"V4": 2,
"V5": 0,
"V6": 1,
"V7": 1,
"V8": 1,
"V9": 2
},
"direct_assignment_fraction": 0.5714285714285714,
"direct_assignments": 16,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 2,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-018-manor": {
"constraint_kinds": {
"eq": 10,
"implies": 18
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 1,
"V11": 1,
"V12": 2,
"V13": 3,
"V14": 4,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 2,
"V19": 3,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 2,
"V24": 4,
"V25": 1,
"V26": 1,
"V27": 1,
"V3": 0,
"V4": 3,
"V5": 0,
"V6": 0,
"V7": 2,
"V8": 3,
"V9": 3
},
"direct_assignment_fraction": 0.2857142857142857,
"direct_assignments": 8,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 4,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
},
"inst-strata-L05-019-theater": {
"constraint_kinds": {
"eq": 14,
"implies": 14
},
"derivation_rounds": {
"V0": 0,
"V1": 1,
"V10": 0,
"V11": 1,
"V12": 0,
"V13": 3,
"V14": 0,
"V15": 0,
"V16": 0,
"V17": 0,
"V18": 0,
"V19": 4,
"V2": 2,
"V20": 0,
"V21": 1,
"V22": 2,
"V23": 1,
"V24": 0,
"V25": 0,
"V26": 1,
"V27": 2,
"V3": 3,
"V4": 1,
"V5": 0,
"V6": 1,
"V7": 2,
"V8": 0,
"V9": 4
},
"direct_assignment_fraction": 0.4642857142857143,
"direct_assignments": 13,
"forward_chain_complete": true,
"forward_chain_fraction": 1.0,
"forward_chain_recovered": 28,
"forward_chain_rounds": 4,
"n_constraints": 28,
"n_variables": 28,
"note": "Depth measures this propagator, not minimum proof length or intrinsic difficulty.",
"unresolved_by_forward_chain": []
}
},
"families_measured": 120,
"numeric_summary": {
"direct_assignment_fraction": {
"max": 1.0,
"mean": 0.5939831349206349,
"min": 0.25,
"n": 120
},
"direct_assignments": {
"max": 20,
"mean": 10.383333333333333,
"min": 3,
"n": 120
},
"forward_chain_fraction": {
"max": 1.0,
"mean": 0.9972222222222222,
"min": 0.75,
"n": 120
},
"forward_chain_recovered": {
"max": 28,
"mean": 17.958333333333332,
"min": 8,
"n": 120
},
"forward_chain_rounds": {
"max": 5,
"mean": 2.158333333333333,
"min": 0,
"n": 120
},
"n_constraints": {
"max": 28,
"mean": 18.05,
"min": 8,
"n": 120
},
"n_variables": {
"max": 28,
"mean": 18.0,
"min": 8,
"n": 120
}
}
}The full solver inputs (all realizations), hidden fact patterns, and per-record details are large, so they live in a companion file that loads separately:
The companion file contains every solver input, the hidden fact pattern per world, decision policies with measured difficulty, and the full per-record JSON for all providers.
Earned decisions (/100) — correct actions on an exactly-right world
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L00-000-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-000-theater--explicit-r1 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "1"
} |
| inst-strata-L00-000-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-000-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-000-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-000-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-000-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-000-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "1"
} |
| inst-strata-L00-000-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-000-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-000-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "1"
} |
| inst-strata-L00-000-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-000-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": 1
} |
| inst-strata-L00-000-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-001-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-001-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-001-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-001-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-001-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-001-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-001-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-001-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-001-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L00-001-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-001-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-001-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-001-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": "3"
} |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-001-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 3
} |
| inst-strata-L00-001-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-001-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-001-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-002-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-002-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-6-sol | answered | 0.0% | {
"operation": "register",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": "1"
} |
| inst-strata-L00-002-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-002-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-002-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-002-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-002-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-002-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-002-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-002-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L00-002-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-002-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-002-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-002-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": "1"
} |
| inst-strata-L00-002-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": 1
} |
| inst-strata-L00-002-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-002-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-002-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-003-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-003-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-003-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-003-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-003-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-003-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-003-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-003-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-003-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-003-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-003-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Juno"
} |
| inst-strata-L00-003-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-004-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-004-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L00-004-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-004-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-004-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-004-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-004-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-004-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-004-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-004-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-004-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-004-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-004-manor--implicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "3"
} |
| inst-strata-L00-004-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "seed ledger",
"value": 3
} |
| inst-strata-L00-004-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-004-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-004-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L00-005-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-005-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-005-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-005-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-005-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-005-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-005-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-005-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-005-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L00-005-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-005-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-005-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L00-005-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L00-005-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-005-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-005-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-006-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-006-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-006-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-006-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-006-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "signed out under Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-006-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-006-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-006-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-006-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-006-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia's mark"
} |
| inst-strata-L00-006-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "carried Galia's mark"
} |
| inst-strata-L00-006-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-006-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-006-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-006-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "signed out under Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-006-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "understudy roster",
"value": "Galia"
} |
| inst-strata-L00-006-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-007-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-007-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--explicit-r2 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-007-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-007-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-007-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-007-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-007-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-007-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-007-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-007-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-007-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lift log",
"value": "Iris"
} |
| inst-strata-L00-007-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-007-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-007-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-008-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-008-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-008-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-008-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-008-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-008-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-008-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-008-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-008-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-008-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-008-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-008-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | claude-fable-5.1 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": "Roan"
} |
| inst-strata-L00-008-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-008-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-009-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-009-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | glm-5.3-flash | transport_error | — | null |
| inst-strata-L00-009-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-009-maritime--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-009-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro's mark"
} |
| inst-strata-L00-009-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-009-maritime--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-009-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-009-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-009-maritime--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-009-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-009-maritime--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-009-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro's mark"
} |
| inst-strata-L00-009-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-009-maritime--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Ferro"
} |
| inst-strata-L00-009-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-010-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-010-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L00-010-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-010-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-010-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-010-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-010-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-010-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-010-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-010-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-010-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "call sheet",
"value": "Ansel"
} |
| inst-strata-L00-010-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-010-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-010-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-011-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-011-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L00-011-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-011-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-011-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-011-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-011-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-011-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-011-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-011-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-011-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-011-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L00-011-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 2
} |
| inst-strata-L00-011-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-011-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-011-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-012-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L00-012-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": "4"
} |
| inst-strata-L00-012-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-012-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-012-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-012-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-012-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L00-012-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-012-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-012-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-012-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-012-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-012-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-012-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-012-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-012-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-012-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-012-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-012-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-012-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "register",
"subject": "mill ledger",
"value": 2
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-012-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mill ledger",
"value": 4
} |
| inst-strata-L00-012-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-012-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-012-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-013-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-013-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-013-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-013-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-013-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-013-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-013-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-013-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-013-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-013-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-013-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-013-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-013-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-013-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-013-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L00-013-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "bar ledger",
"value": "Galia"
} |
| inst-strata-L00-013-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-013-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-013-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-014-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-014-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-014-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-014-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-014-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-014-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-014-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-014-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-014-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-014-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-014-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-014-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": "Cassia"
} |
| inst-strata-L00-014-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-014-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-014-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-015-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-015-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-015-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-015-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-015-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-015-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-015-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-015-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-015-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L00-015-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": "1"
} |
| inst-strata-L00-015-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-015-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-015-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-015-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": "1"
} |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-015-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-015-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": "1"
} |
| inst-strata-L00-015-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "broker's stamp",
"value": 1
} |
| inst-strata-L00-015-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-015-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-015-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-016-maritime--explicit-r1 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-016-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-016-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-6-sol | answered | 0.0% | {
"operation": "hold",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-016-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-016-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-016-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-016-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-016-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-016-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "crate mark",
"value": "3"
} |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L00-016-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "crate mark",
"value": "3"
} |
| inst-strata-L00-016-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-016-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-016-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-016-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "crate mark",
"value": 3
} |
| inst-strata-L00-016-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-016-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-016-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gemini-3.8-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-sonnet-5 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | minimax-m3 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": "4"
} |
| inst-strata-L00-017-manor--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-017-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-6-sol | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | gemini-3.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gemini-3.6-flash | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-017-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-017-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-017-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-017-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-017-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-017-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-017-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-017-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-017-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L00-017-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tenant roll",
"value": 4
} |
| inst-strata-L00-017-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-017-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-017-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin's mark"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin's mark"
} |
| inst-strata-L00-018-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-018-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-018-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin's mark"
} |
| inst-strata-L00-018-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-018-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-018-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-018-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-018-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-018-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin's mark"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-018-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin's mark"
} |
| inst-strata-L00-018-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-018-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-018-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-018-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L00-018-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "Corin"
} |
| inst-strata-L00-018-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-018-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-018-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-019-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--explicit-r2 | glm-4.7-flash | answered | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-019-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-019-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L00-019-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-019-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-019-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L00-019-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-019-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-019-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L00-019-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "call sheet",
"value": "Wren"
} |
| inst-strata-L00-019-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 2
} |
| inst-strata-L00-019-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "call sheet",
"value": "2"
} |
| inst-strata-L00-019-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L00-019-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L00-019-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L01-000-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-000-hotel--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-000-hotel--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-000-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-000-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-000-hotel--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-000-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-000-hotel--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-000-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-000-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-000-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-000-hotel--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "patrol book",
"value": "Silas"
} |
| inst-strata-L01-000-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L01-001-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-001-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-001-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L01-001-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-001-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-001-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-001-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-001-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-001-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-001-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L01-001-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-001-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-001-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-001-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-001-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L01-001-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-001-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-001-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Lyra"
} |
| inst-strata-L01-001-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-001-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-001-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-002-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-002-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-002-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-002-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-002-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-002-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-002-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-002-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-002-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-002-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-002-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-002-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-002-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-002-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-002-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-002-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-002-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": "Petra"
} |
| inst-strata-L01-002-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-002-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-002-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-003-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-6-sol | answered | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-003-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-003-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-003-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-003-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-003-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-003-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-003-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-003-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-003-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "Wren"
} |
| inst-strata-L01-003-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-003-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-003-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-004-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-004-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-004-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-004-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-sonnet-5 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-opus-5 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | qwen3.8-27b | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-004-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-004-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-004-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-004-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-004-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-004-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-004-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L01-004-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": "2"
} |
| inst-strata-L01-004-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-004-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-004-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-004-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | qwen3.8-27b | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": 2
} |
| inst-strata-L01-004-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-004-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-004-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-005-maritime--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-005-maritime--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-005-maritime--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-005-maritime--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-005-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-005-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-005-maritime--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": 3
} |
| inst-strata-L01-005-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-006-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-006-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-006-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-006-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-006-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-006-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-006-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-006-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-006-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-006-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-006-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-006-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prompt book",
"value": "Halden"
} |
| inst-strata-L01-006-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-006-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-006-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-007-hotel--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-007-hotel--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-007-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-007-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-007-hotel--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-007-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | glm-5.3-flash | answered | 0.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "3"
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-007-hotel--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L01-007-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "3"
} |
| inst-strata-L01-007-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-007-hotel--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": 3
} |
| inst-strata-L01-007-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-008-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-008-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-008-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-008-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-008-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r2 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": 4
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-008-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Sable"
} |
| inst-strata-L01-008-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-opus-5.5 | answered | 0.0% | {
"operation": "register",
"subject": "costume plot",
"value": "2"
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-009-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-009-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | glm-5.3-flash | transport_error | — | null |
| inst-strata-L01-009-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "costume plot",
"value": "2"
} |
| inst-strata-L01-009-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-009-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-009-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-009-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-009-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-009-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-009-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "costume plot",
"value": "2"
} |
| inst-strata-L01-009-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-009-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-009-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-009-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-009-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-009-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "costume plot",
"value": 2
} |
| inst-strata-L01-009-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "costume plot",
"value": "2"
} |
| inst-strata-L01-009-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-009-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-009-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-010-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-010-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-010-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-010-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-010-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-010-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-010-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-010-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-010-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L01-010-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "parish register",
"value": "1"
} |
| inst-strata-L01-010-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-010-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "parish register",
"value": 1
} |
| inst-strata-L01-010-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-011-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-011-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-011-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-011-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-011-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-011-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-011-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-011-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-011-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-011-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-011-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-011-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-011-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-011-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-011-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-011-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-011-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-011-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-011-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "costume plot",
"value": "Silas"
} |
| inst-strata-L01-011-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-011-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-011-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-012-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-012-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-012-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-012-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-012-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-012-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-012-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-012-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-012-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-012-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas's mark"
} |
| inst-strata-L01-012-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-012-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-012-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-012-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-012-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "Silas"
} |
| inst-strata-L01-012-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-013-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-013-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-013-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-013-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-013-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-013-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-013-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-013-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-013-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-013-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-013-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 1
} |
| inst-strata-L01-013-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-014-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-014-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-014-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-014-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-014-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-014-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-014-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-014-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-014-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-014-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": "3"
} |
| inst-strata-L01-014-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-014-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": "3"
} |
| inst-strata-L01-014-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-014-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L01-014-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-015-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-015-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-015-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-015-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-015-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-015-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-015-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-015-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-015-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-015-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-015-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-015-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-015-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "parish register",
"value": 2
} |
| inst-strata-L01-015-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-015-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-015-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-016-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-016-maritime--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-016-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | claude-fable-5.1 | transport_error | — | null |
| inst-strata-L01-016-maritime--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-016-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-016-maritime--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-016-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-016-maritime--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-016-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-016-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-016-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": "3"
} |
| inst-strata-L01-016-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-016-maritime--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 3
} |
| inst-strata-L01-016-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-017-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-017-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-017-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-017-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-017-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-017-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-017-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-017-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-017-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-017-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L01-017-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-017-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-017-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-017-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-017-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Odile"
} |
| inst-strata-L01-017-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-017-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L01-017-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-018-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-018-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-018-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-018-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-018-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-018-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-018-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-018-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-018-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-018-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-018-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-018-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-018-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "tithe receipt",
"value": 4
} |
| inst-strata-L01-018-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L01-019-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-019-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-019-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | glm-5.3-flash | transport_error | — | null |
| inst-strata-L01-019-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-019-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L01-019-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-019-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L01-019-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": "2"
} |
| inst-strata-L01-019-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-019-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L01-019-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L01-019-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L01-019-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "rehearsal log",
"value": 2
} |
| inst-strata-L01-019-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L02-000-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-000-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-000-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-000-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-000-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-000-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-000-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-000-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-000-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-000-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-000-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-000-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-000-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Cassia"
} |
| inst-strata-L02-000-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-001-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L02-001-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-001-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-001-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "register",
"subject": "left-luggage ticket",
"value": "Lyra"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-6-luna | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-001-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-001-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-001-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-001-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-001-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-001-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-001-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-001-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-001-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "left-luggage ticket",
"value": "Halden"
} |
| inst-strata-L02-001-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-001-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-001-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-002-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-002-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-002-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-002-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-002-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-002-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-002-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-002-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-002-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-002-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-002-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-002-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-002-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "consignment tag",
"value": "Marek"
} |
| inst-strata-L02-002-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-002-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-002-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-003-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-003-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": "3"
} |
| inst-strata-L02-003-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-003-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-003-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-003-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": "3"
} |
| inst-strata-L02-003-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-003-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-003-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-003-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-003-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-003-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-003-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-003-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-003-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-003-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": "3"
} |
| inst-strata-L02-003-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-003-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-003-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-003-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L02-003-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "switchboard log",
"value": 3
} |
| inst-strata-L02-003-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-003-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-003-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-004-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-004-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-004-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-004-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-004-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-004-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-004-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-004-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-004-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-004-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-004-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-004-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-004-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Halden"
} |
| inst-strata-L02-004-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-004-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-004-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-005-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-005-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-005-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-005-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-005-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": "4"
} |
| inst-strata-L02-005-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-005-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-005-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-005-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-005-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-005-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-005-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | glm-5.3-flash | answered | 0.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 3
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-005-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-005-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-005-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": 4
} |
| inst-strata-L02-005-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "ticket manifest",
"value": "4"
} |
| inst-strata-L02-005-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-005-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-005-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-006-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-006-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-006-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-006-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-006-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-006-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-006-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-006-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-006-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-006-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "plate register",
"value": "2"
} |
| inst-strata-L02-006-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-006-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-006-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "plate register",
"value": 2
} |
| inst-strata-L02-006-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-006-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-006-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin's mark"
} |
| inst-strata-L02-007-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-007-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-fable-5 | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-007-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-007-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-007-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-007-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-007-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-007-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-007-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-007-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-007-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-007-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-007-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "mess roster",
"value": "Corin"
} |
| inst-strata-L02-007-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "mess roster",
"value": 1
} |
| inst-strata-L02-007-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-007-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-007-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-008-hotel--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-008-hotel--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-008-hotel--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-008-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-008-hotel--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-008-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile's mark"
} |
| inst-strata-L02-008-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-008-hotel--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cellar card",
"value": "Odile"
} |
| inst-strata-L02-008-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": "1"
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-009-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L02-009-observatory--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-009-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-009-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | glm-5.3-flash | invalid_response | 0.0% | null |
| inst-strata-L02-009-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": "1"
} |
| inst-strata-L02-009-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-009-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "cryostat sheet",
"value": 1
} |
| inst-strata-L02-009-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-010-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "power schedule",
"value": "4"
} |
| inst-strata-L02-010-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-010-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-010-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-010-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-010-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-010-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-010-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-010-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-010-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-010-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-010-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "power schedule",
"value": 4
} |
| inst-strata-L02-010-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "power schedule",
"value": "4"
} |
| inst-strata-L02-010-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-010-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-010-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-011-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-011-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-011-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-011-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L02-011-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-011-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-011-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-011-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-011-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-011-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-011-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-011-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-011-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-011-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-011-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-011-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-011-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-011-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 2
} |
| inst-strata-L02-011-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-011-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-011-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-012-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-012-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L02-012-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | glm-5.3-flash | transport_error | — | null |
| inst-strata-L02-012-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-012-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-012-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-012-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-012-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-012-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-012-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L02-012-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-012-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-012-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-012-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-012-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L02-012-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "Vela"
} |
| inst-strata-L02-012-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-012-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-012-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "register",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-013-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-013-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-013-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": "2"
} |
| inst-strata-L02-013-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-013-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "register",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-013-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "carrier's waybill",
"value": 2
} |
| inst-strata-L02-013-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-014-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gemini-3.8-flash | transport_error | — | null |
| inst-strata-L02-014-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-014-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-014-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-014-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-014-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-014-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-014-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-014-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-014-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-014-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-014-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-014-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-014-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-014-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-014-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-014-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-014-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "funicular ticket",
"value": "Galia"
} |
| inst-strata-L02-014-hotel--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L02-014-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-014-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-014-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-015-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-015-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-015-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-015-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-015-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-015-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-015-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-015-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-015-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-015-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-015-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-015-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-015-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Sable"
} |
| inst-strata-L02-015-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-015-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-015-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-015-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-015-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-015-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wardrobe ledger",
"value": "Bram"
} |
| inst-strata-L02-015-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-015-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-015-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-016-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-016-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-016-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-016-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-016-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-016-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-016-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-016-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-016-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-016-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-016-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-016-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-016-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-016-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "terrace menu",
"value": "Ferro"
} |
| inst-strata-L02-016-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-016-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-016-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-017-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-017-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-017-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-017-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": "1"
} |
| inst-strata-L02-017-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-017-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-017-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-017-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-017-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-017-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-017-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 1
} |
| inst-strata-L02-017-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-018-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-018-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-018-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-018-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-018-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-018-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "wool account",
"value": "3"
} |
| inst-strata-L02-018-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-018-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-018-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-018-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-018-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "wool account",
"value": 3
} |
| inst-strata-L02-018-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": "2"
} |
| inst-strata-L02-019-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": "2"
} |
| inst-strata-L02-019-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-019-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-019-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-019-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-019-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L02-019-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-019-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-019-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-019-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L02-019-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-019-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-019-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L02-019-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L02-019-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tide-table entry",
"value": 2
} |
| inst-strata-L02-019-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L02-019-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L02-019-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L03-000-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-000-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-000-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-000-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-6-sol | answered | 0.0% | {
"operation": "register",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-000-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-000-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-000-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-000-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-000-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-000-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-000-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-000-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-000-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-000-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | minimax-m3 | answered | 0.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": "2"
} |
| inst-strata-L03-000-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "chandlery invoice",
"value": 2
} |
| inst-strata-L03-000-maritime--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-000-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-000-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-000-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-001-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-001-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-001-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-001-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-001-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-001-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-001-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-001-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-001-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | glm-5.3-flash | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-001-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-001-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L03-001-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-001-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-001-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "guest register",
"value": "altered after the fire"
} |
| inst-strata-L03-002-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r1 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r1 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-002-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-002-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "guest register",
"value": "5"
} |
| inst-strata-L03-002-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-002-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-002-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-002-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-002-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-002-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-002-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | qwen3.8-27b | answered | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-002-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "guest register",
"value": 5
} |
| inst-strata-L03-002-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-002-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-002-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-003-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-003-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-003-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-003-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-003-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-003-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-003-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-003-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-003-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-003-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-003-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-003-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-003-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-003-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "prop list",
"value": 3
} |
| inst-strata-L03-003-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-003-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-003-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-004-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r1 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": "4"
} |
| inst-strata-L03-004-maritime--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-004-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-004-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-004-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-004-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-004-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-004-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-004-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-004-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-004-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-004-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-004-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-004-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-004-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-004-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-004-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-004-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-004-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-004-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L03-004-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "pilot's note",
"value": 4
} |
| inst-strata-L03-004-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-004-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-004-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": "2"
} |
| inst-strata-L03-005-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-005-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-005-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-005-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-005-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-005-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-005-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-005-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-005-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-005-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-005-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-005-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-005-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": 2
} |
| inst-strata-L03-005-hotel--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "shuttle manifest",
"value": "2"
} |
| inst-strata-L03-005-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-005-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-005-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-006-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-006-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-006-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-006-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-006-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-006-observatory--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-006-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": 4
} |
| inst-strata-L03-006-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-007-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-007-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-007-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-007-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-007-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-007-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-007-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-007-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-007-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-007-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-007-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-007-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-007-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "tape index",
"value": "Petra"
} |
| inst-strata-L03-007-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-007-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-007-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-008-theater--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "2"
} |
| inst-strata-L03-008-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-008-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L03-008-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "register",
"subject": "box-office tally",
"value": "2"
} |
| inst-strata-L03-008-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-008-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-008-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": "2"
} |
| inst-strata-L03-008-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-008-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-008-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-008-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-008-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "box-office tally",
"value": 2
} |
| inst-strata-L03-008-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-009-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "seeing report",
"value": "1"
} |
| inst-strata-L03-009-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-009-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-009-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-009-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-009-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seeing report",
"value": 1
} |
| inst-strata-L03-009-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-6-sol | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gemini-3.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-fable-5 | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r1 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-010-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-010-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r1 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-010-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-010-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "generator book",
"value": 3
} |
| inst-strata-L03-010-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-011-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-011-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-011-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-011-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-011-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-011-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-011-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-011-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-011-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-011-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-011-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-011-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-011-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "stable diary",
"value": "Bram"
} |
| inst-strata-L03-011-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-012-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-012-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-012-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-012-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-012-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-012-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-012-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-012-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-012-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-012-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-012-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-012-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Lyra"
} |
| inst-strata-L03-012-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-013-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-013-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-013-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-013-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-013-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-013-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-013-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-013-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-013-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-013-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-013-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-013-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | minimax-m3 | answered | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-013-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-013-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "wax-bench log",
"value": "Cassia"
} |
| inst-strata-L03-013-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-013-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-013-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-014-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-014-hotel--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-014-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-014-hotel--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-014-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-014-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-014-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-014-hotel--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-014-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-014-hotel--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-014-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-014-hotel--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "night auditor sheet",
"value": "Petra"
} |
| inst-strata-L03-014-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-015-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-015-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-015-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-015-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-015-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-015-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-015-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-015-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-015-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-015-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-015-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-015-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-015-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-015-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Marek"
} |
| inst-strata-L03-015-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-015-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-015-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-016-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-016-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-016-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-016-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-016-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-016-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-016-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-016-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-016-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-016-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-016-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-016-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-016-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-016-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-016-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-016-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-016-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seating chart",
"value": "Sable"
} |
| inst-strata-L03-016-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-016-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-016-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-017-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | glm-5.3-flash | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-017-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-017-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-017-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-017-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-017-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-017-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-017-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-017-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-017-theater--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r1 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-017-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-017-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-017-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-017-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": "green-room register"
} |
| inst-strata-L03-017-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-017-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-017-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "understudy roster",
"value": 2
} |
| inst-strata-L03-017-theater--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-017-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-017-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-017-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "prop list",
"value": "5"
} |
| inst-strata-L03-018-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-018-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-018-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-018-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L03-018-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-018-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-018-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-018-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-018-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-018-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-018-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-018-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-018-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-018-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "prop list",
"value": 5
} |
| inst-strata-L03-018-theater--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "prop list",
"value": "5"
} |
| inst-strata-L03-018-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-018-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L03-018-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": "5"
} |
| inst-strata-L03-019-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-019-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-019-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-019-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L03-019-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": "5"
} |
| inst-strata-L03-019-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L03-019-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "cryostat sheet",
"value": 5
} |
| inst-strata-L03-019-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L04-000-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-000-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | deepseek-v4-pro | answered | 0.0% | null |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile's mark"
} |
| inst-strata-L04-000-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-000-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-000-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-000-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-000-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-000-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-000-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "register",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-000-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile's mark"
} |
| inst-strata-L04-000-observatory--implicit-r2 | minimax-m3 | answered | 0.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile's mark"
} |
| inst-strata-L04-000-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L04-000-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "flare alert sheet",
"value": "Odile's mark"
} |
| inst-strata-L04-000-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-000-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "flare alert sheet",
"value": "Odile"
} |
| inst-strata-L04-000-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-6-sol | answered | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | deepseek-4.1-flash | transport_error | — | null |
| inst-strata-L04-001-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-001-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-001-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-001-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-001-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-001-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-001-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-001-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-001-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-001-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "call sheet",
"value": "1"
} |
| inst-strata-L04-001-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-001-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "call sheet",
"value": 1
} |
| inst-strata-L04-001-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-001-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-001-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "kennel book",
"value": "4"
} |
| inst-strata-L04-002-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-002-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-002-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-002-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-002-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-002-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-002-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-002-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-002-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-002-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-002-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-002-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gemini-3.8-flash | transport_error | — | null |
| inst-strata-L04-002-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "kennel book",
"value": 4
} |
| inst-strata-L04-002-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-002-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-002-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | deepseek-4.1-flash | transport_error | — | null |
| inst-strata-L04-003-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | mimo-v2.6-flash | transport_error | — | null |
| inst-strata-L04-003-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-003-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-003-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-003-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-003-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-003-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-003-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-003-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-003-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-003-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-003-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-003-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-003-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-003-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Wren"
} |
| inst-strata-L04-003-hotel--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-003-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-003-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-003-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-004-maritime--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-004-maritime--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-004-maritime--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--implicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-004-maritime--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-004-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-004-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-004-maritime--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": 1
} |
| inst-strata-L04-004-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | baseline:forward-chain | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gemini-3.7-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gemini-3.8-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | deepseek-4.1-flash | transport_error | — | null |
| inst-strata-L04-005-hotel--explicit-r1 | gemini-3.6-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 1
} |
| inst-strata-L04-005-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r1 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-005-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-005-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | baseline:forward-chain | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-6-sol | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "unresolved"
} |
| inst-strata-L04-005-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | kimi-k3 | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 2
} |
| inst-strata-L04-005-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--explicit-r2 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-005-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-005-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-005-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | baseline:forward-chain | answered | 0.0% | null |
| inst-strata-L04-005-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-005-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-005-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-005-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-005-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | baseline:forward-chain | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | gemini-3.7-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-5.6-terra | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 1
} |
| inst-strata-L04-005-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | gemini-3.6-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": "Halden"
} |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-005-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | grok-4.6 | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-005-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-005-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | baseline:forward-chain | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gemini-3.6-flash | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | gpt-5.6-luna | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 1
} |
| inst-strata-L04-005-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-005-hotel--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 5
} |
| inst-strata-L04-005-hotel--implicit-r2 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "hold",
"subject": "cellar card",
"value": 1
} |
| inst-strata-L04-005-hotel--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-005-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-005-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-005-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-006-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-006-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-006-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-006-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-006-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-006-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-006-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-006-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-006-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cashier's drop slip",
"value": "Dover"
} |
| inst-strata-L04-006-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-006-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-006-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-007-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-007-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-007-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-007-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-007-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-007-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-007-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-007-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-007-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-007-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-007-manor--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-007-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "apiary log",
"value": "Corin"
} |
| inst-strata-L04-007-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "filter log",
"value": "3"
} |
| inst-strata-L04-008-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-008-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-008-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-008-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-008-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-008-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-008-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-008-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-008-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-008-observatory--implicit-r1 | gpt-5.6-luna | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--implicit-r1 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L04-008-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-008-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-008-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-008-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-008-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "filter log",
"value": "3"
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "filter log",
"value": 3
} |
| inst-strata-L04-008-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "filter log",
"value": "3"
} |
| inst-strata-L04-008-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-008-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-008-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-009-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | glm-5.3-flash | answered | 0.0% | {
"operation": "register",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-009-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-009-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-009-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-009-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-009-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-009-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-009-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-009-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-009-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-009-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-009-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-009-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighthouse log",
"value": "Wren"
} |
| inst-strata-L04-009-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-009-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-009-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-010-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-010-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-010-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-fable-5 | answered | 0.0% | {
"operation": "hold",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-010-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-010-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-010-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-010-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-010-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-010-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-010-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-fable-5 | answered | 0.0% | {
"operation": "hold",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-010-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "tape index",
"value": 3
} |
| inst-strata-L04-010-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-011-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-011-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-fable-5 | transport_error | — | null |
| inst-strata-L04-011-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-011-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "instrument log",
"value": "wrong drop"
} |
| inst-strata-L04-011-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-011-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-011-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-011-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-011-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-011-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-011-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-011-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "instrument log",
"value": 4
} |
| inst-strata-L04-011-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-011-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-011-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-012-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | deepseek-4.1-flash | transport_error | — | null |
| inst-strata-L04-012-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-012-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-012-theater--explicit-r1 | glm-5.3-flash | answered | 0.0% | {
"operation": "register",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-012-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-012-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-012-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-012-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-012-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-012-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-012-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-012-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-012-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-012-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-012-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-012-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-012-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-012-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-012-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-012-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-012-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-012-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-012-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-012-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "hotel chit",
"value": 5
} |
| inst-strata-L04-012-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-012-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-012-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-013-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-013-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-013-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-013-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-013-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-013-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-013-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-013-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-013-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-013-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-013-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-013-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-013-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-013-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L04-013-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-013-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "green-room register",
"value": 5
} |
| inst-strata-L04-013-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-013-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-013-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | glm-5.3 | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-014-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-014-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-014-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-014-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-014-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-014-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-014-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-014-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-014-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-014-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": "5"
} |
| inst-strata-L04-014-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-014-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-014-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-014-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "churchwarden's book",
"value": 5
} |
| inst-strata-L04-014-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-014-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-014-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-015-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-015-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-015-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-015-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-015-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-015-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-015-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-015-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-015-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-015-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-015-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-015-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-015-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-015-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-015-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-015-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-015-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": "Tomas"
} |
| inst-strata-L04-015-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-015-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-015-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-016-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-016-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-016-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-016-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-016-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-016-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-016-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-016-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-016-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-016-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-016-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-016-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-016-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-016-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-016-observatory--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-016-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-016-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-016-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-016-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dome log",
"value": "Juno"
} |
| inst-strata-L04-016-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-016-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-016-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-017-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-017-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L04-017-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-017-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-017-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-017-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-017-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-017-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-017-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-017-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-017-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-017-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-017-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-017-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-017-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-017-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | deepseek-v4-pro | answered | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-017-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-017-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-017-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-017-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-017-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-017-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "aurora watch",
"value": 1
} |
| inst-strata-L04-017-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-017-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-017-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-018-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-018-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-018-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-018-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-018-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-018-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-018-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-018-hotel--formal-r0 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-018-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-018-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-018-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-018-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-018-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-018-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-018-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-018-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "housekeeping sheet",
"value": "Lyra"
} |
| inst-strata-L04-018-hotel--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L04-018-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-018-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-018-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-019-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-019-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-019-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-019-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-019-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-019-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | qwen3.8-27b | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-019-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-019-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-019-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-019-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-019-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-019-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L04-019-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L04-019-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L04-019-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "harbor manifest",
"value": "Sable"
} |
| inst-strata-L04-019-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L04-019-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L04-019-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| Instance | Provider | Status | Correct | Submitted action |
|---|---|---|---|---|
| inst-strata-L05-000-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-000-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-000-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-000-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-000-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-000-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L05-000-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-000-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-000-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-000-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-000-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-000-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-000-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-000-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-000-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-000-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-000-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-000-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r1 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-000-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-000-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-000-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": 1
} |
| inst-strata-L05-000-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-000-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-000-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-001-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-001-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": "2"
} |
| inst-strata-L05-001-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-001-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-001-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-001-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-001-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-001-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-001-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-001-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-001-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-001-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-001-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-001-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-001-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-001-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-001-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": "2"
} |
| inst-strata-L05-001-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-001-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-001-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-001-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-001-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-001-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "watch rotation",
"value": 2
} |
| inst-strata-L05-001-maritime--implicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L05-001-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-001-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-001-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-002-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "storm glass",
"value": "3"
} |
| inst-strata-L05-002-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-002-maritime--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gemini-3.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-002-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-002-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-002-maritime--explicit-r2 | glm-5.3-flash | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-002-maritime--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-002-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-002-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-002-maritime--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-002-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-002-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-002-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-opus-5.5 | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-fable-5 | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-002-maritime--implicit-r2 | grok-4.6 | answered | 0.0% | {
"operation": "hold",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-002-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-002-maritime--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "storm glass",
"value": 3
} |
| inst-strata-L05-002-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-003-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-003-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-003-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-003-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-003-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-003-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | minimax-m3 | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-003-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-003-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-003-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-003-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-003-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-003-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-003-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-003-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-003-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "dressmaker's bill",
"value": 2
} |
| inst-strata-L05-003-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-004-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-004-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-004-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L05-004-maritime--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | kimi-k2.6 | answered | 0.0% | {
"operation": "hold",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-004-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-004-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-004-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-004-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-004-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-004-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-004-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-haiku-4.5 | answered | 0.0% | {
"operation": "register",
"subject": "dock_pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-004-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-004-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | deepseek-v4-pro | answered | 0.0% | {
"operation": "hold",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "dock pass",
"value": "Halden"
} |
| inst-strata-L05-004-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-004-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-004-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-005-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-005-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | claude-opus-5.5 | content_filter | — | null |
| inst-strata-L05-005-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-005-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | kimi-k2.6 | transport_error | — | null |
| inst-strata-L05-005-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-005-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-005-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-005-theater--formal-r0 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--formal-r0 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-005-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-005-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-005-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-005-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-005-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | kimi-k3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-005-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-005-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-005-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-005-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-005-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-005-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "fly schedule",
"value": "Iris"
} |
| inst-strata-L05-005-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-006-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-006-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-006-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-006-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-006-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-006-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-006-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-006-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-006-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-006-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-006-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-006-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-006-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-006-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-006-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-006-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-006-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-006-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-006-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-006-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-006-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-006-hotel--implicit-r2 | glm-5.3-flash | answered | 0.0% | {
"operation": "register",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry chit",
"value": 3
} |
| inst-strata-L05-006-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-006-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-006-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-007-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-007-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-007-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-007-manor--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-007-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-007-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-007-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-007-manor--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-007-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-007-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-007-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-007-manor--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-007-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-007-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-007-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-007-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-007-manor--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "seed ledger",
"value": "Wren"
} |
| inst-strata-L05-007-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-008-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-008-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-008-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": "4"
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-008-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-008-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-008-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-008-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-008-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-008-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-008-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-008-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-008-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-008-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-008-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "per diem sheet",
"value": 4
} |
| inst-strata-L05-008-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-009-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-009-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-009-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-009-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-009-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-009-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-009-theater--formal-r0 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": "2"
} |
| inst-strata-L05-009-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-009-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-009-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-009-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-009-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": "2"
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-009-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-009-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-009-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-009-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | deepseek-4.1-flash | transport_error | — | null |
| inst-strata-L05-009-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-009-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "laundry bundle chit",
"value": 2
} |
| inst-strata-L05-009-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-009-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-009-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-010-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-010-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-010-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-010-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-010-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-010-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-010-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-010-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-010-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-010-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-010-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-010-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-010-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-010-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-010-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-010-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-010-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-010-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-010-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-010-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "advance booking sheet",
"value": "Cassia"
} |
| inst-strata-L05-010-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-010-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-010-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-011-hotel--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-011-hotel--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-011-hotel--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-011-hotel--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "valet_ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-011-hotel--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-011-hotel--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-011-hotel--formal-r0 | gpt-6-sol | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-011-hotel--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-011-hotel--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-011-hotel--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-011-hotel--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-011-hotel--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-011-hotel--implicit-r1 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-011-hotel--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-011-hotel--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-011-hotel--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-011-hotel--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-011-hotel--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "valet ticket",
"value": "Silas"
} |
| inst-strata-L05-011-hotel--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-011-hotel--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-011-hotel--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-012-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-012-observatory--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-012-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-012-observatory--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-012-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-012-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-012-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-012-observatory--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-012-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-012-observatory--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-012-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-012-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-012-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-012-observatory--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "generator book",
"value": "Galia"
} |
| inst-strata-L05-012-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-013-observatory--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-013-observatory--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-013-observatory--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-013-observatory--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | glm-4.7-flash | answered | 0.0% | {
"operation": "register",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-013-observatory--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-013-observatory--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-013-observatory--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-013-observatory--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-013-observatory--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L05-013-observatory--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-013-observatory--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-013-observatory--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-013-observatory--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-013-observatory--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "snow-cat log",
"value": "Petra"
} |
| inst-strata-L05-013-observatory--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-013-observatory--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-013-observatory--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-014-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-014-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-014-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-014-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-014-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-014-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-014-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-014-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-014-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-014-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-014-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-014-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-014-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-014-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-014-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-014-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-014-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-014-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-014-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-014-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "register",
"subject": "stage manager's log",
"value": 3
} |
| inst-strata-L05-014-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-015-maritime--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L05-015-maritime--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-015-maritime--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-015-maritime--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | claude-haiku-4.5 | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-015-maritime--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-015-maritime--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-015-maritime--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-015-maritime--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-015-maritime--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-015-maritime--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-015-maritime--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-015-maritime--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | claude-opus-5.5 | answered | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-015-maritime--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "customs seal",
"value": 3
} |
| inst-strata-L05-015-maritime--implicit-r2 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-015-maritime--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-015-maritime--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-016-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-016-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-016-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-016-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r1 | glm-4.7-flash | answered | 0.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": "1"
} |
| inst-strata-L05-016-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-016-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-016-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-016-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gpt-6-luna | answered | 0.0% | {
"operation": "register",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | mimo-v2.6-flash | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-016-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--explicit-r2 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L05-016-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-016-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-016-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gpt-6-astra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-sonnet-5 | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-016-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-016-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-016-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-016-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-016-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-016-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-016-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | minimax-m3 | answered | 0.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": "1"
} |
| inst-strata-L05-016-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-016-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-016-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-016-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-016-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-016-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-016-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "cheese room slate",
"value": 1
} |
| inst-strata-L05-016-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-016-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-016-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-017-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-017-theater--explicit-r1 | glm-5.3-flash | answered | 0.0% | {
"operation": "register",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-017-theater--explicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-017-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-017-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-017-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-017-theater--explicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-017-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-017-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-017-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-017-theater--formal-r0 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-017-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | deepseek-v4-pro | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-017-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-opus-5 | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-017-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-017-theater--implicit-r1 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-017-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-017-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | glm-5.3 | transport_error | — | null |
| inst-strata-L05-017-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-017-theater--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-017-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-017-theater--implicit-r2 | baseline:direct-facts | answered | 100.0% | {
"operation": "hold",
"subject": "armory check-out",
"value": 1
} |
| inst-strata-L05-017-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-018-manor--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-018-manor--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | mimo-v2.6-pro | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-018-manor--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r1 | glm-4.7-flash | transport_error | — | null |
| inst-strata-L05-018-manor--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-018-manor--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-018-manor--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-018-manor--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-018-manor--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-018-manor--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-018-manor--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-018-manor--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-018-manor--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-018-manor--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | glm-4.7-flash | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--formal-r0 | claude-fable-5.1 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-018-manor--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-018-manor--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-018-manor--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-018-manor--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-018-manor--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-018-manor--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-sonnet-5 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-018-manor--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "register",
"subject": "game book",
"value": "Vela"
} |
| inst-strata-L05-018-manor--implicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-018-manor--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-018-manor--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-019-theater--explicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-019-theater--explicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-019-theater--explicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-019-theater--explicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-019-theater--explicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-019-theater--explicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-019-theater--explicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-019-theater--explicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-019-theater--explicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--explicit-r2 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-019-theater--explicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-019-theater--explicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-019-theater--formal-r0 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-019-theater--formal-r0 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | gpt-5.6-luna | answered | 0.0% | {
"operation": "register",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | mimo-v2.6-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-opus-5 | content_filter | — | null |
| inst-strata-L05-019-theater--formal-r0 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-019-theater--formal-r0 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--formal-r0 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-019-theater--formal-r0 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-019-theater--formal-r0 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-019-theater--implicit-r1 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-019-theater--implicit-r1 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | glm-5.3 | answered | 0.0% | {
"operation": "register",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-fable-5 | content_filter | — | null |
| inst-strata-L05-019-theater--implicit-r1 | mimo-v2.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-019-theater--implicit-r1 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r1 | claude-fable-5.1 | content_filter | — | null |
| inst-strata-L05-019-theater--implicit-r1 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-019-theater--implicit-r1 | baseline:story-copy | invalid_response | 0.0% | null |
| inst-strata-L05-019-theater--implicit-r2 | baseline:constraint-solver | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | baseline:forward-chain | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gpt-6-astra | transport_error | — | null |
| inst-strata-L05-019-theater--implicit-r2 | gpt-6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gemini-3.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gpt-5.6-sol | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gemini-3.8-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | glm-5.3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gpt-5.6-terra | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | deepseek-4.1-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gemini-3.6-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | deepseek-v4-pro | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gpt-6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-sonnet-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | gpt-5.6-luna | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-opus-5.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-fable-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | mimo-v2.6-flash | answered | 0.0% | {
"operation": "register",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | mimo-v2.6-pro | answered | 0.0% | {
"operation": "register",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-opus-5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | grok-4.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | kimi-k2.6 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | minimax-m3 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | qwen3.8-27b | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | kimi-k3 | transport_error | — | null |
| inst-strata-L05-019-theater--implicit-r2 | glm-5.3-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-haiku-4.5 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | glm-4.7-flash | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | claude-fable-5.1 | answered | 100.0% | {
"operation": "hold",
"subject": "lighting plot",
"value": "Ansel"
} |
| inst-strata-L05-019-theater--implicit-r2 | baseline:direct-facts | answered | 0.0% | null |
| inst-strata-L05-019-theater--implicit-r2 | baseline:story-copy | invalid_response | 0.0% | null |
Appendix: run provenance and generation completeness. Also available in results.json.
Generation completeness
{
"available": 120,
"classification": "v3",
"complete": true,
"conditions": [
"formal",
"explicit",
"implicit"
],
"expected": 120,
"expected_variants": 600,
"realizations": [
1,
2
]
}Provenance
{
"benchmark_version": "3",
"grader_version": "3.1",
"sources": {
"__init__.py": "0a0b0ed757aacfdd844c7ea335c2a134019b6d309245d2cfc912b1abac872060",
"compile.py": "d2e5c36a2baf4c6173d399a4adbf74a42d0ccb27a3a20a13b280ce757ea205d0",
"decisions.py": "52b6b7298cb7fd7265635a435530b7b0250591aed5ecaedc411f05425a54cf9a",
"evaluate.py": "f425359a63343d6c7ecf4218f17c962de5410e995bb915d2accf74880fc5a677",
"experiments.py": "ce06e18d24e27eb81fd6328f1228544605c3081a765046ab0e5553dd25baac5a",
"generator.py": "37877a6b21815a37538c68167272ee16fe9c9e0da4408f768e926d05b8bb62ee",
"genres.py": "2a0ab884239cadc8d4175ac978854c88732e96e4b77caa8e0ed36e6ae3e6b151",
"grading.py": "4f7336bc23085da839b18f24af79431e79e6d9a36c4551a416e72fbc90b368f1",
"harness.py": "04c2f73bc3a143c49e9cbece56f2a4df451483e783d6aef4cc58d9faac312b77",
"interactive.py": "030faebbbec71f4ae6eedb650278c60da432679d978a14662c6aa3c1eec78293",
"judge.py": "9be37e0f06c7caa88d3171dd9f5bc71575c16d9441059f0b6c0d2a50a924268d",
"llm.py": "508a5a37ac6d9071a522aa659334325f966b9552af4a098abde8cf8c12f2aad4",
"narrative.py": "66455bc38d907edf2dfcc5668f96e5628ecfaa0e9a7c2fefcaab3bb805d93569",
"oracle.py": "ab5878f8ed180bccc9653a0b2fc601220d0d81cbaa24a91622d764aacfc37c55",
"pipeline.py": "e05a0889f24003a90ef4b6c4a9ce60bd50759939e839a9bc4d03377e3dc572ca",
"populate.py": "2dab5370fd9895de2f25c92cd851dff84a642bb125d6e07666241661136577a3",
"reporting.py": "6cf96e7d1c2ddeae233062be20facad57e41ed7cf8ebba78b5479403917fc013",
"rng.py": "3de3ceaced32995ab61a317be204656229ad67b10a05f21736cac082b1941917",
"sat.py": "bfbb94117dca5e649f609b388dc741d5082be17cb278c581a7da78cd3b81797d",
"story.py": "391f2f97fd142cbd0cbcdf12fbdef194ba4377896fd315ee19ec1bddf7d1f965",
"verify.py": "998c048a3185bd959bc34a892217314e465ec30c3efc7db94191fcff90f8e321",
"world.py": "963011d7ec5249030aed8c33ff0faa3b9d086bdfab024095ec0852202d59628e"
}
}