{ "_comment": "The grid and the CPM network were solved by brute force, not by re-reading the derivation. Both confirmed unique.", "test1.txt": { "note": "Commonsense mechanisms. Only the four items with an unambiguous physical answer are checked: water expands on freezing so that jar cracks (honey does not); the 3 a.m. chirp is a low battery, whose voltage sags as the house cools overnight; resting a roast lets juices redistribute; metal feels hotter than wood at equal temperature because it conducts heat away from the skin faster.", "must_contain_any": ["expand", "expansion", "less dense", "more space", "greater volume"], "regex": "(?i)(batter(y|ies))", "_reasoning": "Items 2 and 3 - the coffee mug and the indirect refusal - are deliberately unchecked. Both are correct in too many phrasings to pattern-match without risking a false failure. The remaining two mechanisms are covered by test1_extra below, kept separate so one missing term does not mask another." }, "_test1_extra_note": "Held as documentation rather than a matcher: 'juice'/'redistribute' for the roast and 'conduct' for the spoons were considered, but folding four independent facts into one score made a partial answer indistinguishable from a wrong one. Split them only if per-item scoring is ever added.", "test2.txt": { "note": "Causal structures. The discriminating content is the lurking variable behind finding A - ice cream and drowning are both driven by hot weather - because that word appears nowhere in the prompt. The structure names do appear in the prompt as a list of options, so containing them proves little; they are checked anyway because a correct answer certainly contains them and the check cannot fail correct work.", "must_contain_any": ["temperature", "summer", "hot weather", "warm weather", "heat"], "regex": "(?i)regression to the mean", "_reasoning": "Finding F is regression to the mean and finding E is a selection effect, but both terms are supplied by the prompt, so a model could echo them while assigning them to the wrong findings. The weather confounder is the one answer the prompt does not hand over." }, "test3.txt": { "note": "Unique solution: Alvarez/South/comets, Brandt/North/dialects, Chen/East/basalt, Dube/West/algae. Clue 5 is redundant - removing it still leaves exactly one solution; removing any other clue does not.", "must_contain": ["Alvarez", "Brandt", "Chen", "Dube"], "must_contain_any": ["South", "north"], "regex": "(?i)clue\\s*5|fifth clue", "_reasoning": "Part (d) asks which clue is redundant. Verified by re-solving with each clue dropped in turn." }, "test5.txt": { "note": "Critical path A-D-E-F-H, project finish 32 days. Slack: B=12, G=11 (and C=5). D slipping 3 days pushes the finish to 35; G slipping 3 days changes nothing.", "numbers": [32, 12, 11], "must_contain_any": ["A → D → E → F → H", "A->D->E->F->H", "A-D-E-F-H", "A, D, E, F, H"], "_reasoning": "Path notation varies, so several renderings are accepted." }, "test6.txt": { "note": "Cognitive-reflection answers: ball is 5 cents (not 10), 100 machines take 5 minutes (not 100), lake is half covered on day 47 (not 24).", "numbers": [5, 47], "_reasoning": "The intuitive-but-wrong answers are 10c, 100 minutes and 24 days. Only the correct values are checked: Stage 2 explicitly asks the model to state the wrong answer it was meant to elicit, so banning '24 days' would fail a correct answer. The 5-minute answer collides numerically with the 5-cent one and is not checked separately." } }