6 of DeepSWE's 30 hardest tasks reject a correct solution.

[bKNqFKp] task.toml declares `language = "typescript"`, but the change and all tests are Go (cmd/prometheus/*.go, internal/txnreload/*.go, run with `go test`). [task].description is also empty.

[bKNqFKp] - The opt-in gating is never checked for reload behaviour. No test reloads without --enable-feature=transactional-reload-config and confirms there is no rollback and the old behaviour remains; only the /api/v1/features flag is checked.
- After a failed apply, nothing checks failed_reloader, the contents of applied_reloaders, rollback_successful, error_message or the contents of reloader_timings_ms. They are only checked for type or emptiness, or only in the before-first-reload case, so wrong values pass.
- rollback_error is never forced. Tests accept either apply_error or rollback_error, so the categories are never told apart.
- 'executes reloaders in sequence' (stop at the first failure) is not observable in any test.

language=typescript; grade.tool_label=go-ctrf-json-reporter

[vrxu2TP] - "Diagrams annotate state data variables" is not tested at all. No f2p test touches the diagram renderers.
- "Re-entering a state resets data to the original defaults" is only tested without history. The reference's _init_state_data restores a saved history snapshot whenever one exists, even on a normal (non-history) re-entry, and no test notices.
- "DataVar and DataChangeInfo are importable from the statemachine package": DataChangeInfo is never imported, and the state_id attribute of change records is never asserted.
- "Data is stored per instance, not on the shared State class" is never checked with two instances of the same machine.
- set_state_data type validation is only checked for a non-type value, not for a None value or a tuple of types.

[rVYx4Rf] - "Button controls ... removed from a shared toolbar container ... avoid stale listeners" is tested only with the custom clean button. Built-in format buttons added later are never exercised, and neither is dynamic select handling.
- "Shared toolbar actions must do nothing until a remaining live editor becomes active" is tested only with the bold button. Pickers and the image/link handlers are never clicked after the active editor is removed.
- "Removing the active editor must not leave ... stale theme-managed UI" is never checked directly, e.g. whether the removed editor's tooltip still reacts to toolbar actions.
- The read-only case is tested via disable() only. The readOnly config option and enable() toggling while focus stays put are not exercised.

[5kEETPM] - The core end-to-end behaviour, a real run stopping early when a test fails, is never checked. The test 'failure reported to Reporter triggers server broadcast and runner abort' wires things up itself ('// Wire the reporter to app the same way app.start does' / reporter.on('test-failure', ... app.abortRunners())). A solution that never connects the Reporter's test-failure event to app.abortRunners inside App would still pass.
- Validation is only partly checked: zero, negative and float values are only shown not to bail; only the string 'always' is checked for an npmlog warning, so a solution that warns only for strings would pass.
- 'Runner abort ... suppresses all subsequent results' is not checked for TapProcessTestRunner/ProcessTestRunner test results (only exit errors are), and TapProcessTestRunner's suppression of exit errors is only lightly exercised.
- Nothing checks that the app calls resetBailState at the start of each run.

[cTa9qn6] - TestTrackerNegativeSSLDaysDoesNotTriggerExpiry is in test.patch and is run by `-run 'TestTracker'`, but it is not in config.json f2p_node_ids. So "negative SSLDaysRemaining ... never triggers SSL expiry" is never graded.
- "Measure from the last non-suppressed non-recovery event": the reference updates LastEventAt after every non-suppressed event, recovery and healthy events included (`if decision.Event != EventNone && !decision.Suppressed { ... LastEventAt = now }`). That contradicts the instruction, and no test sends a non-recovery event after a recovery while the cooldown window is open.
- Nothing checks that the tracker is actually wired into the simple or TUI monitoring loops, or that desktop notifications use decisions. The output tests pass a pre-filled AlertDecision to PrintResult, so a solution that never calls Evaluate during monitoring still passes.
- Snapshot fields on EventNone decisions ("should match tracker state even when Event == EventNone") are only partly checked; for example, ConsecutiveFailures/Recoveries on non-event checks are never asserted.

[6G44Wh4] - 'Whitelist file changes invalidate affected modules' is only tested with an explicit whitelist passed as a scan path (test_cache_with_explicit_whitelist_changes), which behaves like any other changed module. Invalidation when a bundled vulture/whitelists/*_whitelist.py file changes is never tested.
- 'KeyboardInterrupt ... saves the partial cache safely and then re-raises' is tested only for the module-scanning phase. No test interrupts during whitelist scanning or during the save itself.

[y7bkFj6] - CSS-wide keyword expansion is only tested for margin. A solution that handles keywords only for box-model shorthands would still pass.
- "Returns null ... when the value does not match the property's syntax" is only tested with margin: 'not-a-valid-value'. Invalid input for background, font, border-top etc. is never checked.
- Compression returning null for an incomplete longhand set is only tested for margin.
- "Must work with custom syntax created via fork()" is only checked by forking an unrelated custom-prop and expanding margin. A fork that redefines a shorthand's syntax is never tested.
- Some listed behaviours have no test: padding/inset 1- and 3-value cases, flex partial values, background-origin/clip with two box values, and font without line-height.

[NfYjRrZ] - The instruction says 'Handle non-incremental responses gracefully', but every HTTP test serves 'multipart/mixed; boundary=graphql', so a plain application/json response is never tested.
- TestUnsupportedTransport.test_execute_incremental_on_unsupported_transport_raises is in test.patch but is not an f2p id, so it never affects the score.
- The WebSocket test checks only merged data. It does not check per-payload .extensions or has_next over WebSocket. The reference actually injects hasNext and incremental into ExecutionResult.extensions (websockets_protocol.py), and nothing notices.
- The stated 'boundary=graphql, deferSpec=20220824' negotiation (the Accept header) is never checked, because the fixture server ignores request headers.

[kxQ6uJ7] - Requirement 6, '%' rootMargin units: only the normalised string ('10px 20% 10px 20%') is checked. No test checks that percentage margins actually change the intersection geometry.
- Requirement 10, the 'zero-area targets ... otherwise 0' half: only the contained case is tested.
- Requirement 12, 'disconnect() must ... clear pending records': the test calls takeRecords() only after the records were already delivered, so pending records are never actually exercised.
- Requirement 9 with multiple thresholds (number arrays during crossing detection) is not tested.
- unobserve() is tested only with a single target.

[97w69AK] - Install CLI overrides are not checked for behaviour. TestHarness_Install_WithCLIMergeStrategy only asserts `res.Config["items"]` == ["user1"], which holds whether or not `Install.MergeStrategies` has any effect, so it only needs the field to exist.
- Global strategies are effectively untested. TestHarness_CoalesceValues_GlobalAppendStrategy passes through ordinary per-chart application of the `global.tags` path in the child's coalesceValues. The required behaviour ("The `global.` prefix is stripped before applying the strategy to the globals map") during coalesceGlobals is never exercised.
- "This applies to both stable and internal chart formats" for the accessor: only the v2 accessor's Annotations() is tested. The v3 accessor is not.
- The `MergeKeys` CLI override taking precedence over a chart's merge-key annotation for the same path is never tested. In TestHarness_Upgrade_CLIMergeStrategyOverridesAnnotation the chart has no merge-key annotation, so CLI precedence is only shown for the strategy itself.
- The empty-path lint warning and the dotted merge-key behaviour in lint are not exercised.

[Z2HFi4u] - The instruction says to 'Update the `display` style property to accept "grid"' (a TypeScript type change in styles.ts), but nothing enforces it: test.sh runs `npx tsc --noEmit 2>/dev/null || true` (advisory only), and AVA runs the .tsx tests without type-checking. A solution that never updates the Styles type still passes.
- Implicit rows created past an explicit gridTemplateRows are never tested, and neither is a grid container with padding or border.

[VPXTj32] - The core goal is untested: dumps must actually be encrypted on upload. The instruction asks to 'Integrate encryption into the handler's storageReadWriteCloser pipeline', and the reference also passes `job.Encrypted()` to EnsureFileName in save(). The only handler-level test, TestHandlerEncryptionKeyMissing, sets `job.Storage.Local = nil` and checks only the key-load error. TestPipelineRoundTrip calls storageReadWriteCloser directly. A solution that loads the key in save() but still calls storageReadWriteCloser with nil, or never adds .enc to uploaded filenames, passes every test.
- 'Export Job's validate method ... it must run encryption config validation' is checked only by the Job.Validate() call in TestJobValidationWithEncryption. Whether Dump.Validate still applies job validation is not exercised by any f2p test.
- 'Two encryptions ... must differ' is tested, but 'each chunk uses unique nonce' is only checked within a single stream (fine), and the TrimSpace behaviour for file keys is checked only with a trailing newline.

[mtXKV5q] - RestoreOnErr requirement is effectively untested. The instruction says 'A RestoreOnErr-wrapped alternative qualifies when its inner expression qualifies; its wrapper is stripped from the coalesced result'. The restorer only wraps alternatives that change the stack (Push), so no test ever builds a RestoreOnErr around a single-character alternative. The tests named after this rule (restorer_wrapped_alternatives_stripped_when_coalesced, restore_stripped_both_sides_of_push, restore_on_err_stripped_from_qualifying_alternatives) pass with or without the unwrapping logic.
- The 'Case-insensitive alphabetic characters expand' rule is tested only with ASCII letters, so how non-ASCII Insens characters expand is never checked.
- No f2p test checks that generated parsers (pest_generator) or the VM actually match CharClass/NegCharClass correctly. Only compiling, plus incidental p2p grammar tests, exercises that code.

[WiGR8RL] - OPTIONS(...) handling (requirement 6) has no f2p test. test_options_clause_at_depth_0 and test_options_with_partition_by_on_separate_lines are p2p and feed already-formatted input, which the unmodified base already leaves unchanged; no unformatted OPTIONS input is formatted anywhere.
- The line-length constraint is checked only by p2p tests whose input is already within the limit (test_normal_columns_within_line_length, test_long_column_definition_not_truncated). No test checks that an over-long non-column line actually gets split, or what happens to an over-long table constraint.
- 'CREATE TABLE AS SELECT and CREATE TABLE ... LIKE ... must pass through unchanged' is covered only by p2p tests that pass trivially at base. That is acceptable as a regression guard but proves nothing about the new code.
- has_inline_constraint for REFERENCES, CONSTRAINT and CHECK, and type_name termination at those keywords, are not tested through parse_ddl_table; only NOT NULL and DEFAULT are.

[Ss44wqk] - "`deep` keeps causes recursively up to `maxCauseDepth`": no test truncates at an explicit positive value. Tests only use maxCauseDepth 0, 5/10 with chains shorter than the limit, and the default 16. An implementation that ignores any explicit value other than 0 and always uses 16 would pass.
- The hook payload "plus any of `stack`, `stackFrames`, `cause`, `errors`": no test checks that a registered processor receives `errors` for an AggregateError or `stackFrames` in frames mode.
- `redactPaths=strip_cwd` is tested only in string mode, not frames mode.

[qxsykzf] - The instruction says auto-end is 'based on leading whitespace of the line, not the column position of the directive itself'. No test puts a mid-line begin directive on an indented line (test_079 is at column 0), so an implementation that uses the comment's column is never caught.
- Skipping semicolon-only lines for nosec-next-line is effectively untested. The only test, test_078, uses "\n;subprocess.Popen(...)", which is a Python SyntaxError, so Bandit reports nothing and the test always passes. It is also listed as p2p, not f2p.
- 'Extra text after nosec-end is ignored' (test_039, '# nosec-end B602') expects no findings, but the region already suppresses the only finding, so any handling of the trailing text passes.
- The parse-error fallback is only checked in the direction that more suppression passes; an implementation that turns parse errors into a blanket suppression would also pass test_116.

[dUi8dX4] - HTML spanning layout and the alignment override in HTML are never checked. The only HTML assertions are relative `vertical-separator` counts and that '(' or '[' appear. A solution that renders the \multicolumn cell as an ordinary single-column cell in HTML, or ignores its alignment there, would still pass, even though the instruction says the span 'overrides the column's declared alignment'.
- The instruction allows 'optional | for vertical rules' in the alignment argument, but no test checks that these rules are drawn. Tests with `{|c|}` only check MathML columnspan. The reference parses hasLeftRule/hasRightRule and never renders them, a sign nobody checked this.
- Validation of 'exceeds remaining columns' is tested only for `array` with an explicit column spec, never for matrix, cases or aligned.
- Suppressing separators in only some rows ('per-row basis') is checked only by comparing total separator counts, which says little about rendering correctness.

[NGavBeA] - Cycle detection ('Cycle detection prevents a message visiting the same queue twice') is effectively unchecked. test_cycle_detection asserts only `count <= 1`, and a single dead_letter call re-puts the message at most once even with no cycle detection, so an implementation without it still passes.
- 'Missing DLX exchange: silently dropped' and 'No DLX: silently discarded' are checked only by the absence of an exception (test_dlx_exchange_not_exist_silently_drops, test_no_dlx_silently_discards). Wrongly routing the message elsewhere would go unnoticed.
- 'QoS.reject(..., requeue=True) restores normally' (test_reject_requeue_does_not_dead_letter) is not in the f2p list.
- Per-message expiration precedence is checked only by `remaining < 2.0`.

[neYWNKD] - The instruction lists 'non-deferred mutation on an entity with pending commands' as an execution trigger, but no test can tell whether it happens. The only test for it, 'should auto-flush when non-deferred operation is attempted on entity with pending commands', checks just `entity.has(Velocity)` and `entity.has(Health)`. Read-through projection already makes those true without any flush, so an implementation that never auto-flushes still passes.
- 'should allow explicit flush of deferred commands' has the same weakness: it only checks has() before and after flush(), which projection satisfies either way. Other tests do cover flush() indirectly through events and query lengths.

[imDM8tn] - 'Query result iteration resolves per-target relation data for pair-tracked traits' is only checked through readEach. No test checks that updateEach gives per-target data or writes mutations back to the specific pair, even though the reference adds commitPairData for this; an agent could skip updateEach entirely.
- The test 'should not fire pair-level Changed when updateEach modifies a non-pair-tracked trait' calls updateEach on `Changed(Likes(alice)), Position` right after that tracking state was cleared. The result set is empty, the callback never runs, and the test passes vacuously.
- 'Pair modifiers compose with `Or`' is only tested with Added. Removed/Changed pair modifiers inside Or are not tested, and neither is wildcard Removed for a non-last pair.

[PgDkwGW] - 'Dependency changes during updateEach defer re-evaluation until iteration ends' is only weakly checked. The test named 'should defer predicate re-evaluation during updateEach until iteration ends' asserts only `world.query(HighHealth).length` after the updateEach loop has finished. Immediate re-evaluation would produce the same final state, so this test cannot tell the two apart. The only other test of this requirement is 'should not re-evaluate predicates mid-updateEach iteration' (visited.length === 2), and it detects missing deferral only if updateEach iterates a live entity array rather than a copy. I could not confirm which from the task files.
- No test covers `add` of a trait the entity already has, or updateEach with changeDetection 'never'.

[vMbdxGK] - The instruction says 'Protection should apply only where the full selector relationship is implicated, not merely where one piece of that selector appears nearby.' No test has a group that matches one compound of a structural selector while the full relationship does not hold (e.g. `g.keep > rect` where g.keep contains no rect). The reference does not implement this either: selector_branch_implicates_element returns true if ANY compound matches the element, so 'one piece nearby' is exactly what it protects.
- The instruction says implication 'must be determined from the structure and selector anchors that exist before the rewrite'. No test checks this: there is no case where an earlier flatten erases the evidence for a later decision.
- Only the descendant, child and adjacent-sibling combinators are tested. General sibling (~), structural pseudo-classes (:first-child, :nth-child, :empty), :is()/:where(), and rules nested in @media are never exercised, so a partial selector analysis would still pass.

[usqVSgN] - "Negation nodes produce no conflicts" is not really tested. The only negation test, TestAnalyzeNegationDoesNotCauseConflict, uses `!"reserved" @Ident`, which has no disjunction and no optional group, so it reports zero conflicts even if negation subtrees are analysed like any other node. A grammar such as `!(@Ident | @Ident)` would be needed to catch this.
- The dedup tests only use two-alternative grammars. The reference itself emits pairwise first/first conflicts that all share the same Location and whole-disjunction GrammarSnippet. For three or more overlapping alternatives, its own Analyze output therefore contains entries with the same dedup key, which Dedup/Merge would collapse. No test checks that Analyze output is consistent with the dedup key on larger disjunctions.
- "Independent of SuppressConflictType" for StrictMode is checked only by building with StrictMode alone (TestAnalyzeStrictModeWithSuppressStillFails passes no suppress option).

[jHUUtyD] - The instruction says CSI/OSC sequences 'have zero visible width', but nothing tests non-SGR CSI (e.g. ESC[2J) or non-OSC8 OSC sequences. The reference parser.go classifies them as TokenText with Text=raw, so StripANSI keeps them and ANSIWidth counts their printable bytes, and no test notices.
- 'Under Ascii ... Output.Truncate returns text with tail; no ANSI emitted' is only tested with plain input ("abcdef"). The reference Output.Truncate never checks the profile and would pass ANSI input through unchanged, and no test notices.
- 'Output.TemplateFuncs() propagates the default to all template helpers' is only checked for Bold. Color, Foreground, Background, Truncate and truncate are not checked under preserve-resets.
- The Ascii template path is untested: the reference's noopTemplateFuncs does not define Truncate/truncate.