schemaVersion: 1 analysis: parentHarness: H052 generation: H052 analysisRunId: h052-postformal-20260814-019ff5c4-a stableNamespace: H052/h052-postformal-20260814-019ff5c4-a source: user-confirmed-retrospective-weaknesses sourceWeaknessFile: weakness-shortlist.yaml implementedSuccessor: H0601 status: proposalCardsRecorded: 2 confirmedWeaknesses: - W-01 - W-02 historicalUserProposalSelection: - P-01 - P-02 historicalImplementationStatus: complete historicalEvaluationStatus: complete newImplementationAuthorized: false newExperimentAuthorized: false cards: - proposal_id: P-01 stable_ref: H052/h052-postformal-20260814-019ff5c4-a/P-01 parent_harness: H052 title: Semantic Closure as the Change Spec Handoff Predicate source_weakness: W-01 evidence: summary: >- H052 could hand a fresh Implement Session a Change Spec whose public API identity, adjacent preservation, precedence, docs/changelog obligations, or internal consistency was not actually closed. D8 fixed r3 is the deterministic example: the Operator explicitly returned CONTINUE for an internal conflict, but turn exhaustion remained nonblocking and the same digest became confirmedSpec. D5 bln-r1 and D7 aln-r3 show that a true CONFIRM could also miss result-changing closure. affected_tasks: - bat-sanitize - axum-custom-executor - prometheus-utf8-negotiation-evaluator-fixed-v1 affected_runs: - bln-r1 - aln-r3 - h052-d8-fixed-r3 observed_impact: quality: >- Implement followed or resolved an incomplete/incorrect product contract; affected Judge pairs were 85/86, 84/81, and 71/76 respectively, with D7's score explicitly limited by evaluator-overconstraint provenance. completion: >- H052 workflow completion remained nonblocking by design, so the defect appeared as false semantic handoff rather than workflow failure. cost: >- Product decisions were rediscovered or guessed during implementation and evaluation; no total-token reduction is inferred. stability: >- The mechanism appeared through both true-but-insufficient CONFIRM and deterministic CONTINUE-to-EXHAUSTED handoff paths across three tasks. root_cause: hypothesis: >- H052 treated artifact/digest availability and nominal closure fields as sufficient for handoff. It did not enforce exact public identity and result-changing semantic decisions, did not make structural findings pre-accept blockers, and converted an exhausted CONTINUE into a confirmed operational contract. confidence: high strategy_change: current_strategy: >- One exact-digest Change Spec is authoritative, but public identity can be described generically, adjacent preservation can be compressed into broad prose, docs/changelog decisions can be omitted, and an unconfirmed exhausted artifact can still enter fresh Implement. proposed_strategy: >- Preserve H052's two-stage topology and single Change Spec authority, but make semantic closure the handoff predicate. Every affected public surface must state an exact name/signature or a genuine repository-supported not-product-contract reason. Adjacent preserved behavior and all applicable default/order/precedence/conflict/failure/fallback/docs/changelog decisions must be explicit and internally consistent. Operator confirmation is bound to the exact digest and legal product authority. Structural findings, CONTINUE, and EXHAUSTED retain the artifact for revision but cannot create a confirmed binding or start Implement. rationale: >- The single lever is eligibility for Spec-to-Implement handoff. It closes product decisions where they are owned without reintroducing Proposal, Design, Tasks, Validation, or Review as competing authorities and without asking Controller or Implement to invent semantics. expected_effect: behavioral_change: >- A fresh Implement Session starts only from an exact-digest Operator CONFIRM whose public identity, preservation, precedence/default/fallback, and docs/changelog decisions are closed. Unresolved or contradictory artifacts remain in the Spec Session even when the turn budget is exhausted. targeted_pattern_change: >- D5-like wrong precedence/changelog omissions, D7-like unnamed public identities, and D8-r3-like CONTINUE/EXHAUSTED conflicts should no longer first surface after confirmed handoff. The change must not merely move the same ambiguity into Implement prose or hidden evaluator feedback. expected_metrics: quality: >- Fewer Judge/evaluator mismatches whose First Divergence is a known Spec conflict or unresolved public identity; no universal product-green or score uplift is assumed. completion: >- False semantic handoffs should fall. Raw workflow completion may stay unchanged or decrease when genuinely unresolved decisions remain blocked; no completion uplift is promised. cost: >- Spec/Operator work may increase before handoff, while downstream guessing may decrease; no net token, tool-call, or latency reduction is assumed. stability: >- A given exact digest has one unambiguous state: confirmed and handoff- eligible, or unconfirmed and revision-eligible, never both. rejection_conditions: - >- Structural findings or an Operator CONTINUE/EXHAUSTED can still create a confirmed binding or start Implement. - >- Generic placeholders such as "submission method" satisfy public identity, or not-product-contract can be asserted without a real reason. - >- D5/D7/D8-type result-changing ambiguity is merely moved into Implement, another marker, or a second operational contract. - >- Hidden evaluator cases, reference-only symbols, or Judge feedback are copied into the Spec or used as Implement repair input. - >- Controller begins inventing product behavior, or the strategy restores Proposal/Design/Tasks/Review as confirmed authorities. - >- Internal files, helpers, patch layout, or development order are falsely promoted into mandatory product decisions. - >- A quality or completion regression is attributable to the new handoff predicate, especially permanent blocking on a non-product internal choice. regression_risks: - >- A vague public-versus-internal boundary could overconstrain implementation details or leave a genuine API identity falsely open. - >- More Spec revision may increase Operator cost without improving downstream behavior if semantic criteria are checked mechanically rather than truly. - >- An overly broad product-authority channel could leak hidden evaluator implementation symbols into the visible contract. - proposal_id: P-02 stable_ref: H052/h052-postformal-20260814-019ff5c4-a/P-02 parent_harness: H052 title: REQ-Bound Candidate-Visible Independent Verification Receipts source_weakness: W-02 evidence: summary: >- D4 paln-r3 confirmed static-typing and documentation obligations but the Controller ran only a static task-manifest pytest suite. Implement saw a relevant mypy failure, then hid it behind later successful commands and an explicit || true pipeline. The unrelated Controller pytest receipt was green. H052 therefore proved receipt currentness but not coverage of the exact confirmed obligations. affected_tasks: - pytest-addini-type-expressions affected_runs: - paln-r3 observed_impact: quality: >- A known static-typing defect coexisted with a green Controller receipt and Judge 96/92 because the evidence authority never represented the typing obligation independently. completion: >- The workflow completed as designed. The weakness is evidence truthfulness, not a claim that product-red should block terminalization. cost: >- Implement spent work on ad hoc verification whose status could not be reused or trusted by Controller; no aggregate cost estimate is inferred. stability: >- Any Spec obligation outside the static task command set could reproduce the disconnect; exact prevalence is not inferred from one direct cell. root_cause: hypothesis: >- Product obligations, Controller command selection, and Implement evidence had separate authorities. Without a Spec-bound declaration and one honest process result per obligation, aggregate shell status and static focused suites could not establish evidence coverage. confidence: high strategy_change: current_strategy: >- The Change Spec states observable behavior, the task manifest independently supplies Controller focused tests, and Implement verification is free-form shell activity whose earlier failures can be overwritten by later commands. proposed_strategy: >- Keep the same single Change Spec authority and two-stage workflow. Each REQ declares at least one Candidate-visible structured verification with category, argv, env, cwd, and timeout. Operator confirms on the exact digest that the declared command genuinely tests the stated observable acceptance. Controller executes every declaration independently without a shell command string, records real process-start/exit/timeout and separately identified stdout/stderr, and binds the ordered results to the exact Spec digest and current product projection. Hidden evaluator hooks remain separate terminal measurement and never enter visible receipts or repair input. Honest visible red records product failure but does not block workflow completion. rationale: >- The single lever is evidence coverage and identity for confirmed obligations. It prevents later command success or || true from rewriting an earlier failure while preserving the user's separation between workflow completion, visible verification, and blind external evaluation. expected_effect: behavioral_change: >- Each confirmed REQ has ordered, Candidate-visible, independently executed results with honest failure identity. A typing failure remains nonzero even if later pytest succeeds, and missing, stale, timed-out, or unstarted evidence cannot be represented as green. targeted_pattern_change: >- D4-r3-like aggregate-shell false green should disappear. The strategy is rejected if the same disconnect survives as an opaque combined command, a task-level suite with no REQ mapping, or hidden-evaluator feedback. expected_metrics: quality: >- Green visible verification should no longer coexist with a nonzero result from a declared visible obligation. No claim is made that all hidden cases or all product defects will be caught. completion: >- Workflow completion remains independent of product-red. Only missing, corrupt, stale, or internally inconsistent evidence may block trustworthy terminalization. cost: >- Additional declared commands may increase verification cost; no total-cost reduction is assumed. Duplicate ad hoc verification may decline. stability: >- Receipt currentness, declaration coverage, stream identity, and product projection binding should be deterministic and auditable across tasks. rejection_conditions: - >- Shell command strings, chains, pipelines, or || true can still collapse multiple obligations into one final status. - >- A confirmed REQ lacks a corresponding ordered result, or command start failure, timeout, or nonzero exit can be counted as green. - >- Operator checks only declaration presence and not whether category/command genuinely tests the stated acceptance. - >- The receipt is not bound to the exact confirmed Spec and current product projection, or changed evidence streams retain validity. - >- Hidden evaluator hooks/cases become Candidate-visible, enter the receipt, or are fed back as an adaptive repair oracle. - >- Honest product-red is converted into a workflow blocker, collapsing the completion and product-quality axes the user required to remain separate. - >- The proposal is claimed to solve D4-r1's missing repair loop, D6/D8 hidden oracle adequacy, or W-01 semantic conflicts without separate evidence. - >- A quality or completion regression is attributable to the new evidence contract. regression_risks: - >- Spec authors may choose superficially passing commands that do not exercise the full observable acceptance. - >- Independent commands may duplicate expensive setup or diverge from the prepared runtime if cwd/env ownership is unclear. - >- Treating command structure as semantic proof could recreate false closure at the evidence layer. historicalDisposition: selectedByUserForSuccessor: true successor: H0601 baseCommit: a601aa86730c620e5a3ef777c32d535d59c82ece implementationTip: afab51580435e8e71d1b80a52d22b72ecd774151 implementationCommits: - a105758 - b74f0ee - 2a5b1d0 - da2b42b - 548c78e - afab515 evaluationStatus: complete evaluationBoundary: >- H0601 formal results evaluate the combined successor, not either proposal in isolation. They cannot by themselves establish separate causal effect sizes. authorizationBoundary: proposalGeneratedByRequest: true historicalProposalSelectedByUser: true historicalImplementationAuthorized: true historicalExperimentAuthorized: true newImplementationAuthorized: false newExperimentAuthorized: false mergeAuthorized: false pushAuthorized: false