A team agrees that build times are slowing delivery. The improvement is accepted. A ticket is created. Weeks later, the ticket closes after a cache change. The retrospective board is clean.
Did delivery improve? The workflow has three reassuring events—agreement, action, completion—and no answer.
The seduction of closure
Work systems are built to reward transitions. A card moves to done. A pull request merges. A proposal receives a green check. These transitions are useful coordination evidence, but they create a strong feeling of closure before the outcome is known.
The measurement literature warns against treating one convenient proxy as the whole system. The SPACE framework argues that developer productivity cannot be reduced to a single metric or dimension. It spans satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. A local improvement can move one dimension while harming another.
DORA’s 2024 research offers a timely example. Its findings associated AI adoption with gains in individual productivity, flow, and job satisfaction while also reporting negative relationships with delivery throughput and stability. A faster local activity did not automatically become a better system outcome.
The event that is easiest to count is rarely the event that settles the question.
An evidence ladder for improvement
Solv separates five states that ordinary action tracking tends to collapse:
A signal says something may deserve attention. It is not yet a shared conclusion.
A person decides that a proposed response is worth attempting.
The agreed intervention happened as described.
A defined measure changed after the intervention, with context preserved.
The change held long enough, or repeated clearly enough, to influence future decisions.
Each rung can support a different claim. “We agreed” is valid at acceptance. “We changed the process” needs completion. “The change helped” requires outcome evidence. “We learned” requires enough durable context to alter the next decision.
Why accepted-but-unfinished is negative evidence
Many learning systems label an accepted recommendation as positive because approval is an observable response. That trains the system toward agreement. It rewards proposals that sound right at decision time, even if nobody can execute them or the organization quietly abandons them.
Solv uses a stricter hierarchy: impact over completion over acceptance. An accepted proposal that was never completed is a negative example. This does not mean the idea was bad or the person failed. It means the proposal did not survive contact with priorities, ownership, feasibility, or time—and a learning system should not pretend otherwise.
The distinction is an anti-sycophancy mechanism. If the system learns primarily from approval, it learns to generate what people are inclined to accept. If it learns from completion and impact, it has to confront what the organization could actually carry and what changed afterward.
Measure the outcome, not the person
Outcome measurement creates its own risk. A team trying to understand flow can drift into ranking individuals by activity. A system built to improve work can become a system for evaluating workers.
The safer unit is the intervention and its shared environment:
- Did median build feedback time change after the cache intervention?
- Did failed deployment recovery improve after the runbook experiment?
- Did the same handoff signal recur after ownership changed?
- Did team-reported safety hold while capture became more continuous?
These are not profiles. They are questions about a change and its context. Identity can remain pseudonymous while the organization still becomes accountable for whether its commitments worked.
From signed history to a learning corpus
Outcome labels are valuable only when the history behind them is trustworthy. If a record can be silently edited after the result is known, the learning data can be made to flatter the decision.
Solv uses deterministic event history and signed per-team commitment chains so any reader can verify the sequence before trusting it. The learning corpus carries the proposal, human resolution, commitment, cited outcome, and the graph between them. Tampered history is rejected rather than converted into training data.
This does not solve causal inference. Teams still need baselines, counterevidence, and humility about confounding changes. It does create a stronger minimum: the evidence cannot claim more than the event sequence supports, and later learning cannot erase the path that produced it.
The practical research question is whether this hierarchy improves proposal quality over time. A useful evaluation would compare models trained on acceptance labels with models trained on completion and impact labels, then measure not persuasiveness but executable, outcome-bearing work.