Status: Toolkit · Open. A collection of methods under active development, open to contribution and revision.
The Counterfactual Toolkit provides methods for estimating what would have happened anyway, so that impact is measured as the difference made rather than the activity logged. It exists because the most common measurement error is to count outputs — the quantity of activity — as if they were outcomes attributable to the actor.
Core definition
Impact is the observed outcome minus the counterfactual baseline: the state that would have obtained without the intervention. Where the counterfactual outcome equals the observed one, measured impact is near zero, however large the activity. The toolkit's central operation is therefore subtraction, not accumulation — and the quantity being subtracted, the world in which nothing was done, is precisely the quantity that can never be observed directly. Every method below is an approach to estimating it under that constraint.
Methods
- Baseline construction. Establishing a defensible account of the no-intervention world, from trend data, comparison groups, or reasoned estimate.
- Additionality. Isolating the portion of an outcome that is genuinely additional, net of what would have happened regardless.
- Attribution versus contribution. Determining whether an outcome can be assigned to one actor or is shared among several, and sizing a partial contribution honestly.
- Causal caution. Distinguishing causation from correlation — never assuming that an outcome which followed an action was caused by it.
- Ranged estimates. Reporting counterfactuals as ranges with stated confidence, since the counterfactual world cannot be directly observed.
Known limitation
Reliable causal identification is unsolved. The Chain of Impact — the movement's mechanism for tracing what actually resulted from an action — is specified only in outline, and the distinction between causation and correlation is an explicitly open problem. As a result, many counterfactual figures can be estimated but not proven, and the toolkit requires that they be labeled accordingly.
Status
The toolkit is open and evolving. It is best understood as a discipline rather than a formula: subtract what would have happened anyway, state the basis for the subtraction, and never present an estimate as a measurement. Its methods pair directly with the Impact Measurement Protocol's capture stage and with the Transparency and Disclosure Standard's requirement to disclose the counterfactual used.