Skip to main content

Placebo Inference

GeoSC uses SparseSC in-space placebo inference because geo experiments often have too few treated units for conventional large-sample approximations.

For each treated-sized subset of eligible controls, SparseSC constructs an honest placebo effect. The top-level p-value compares the absolute average treated effect with the absolute average placebo effects and applies an add-one correction. It is two-sided.

Read the empirical reference set

Histogram of honest placebo mean effects, with the observed mean marked and both absolute-effect tails shaded; annotations give the add-one p-value and finite resolution. Histogram of honest placebo mean effects, with the observed mean marked and both absolute-effect tails shaded; annotations give the add-one p-value and finite resolution.

This uses the same synthetic example as the counterfactual chart. With one treated geography and 49 controls, all 49 single-control placebo assignments are enumerated. The bars show honest placebo mean effects from the native result; marks below the axis locate each value. Orange shading identifies both tails where the absolute placebo mean is at least as large as the absolute observed mean. The printed numerator includes one for the add-one correction. Histogram bins are only a display choice; counts and the p-value are computed from individual placebo values, including ties.

The finest p-value step is 1/50, or 0.02, in this example. No Gaussian distribution is fitted. The figure explains the implemented comparison; it does not establish exchangeability or statistical calibration. The numerical source records the observed mean, every placebo mean and the counted exceedances.

Each eligible physical control appears once in the reference matrix. A subset contains distinct geographies, even when some residual paths have equal values. All treated units share one launch and measurement window.

The unchanged backend uses ten honest cross-fitting folds, so GeoSC requires at least ten effective controls before SparseSC fitting. At the ten-control boundary, each fold holds out one control and retains nine for fitting in the retrospective fast-path fixture. This is an operational feasibility check, not a qualification of error rates or donor exchangeability. Adding periods or treated geographies does not increase the control count. Match-space CV settings have separate training-size constraints; lowering sparse_sc_cv_folds does not change honest cross-fitting.

When feasible assignments exceed sparse_sc_max_n_pl, SparseSC samples assignments using sparse_sc_placebo_seed; otherwise it enumerates them. The result reports the possible and effective counts and labels the mode exact, sampled, or unknown. Here exact means exhaustive enumeration of the supported physical-geography assignment set; it does not assert exact causal calibration. Sampled draws can repeat an assignment across draws, so the effective count is the number of draws, not necessarily the number of unique assignments.

Finite support matters. With M∗M^* effective placebos the nominal resolution is 1/(M∗+1)1/(M^*+1). If this exceeds alpha, significance at that alpha cannot be attained. Sampling adds reference-set variability even when the optimiser is stable; preserve the seed.

The p-value is calibrated only relative to the placebo assignment logic and its exchangeability assumptions. Non-comparable donors, spillover, differential measurement changes, or poor pre-fit can make a numerically small p-value misleading. SparseSC placebo intervals are likewise finite-reference-set quantities, not model-free guarantees.