Power and MDE Methodology
GeoSC power is a simulation-based design diagnostic. For each effect and duration, let N be attempted simulations, S successful estimates, F=N-S failed estimates, and R successful estimates with p < alpha. Ordinary power is R/N and is available only when F=0. If any draw fails, primary power and its Wilson interval are unavailable, and the row cannot support ordinary MDE recommendations.
The successful-draw rate remains explicitly named:
Failed draws have unknown rejection outcomes. The observed all-attempt rate therefore has completion bounds:
For eight rejecting successes and two failures, conditional rejection is 100% and the completion bounds are 80% to 100%; ordinary power is unavailable. These are missing-outcome completion bounds for the simulated sample, not confidence intervals or bounds on true population power. When every draw fails, conditional rejection is unavailable and the completion bounds are [0,1]. When none fails, the bounds collapse to the observed rate, while Monte Carlo uncertainty remains.
A row is valid only when its complete positive integer attempt count contains
zero failures. This does not establish DGP credibility, absence of future
failures, identification, or statistical calibration. There is no permissive
failure-rate override; power_failure_rate_threshold is retired.
Implemented workflow
For every effect-duration grid point, GeoSC:
- pivots the long CSV to a unit-by-period matrix and sorts parsed dates with their outcomes at the shared calculator boundary;
- uses periods before
treatment_dateas the fixed observed pre-period; - demeans each unit, fits an SVD factor model, and estimates factor dynamics;
- simulates a post-period panel;
- computes one baseline as the mean across all treated units and all pre-periods;
- multiplies that pooled baseline by
effect_sizeand applies the resulting same absolute effect path to every treated unit; - appends simulated post data to the observed pre-period;
- calls SparseSC with treated periods at the split and counts
results.p_value < alpha.
The direct calculator excludes rows outside its declared treatment and control sets before DGP fitting, retaining eligible rows in source order. File-driven power defines those sets as the configured treated geographies and all other CSV geographies. Unit identifiers must be unique and non-missing, and the two sets must be disjoint and present in the panel.
This denominator means a configured 10% lift is relative to the pooled treated pre-period mean. It is not a separate 10% of each treated unit’s own baseline. The output records the baseline and full absolute injection.
The power loop uses the SparseSC estimator and its top-level placebo p-value,
but it is not identical to geosc infer. It does not run inference’s panel
policies, cooldown resolver, assumption checks, CI configuration, result
processor, or full SparseSC config surface. “Same inference path” is therefore
too strong; align panels and settings explicitly.
Power supports retrospective fast RidgeCV detection and shares inference’s
method/CV option builder for that path. It rejects other method requests and
simulation-owned window, CI or placebo-seed keys rather than discarding them.
Accepted CV controls are consumed by the native learner and recorded as JSON
estimator_settings in each row. Both stages retain their independent YAML
and placebo caps, so align the intended procedure explicitly.
Ignored lasso controls are retired. Match-space scoring uses a documented continuous multi-output regression subset; recognising a sklearn scorer name alone is insufficient because an incompatible scorer can yield all-NaN CV scores without raising an exception. Native checks verify the supported criteria on a manufactured panel; they do not establish campaign calibration.
Each successful simulation must return a finite real numeric p-value in
[0, 1]; booleans, strings, arrays, missing values and out-of-range values count
as failures. The original scalar is used for range checks and the strict
p_value < alpha comparison, avoiding rounding across a boundary. Invalid
outputs appear in n_invalid_p_values; other failures appear in
n_estimation_failures, and both contribute to n_failed.
The diagnostic conditional_power divides detections by successful estimates.
It must not be substituted for ordinary power or used to select an MDE. A failed
row remains unavailable even if the lower completion bound exceeds target power;
no failure-handling estimand is silently introduced. The corrections do not
establish empirical size, coverage or power calibration.
Effect patterns
constantandimmediate: full absolute effect in every simulated period;ramp: linear sequence from zero to full effect, inclusive;decay: full effect followed by decay;- any other value: configuration error.
The 0.1 decay rate is not configurable in version 0.4.0.
Grid-based MDE
For duration and target power , GeoSC reports the smallest valid tested effect that reaches the target:
target_power defaults to 0.80 in the CLI and pipeline but is configurable. No
interpolation is performed. If no effect reaches the target, MDE is above the
tested range.
Monte Carlo uncertainty
The CSV includes a Wilson interval around the estimated detection proportion.
Its confidence setting uses the configured alpha. This interval describes
finite simulation uncertainty, not DGP misspecification, donor-selection
uncertainty, or future-campaign uncertainty. If it straddles target power,
increase n_simulations before making a threshold decision.
Worked output
| duration | effect_size | power | interval | failure_rate | valid |
|---|---|---|---|---|---|
| 28 | 0.05 | 0.42 | 0.36–0.48 | 0.00 | true |
| 28 | 0.10 | 0.76 | 0.71–0.81 | 0.00 | true |
| 28 | 0.15 | 0.91 | 0.87–0.94 | 0.00 | true |
| 35 | 0.10 | 0.84 | 0.79–0.88 | 0.00 | true |
| 42 | 0.10 | unavailable | unavailable | 0.25 | false |
At 80% target power, the 28-period grid-based MDE is 15% and the 35-period MDE is 10%. The 42-period row is withheld because some draws failed.
Power is computationally intensive because total work scales approximately with effect sizes × durations × simulations × SparseSC fit and placebo cost. This differs from a closed-form regression MDE, but computational intensity is not by itself evidence of better calibration.