Skip to main content

Power and MDE Methodology

GeoSC power is a simulation-based design diagnostic. For each effect and duration, let N be attempted simulations, S successful estimates, F=N-S failed estimates, and R successful estimates with p < alpha. Ordinary power is R/N and is available only when F=0. If any draw fails, primary power and its Wilson interval are unavailable, and the row cannot support ordinary MDE recommendations.

The successful-draw rate remains explicitly named:

P^conditional=R/S(S>0).\widehat{P}_{\mathrm{conditional}} = R/S \quad (S>0).

Failed draws have unknown rejection outcomes. The observed all-attempt rate therefore has completion bounds:

[R/N, (R+F)/N].[R/N,\ (R+F)/N].

For eight rejecting successes and two failures, conditional rejection is 100% and the completion bounds are 80% to 100%; ordinary power is unavailable. These are missing-outcome completion bounds for the simulated sample, not confidence intervals or bounds on true population power. When every draw fails, conditional rejection is unavailable and the completion bounds are [0,1]. When none fails, the bounds collapse to the observed rate, while Monte Carlo uncertainty remains.

A row is valid only when its complete positive integer attempt count contains zero failures. This does not establish DGP credibility, absence of future failures, identification, or statistical calibration. There is no permissive failure-rate override; power_failure_rate_threshold is retired.

Implemented workflow

For every effect-duration grid point, GeoSC:

  1. pivots the long CSV to a unit-by-period matrix and sorts parsed dates with their outcomes at the shared calculator boundary;
  2. uses periods before treatment_date as the fixed observed pre-period;
  3. demeans each unit, fits an SVD factor model, and estimates factor dynamics;
  4. simulates a post-period panel;
  5. computes one baseline as the mean across all treated units and all pre-periods;
  6. multiplies that pooled baseline by effect_size and applies the resulting same absolute effect path to every treated unit;
  7. appends simulated post data to the observed pre-period;
  8. calls SparseSC with treated periods at the split and counts results.p_value < alpha.

The direct calculator excludes rows outside its declared treatment and control sets before DGP fitting, retaining eligible rows in source order. File-driven power defines those sets as the configured treated geographies and all other CSV geographies. Unit identifiers must be unique and non-missing, and the two sets must be disjoint and present in the panel.

This denominator means a configured 10% lift is relative to the pooled treated pre-period mean. It is not a separate 10% of each treated unit’s own baseline. The output records the baseline and full absolute injection.

The power loop uses the SparseSC estimator and its top-level placebo p-value, but it is not identical to geosc infer. It does not run inference’s panel policies, cooldown resolver, assumption checks, CI configuration, result processor, or full SparseSC config surface. “Same inference path” is therefore too strong; align panels and settings explicitly.

Power supports retrospective fast RidgeCV detection and shares inference’s method/CV option builder for that path. It rejects other method requests and simulation-owned window, CI or placebo-seed keys rather than discarding them. Accepted CV controls are consumed by the native learner and recorded as JSON estimator_settings in each row. Both stages retain their independent YAML and placebo caps, so align the intended procedure explicitly.

Ignored lasso controls are retired. Match-space scoring uses a documented continuous multi-output regression subset; recognising a sklearn scorer name alone is insufficient because an incompatible scorer can yield all-NaN CV scores without raising an exception. Native checks verify the supported criteria on a manufactured panel; they do not establish campaign calibration.

Each successful simulation must return a finite real numeric p-value in [0, 1]; booleans, strings, arrays, missing values and out-of-range values count as failures. The original scalar is used for range checks and the strict p_value < alpha comparison, avoiding rounding across a boundary. Invalid outputs appear in n_invalid_p_values; other failures appear in n_estimation_failures, and both contribute to n_failed.

The diagnostic conditional_power divides detections by successful estimates. It must not be substituted for ordinary power or used to select an MDE. A failed row remains unavailable even if the lower completion bound exceeds target power; no failure-handling estimand is silently introduced. The corrections do not establish empirical size, coverage or power calibration.

Effect patterns

  • constant and immediate: full absolute effect in every simulated period;
  • ramp: linear sequence from zero to full effect, inclusive;
  • decay: full effect followed by exp⁡(−0.1t)\exp(-0.1t) decay;
  • any other value: configuration error.

The 0.1 decay rate is not configurable in version 0.4.0.

Grid-based MDE

For duration dd and target power qq, GeoSC reports the smallest valid tested effect that reaches the target:

MDEd=min⁡{e∈E:valid(e,d), Power^(e,d)≥q}.MDE_d = \min\{e \in \mathcal E: valid(e,d),\ \widehat{Power}(e,d) \geq q\}.

target_power defaults to 0.80 in the CLI and pipeline but is configurable. No interpolation is performed. If no effect reaches the target, MDE is above the tested range.

Monte Carlo uncertainty

The CSV includes a Wilson interval around the estimated detection proportion. Its confidence setting uses the configured alpha. This interval describes finite simulation uncertainty, not DGP misspecification, donor-selection uncertainty, or future-campaign uncertainty. If it straddles target power, increase n_simulations before making a threshold decision.

Worked output

duration effect_size power interval failure_rate valid
28 0.05 0.42 0.36–0.48 0.00 true
28 0.10 0.76 0.71–0.81 0.00 true
28 0.15 0.91 0.87–0.94 0.00 true
35 0.10 0.84 0.79–0.88 0.00 true
42 0.10 unavailable unavailable 0.25 false

At 80% target power, the 28-period grid-based MDE is 15% and the 35-period MDE is 10%. The 42-period row is withheld because some draws failed.

Power is computationally intensive because total work scales approximately with effect sizes × durations × simulations × SparseSC fit and placebo cost. This differs from a closed-form regression MDE, but computational intensity is not by itself evidence of better calibration.