End-to-End Synthetic Study

This tutorial uses the repository’s synthetic panel to exercise all three stages and their explicit hand-offs.

1. Inspect the stage configs

The pipeline requires three canonical filenames in one directory:

• power_analysis_config.yaml;
• donor_eval_config.yaml;
• geolift_analysis_config.yaml.

Confirm that all three identify the same outcome, geography universe, treatment units, and launch date. The key names differ by stage; use the stage-specific configuration reference.

2. Run the pipeline

geosc pipeline \
  --config data-config/geolift_analysis_config.yaml \
  --output-dir outputs/tutorial_run \
  --no-progress

The command runs power, donor evaluation, and inference independently. It then summarises their artefacts. It does not feed recommended donors into inference.

3. Review the design outputs

Open multicell_power_analysis/power_analysis_results.csv. Retain only rows where valid is true. MDE is the smallest tested effect_size that reaches target_power for a duration; it is grid-based and conditional on the simulated DGP.

Open multicell_donor_eval/donor_pool_quality.json. Treat quality_assessment: "INSUFFICIENT", incomplete metrics, dominant recommendation weights, or poor overlap as redesign signals.

4. Review inference

Open multicell_geolift_analysis/geolift_results.json. The primary estimate is the unscaled post-period average effect (att). The top-level p_value is the two-sided in-space placebo p-value for that average effect. Scaled quantities remain diagnostics and must not be combined with the unscaled interval.

5. Record the hand-off

If the donor screen changes eligibility, materialise a new panel containing the treated units and approved donors. Rerun power and inference on that same panel. Record the input hash, configs, exclusions, software version, and random seeds.

Failed Donor Design

Consider a treated geography with five recommended donors. Two have incomplete metrics, one receives 70% of the design-stage recommendation weight, and the pool-level quality_assessment is INSUFFICIENT.

This is not a prompt to fit SparseSC and hope regularisation fixes the design. The donor screen says the counterfactual may depend on weak or concentrated comparators. Review time overlap, outcome definitions, treatment leakage, regional shocks, and geography aggregation. Exclude a donor only for a pre-specified eligibility reason, not because its removal improves the eventual effect estimate.

If eligibility changes, rerun donor screening, power, and inference on the same revised geography universe. If no defensible pool remains, reject the design and state which condition failed. See Design Rejection.

Underpowered Design

Suppose a valid power grid contains the following rows for a 28-period test:

At an 80% target, the grid-based MDE is 10%. This does not mean that the campaign is expected to deliver 10%, or that effects below 10% are zero. It means the configured procedure detected smaller injected effects less often than the planning threshold under this DGP.

If commercially plausible lift is 4%, the design is underpowered for that decision. Do not solve the problem by reporting an optimistic grid point or by choosing a longer period after seeing outcomes. Consider more comparable donors, lower-noise outcomes, a defensible larger treated population, a longer pre-specified measurement window, or a different design. If none is available, record the design as infeasible before launch.

Also inspect failures. A row with high estimated power and valid: false is not a usable design result. Power is divided by successful simulations, so ignored failures could otherwise make it look better than it is.