Skip to main content

Power Analysis Results Schema

power_analysis_results.csv has one row per effect-size and duration pair.

Field Meaning
schema_version Row contract version (2.0); older rows are withheld from decision consumers.
estimator_settings Deterministic JSON recording effective native method/CV/defaults, match-space learner, cross-fitting folds/seed, T0/T1, placebo cap and effect-CI request. Additive schema 2.0 provenance.
target_power CLI planning threshold copied from YAML.
effect_size, duration, alpha Simulated grid and detection threshold.
power n_significant / n_simulations, available only when every draw succeeds.
conditional_power n_significant / n_successful; unavailable when no draws succeed.
power_all_attempt_lower_bound, power_all_attempt_upper_bound n_significant / n_simulations and (n_significant + n_failed) / n_simulations; completion bounds, not confidence intervals.
power_ci_lower, power_ci_upper Wilson interval for a complete run, using alpha as the tail setting; unavailable if any draw fails.
n_simulations, n_successful, n_significant, n_failed Monte Carlo counts.
n_invalid_p_values, n_estimation_failures Invalid estimator output versus generation/estimation failures; their sum equals n_failed. Failure categories retained from the previous schema.
failure_rate, valid Failure diagnostics; valid requires zero failures. It does not validate DGP plausibility.
random_seed, simulation_seed, placebo_seed_root, placebo_seed_strategy Reproducibility metadata.
dgp_rank, dgp_explained_var, dgp_factor_dynamics Fitted DGP metadata.
requested_max_n_pl, possible_placebos, effective_max_n_pl, p_value_placebo_mode Placebo support.
dgp_backend, generation_backend, seeded_reproducibility_mode Execution metadata.
effect_pattern, effect_baseline, effect_full_absolute, effect_baseline_definition Injection contract.
detection_rule, power_denominator Strict detection rule and primary denominator all_simulations.
warnings Semicolon-delimited row warnings.

Direct calls to SparseSCPowerCalculator.calculate_power() do not add the CLI’s target_power column; target power affects MDE selection, not simulation.

New rows use estimator_settings.model_type: retrospective, fast: true and match_space: ridge_cv. cv_options records explicit settings with null native defaults for cv, scoring and gcv_mode. Honest cross-fitting uses ten folds and seed 110011; T0, T1, max_n_pl and ret_CI: false describe the simulated native call. Per-draw placebo seeds remain described in the existing row seed fields. This does not capture every private solver detail or prove calibration.

Older schema 2.0 rows can lack this new provenance. Numeric decision eligibility is unchanged; regenerate rows when method provenance is required, especially if earlier non-default settings were supplied and silently ignored.

A successful draw must return a finite real numeric p-value in [0, 1]. Missing values, arrays, strings and booleans are invalid outputs. Valid zero and one are retained, and significance uses the original scalar’s strict p_value < alpha comparison. Counts and warnings distinguish invalid p-values from execution failures. If any draw fails, ordinary power and its primary interval bounds are NaN in the DataFrame and empty cells in the CSV. The row is valid: false and cannot produce ordinary plots or MDE recommendations. The conditional rejection rate remains diagnostic; all-failed rows have unavailable conditional rates and all-attempt completion bounds [0, 1].

Completion bounds describe the observed attempts with unknown failed-draw rejection outcomes. They do not include Monte Carlo uncertainty or bound true population power. With no failures they collapse to the observed rejection rate, although its Wilson interval still has sampling width.

The retired failure_rate_threshold output is absent. Remove the retired power_failure_rate_threshold input and regenerate old results. Decision consumers require schema 2.0, complete positive whole attempt counts, matching success counts, zero failures, true validity, and finite numeric primary fields. They normalise accepted CSV numbers before sorting, comparisons and plotting; unsupported or malformed rows remain excluded even with a stale true flag.