Alpha calibration: false alarms and power¶
Generated by scripts/alpha_calibration.py — do not edit.
Each trap and each honest twin of the A15 zoo (tests/zoo/alpha_cases.py) was generated with 20 seeds (40 symbols, 1000 business days unless a case needs more) and run through validate_alpha with 60 placebos and an AUM of 1e6. Power is the share of trap runs where the named check gave one of the statuses that count as caught (see the case list in the test file); a false alarm is a twin run where the same check did not give its expected quiet status. Intervals are Wilson 95 %. Targets: power at least 90%, false alarms at most 5% (judged on the point estimate).
| Check | Trap | Power | Honest twin | False alarms | Targets met |
|---|---|---|---|---|---|
residual_alpha |
repackaged_momentum | 20/20 = 100% (84%–100%) | independent_signal | 0/20 = 0% (0%–16%) | yes |
hidden_short_volatility |
short_volatility | 20/20 = 100% (84%–100%) | symmetric_payoff | 1/20 = 5% (1%–24%) | yes |
universe_subsampling |
three_symbols | 20/20 = 100% (84%–100%) | spread_edge | 0/20 = 0% (0%–16%) | yes |
rebalance_timing_luck |
first_day_of_month | 20/20 = 100% (84%–100%) | uniform_edge | 0/20 = 0% (0%–16%) | yes |
short_feasibility |
illiquid_shorts | 20/20 = 100% (84%–100%) | liquid_shorts | 0/20 = 0% (0%–16%) | yes |
universe_lookahead |
survivor_universe | 20/20 = 100% (84%–100%) | names_come_and_go | 0/20 = 0% (0%–16%) | yes |
universe_lookahead |
membership_ignored | 20/20 = 100% (84%–100%) | membership_respected | 0/20 = 0% (0%–16%) | yes |
cross_market_close_alignment |
same_date_new_york | 20/20 = 100% (84%–100%) | lagged_new_york | 0/20 = 0% (0%–16%) | yes |
post_discovery_decay |
dies_after_discovery | 20/20 = 100% (84%–100%) | persistent_edge | 0/20 = 0% (0%–16%) | yes |
signal_placebo |
noise_score | 20/20 = 100% (84%–100%) | weak_real_edge | 0/20 = 0% (0%–16%) | yes |
synthetic_market_null |
noise_score | 19/20 = 95% (76%–99%) | weak_real_edge | 0/20 = 0% (0%–16%) | yes |
marginal_sharpe |
book_clone | 16/20 = 80% (58%–92%) | independent_book | 0/20 = 0% (0%–16%) | no |
noise_injection |
exact_values | 20/20 = 100% (84%–100%) | robust_ranking | 0/20 = 0% (0%–16%) | yes |
breadth_consistency |
inflated_claim | 20/20 = 100% (84%–100%) | honest_claim | 0/20 = 0% (0%–16%) | yes |
Checks that miss their targets¶
marginal_sharpe(book_clone / independent_book): power 80% < 90%
Median Alpha Readiness Level¶
| Trap | ARL | Honest twin | ARL |
|---|---|---|---|
| repackaged_momentum | 4 | independent_signal | 7 |
| short_volatility | 2 | symmetric_payoff | 6 |
| three_symbols | 7 | spread_edge | 7 |
| first_day_of_month | 2 | uniform_edge | 7 |
| illiquid_shorts | 2 | liquid_shorts | 2 |
| survivor_universe | 7 | names_come_and_go | 6 |
| membership_ignored | 1 | membership_respected | 4 |
| same_date_new_york | 1 | lagged_new_york | 4 |
| dies_after_discovery | 4 | persistent_edge | 7 |
| noise_score | 2 | weak_real_edge | 6 |
| book_clone | 7 | independent_book | 8 |
| exact_values | 6.5 | robust_ranking | 7 |
| inflated_claim | 6 | honest_claim | 6 |
How to read this¶
The traps and twins are synthetic and were written by the maintainers; the numbers measure how the checks behave on these generators at this signal strength, not on real markets. A check with a statistical test (signal_placebo, synthetic_market_null) has a false-positive rate set by its own threshold (p above 0.10 fails a score with no edge), so its power on a pure-noise score cannot exceed about 90 % by design. Twins have a deliberately strong edge; with a weaker edge the checks that look at concentration (universe_subsampling) or at the t-statistic of the residual (residual_alpha) warn more often. With few seeds the intervals are wide: read the interval, not only the point.