Trap Suite

The Trap Suite is a set of strategies that are known to lie, each with the verdict the verifier must return. It lives in tests/traps/ and runs on every CI build. If a change to the verifier stops catching a trap, or starts accusing an honest strategy, CI fails.

Every trap is a normal strategy file with signal(df). The manifest in tests/traps/test_trap_suite.py gives each file a dataset, the allowed verdicts and the check statuses that must appear.

Catalogue

50 traps, 18 honest controls, 2 parameterized strategies for verify_grid and a data-snooping test. "Caught by" lists the checks that flag each trap on the random-walk dataset. "lint (warn)" is a warning only; the dynamic probes produce the REJECT.

Shifted or reversed time

File	How it lies	Caught by
lookahead_shift	shift(-1) puts the next close into today's signal	truncation, perturbation, lint, implausible accuracy
diff_negative	diff(-1) is the next bar's return	truncation, perturbation, lint, implausible accuracy
pct_change_negative	pct_change(periods=-3) compares with three bars ahead	truncation, perturbation, lint, implausible accuracy
roll_negative	np.roll(x, -1) pulls tomorrow into today	truncation, perturbation, lint, implausible accuracy
reverse_rolling	Rolling window over a reversed series	truncation, perturbation, lint, implausible accuracy
last_row_leak	Every bar compared with .iloc[-1], the final close	truncation, perturbation, lint
gradient_leak	np.gradient uses central differences (bar t + 1)	truncation, perturbation, lint, implausible accuracy
reversed_cummax	cummax over the reversed series is the highest price still to come	truncation, perturbation, lint
reversed_accumulate	np.maximum.accumulate over the reversed array	truncation, perturbation, lint
forward_window_indexer	FixedForwardWindowIndexer makes rolling() look ahead	truncation, perturbation, lint, implausible accuracy
tail_threshold	Threshold from .tail(500).mean(), the last bars of the dataset	truncation, perturbation, lint
iat_last	.iat[-1] reads the final close	truncation, perturbation, lint
flip_cumsum	np.flip(np.cumsum(np.flip(ret))) sums the returns still to come	truncation, perturbation, lint
shift_variable	horizon = -1; close.shift(horizon): the negative shift hides in a variable	truncation, perturbation, lint, implausible accuracy

Centered windows and filters

File	How it lies	Caught by
centered_window	rolling(..., center=True)	truncation, perturbation, lint, implausible accuracy
convolve_same	np.convolve(mode="same") centres the kernel	truncation, perturbation, lint, implausible accuracy
centered_variable	centred = True; rolling(21, center=centred)	truncation, perturbation, lint, implausible accuracy
fft_denoise	FFT low-pass over the whole series	truncation, perturbation, lint (warn)

Filling gaps from the future

File	How it lies	Caught by
bfill_leak	Sparse series backward-filled	truncation, perturbation, lint, implausible accuracy
interpolate_leak	interpolate() uses the next known value	truncation, perturbation, lint
reindex_nearest	reindex(method="nearest") aligns bars with the next hour's close	truncation, perturbation, lint
np_interp_fill	np.interp draws a line to the next known point across gaps	truncation, perturbation, lint
merge_asof_forward	merge_asof(direction="forward")	truncation, perturbation, lint

Whole-sample statistics

File	How it lies	Caught by
global_zscore	z-score with the mean and std of the whole series	truncation, perturbation, lint (warn)
numpy_global_stat	np.mean / np.std thresholds over the whole array	truncation, perturbation, lint (warn)
full_rank	Percentile rank against the whole history	truncation, perturbation, lint (warn)
qcut_full	pd.qcut quantile buckets of the whole sample	perturbation, lint (warn)
argsort_rank	Double argsort rank against the whole sample	truncation, perturbation, lint (warn)
idxmax_leak	Long until the bar of the all-time high	perturbation, lint (warn)
cumsum_total_norm	Cumulative share of the whole-sample total	truncation, lint (warn)
full_polyfit	Trend fitted once on the whole series	truncation, perturbation, lint (warn)
target_encoding_leak	Mean forward return per bucket, fitted on everything	truncation, perturbation, lint
sort_values_rank	sort_values ranks every bar among all prices	truncation, perturbation, lint (warn)
np_sort_rank	np.sort + searchsorted ranks each close among all closes	truncation, perturbation, lint (warn)
cut_auto_bins	pd.cut(bins=5) takes edges from the whole-series min and max	truncation, perturbation, lint (warn)
builtin_max	Python's max() over the whole column	perturbation, lint (warn)
describe_threshold	Quartiles from describe() over the whole series	truncation, perturbation, lint (warn)
agg_zscore	z-score from agg(["mean", "std"]) over the whole series	truncation, perturbation, lint (warn)
mode_level	round(-1).mode(): most frequent level over the whole dataset	truncation, perturbation, lint (warn)
value_counts_level	value_counts().idxmax(): busiest level, future included	truncation, perturbation, lint (warn)
nlargest_dates	nlargest(100) finds the dataset's highest closes	perturbation, lint (warn)

Aggregates over the current bucket

File	How it lies	Caught by
hourly_close_leak	Each minute sees its hour's final close	truncation, perturbation, lint (warn)
resample_max_leak	Each bar sees the maximum of its 15-minute bucket	truncation, lint (warn)
hourly_close_map	groupby().last() mapped back onto every minute of the hour	truncation, perturbation, lint (warn)
hour_size_leak	transform("size") knows how many bars the hour will have	truncation, lint (warn)
bars_left_in_hour	cumcount(ascending=False) counts the bars still to come	truncation, lint
resample_ffill_max	resample("h").max() forward-filled from the hour's first minute	truncation, lint (warn)

Invisible to the static lint

These leak without any suspicious call. Only the dynamic probes catch them, which is why Monte-Neo runs the strategy instead of only reading it.

File	How it lies	Caught by
dataset_fraction	np.arange(len(df)) / len(df): a bar's position depends on how many bars come later	truncation
block_mean_reshape	reshape(-1, 60).mean(axis=1).repeat(60): every bar sees the rest of its block	truncation, perturbation

Economics

File	How it lies	Caught by
high_turnover	Honest code, but the edge cannot pay its costs	net profitability
data snooping (test)	Best of 200 random strategies with n_trials hidden	deflated Sharpe once n_trials is declared

Honest controls

Honest controls must never fail a look-ahead check. False alarms cost as much trust as missed leaks.

File	Pattern
sma_cross	SMA crossover
ewm_cross	EMA crossover
momentum	One-bar momentum on a dataset with a real edge (must PASS)
expanding_zscore	z-score against expanding mean and std
expanding_rank	Percentile rank within the past only
resample_shifted	Previous completed bucket, carried forward
cummax_drawdown	Drawdown from the running peak
convolve_causal	Trailing average via np.convolve(mode="full")[:n]
rolling_quantile_band	Breakout above a lagged rolling quantile
prev_hour_close_map	Previous completed hour: groupby().last().shift(1)
bars_into_hour	cumcount() counts only bars already seen
expanding_quantile_band	Expanding quantile of past closes, shifted by one bar
cut_fixed_bins	pd.cut with explicit, fixed bin edges
hour_running_high	Running high of the hour so far: groupby().cummax()
rolling_min_periods	rolling(50, min_periods=1)
rolling_apply_span	rolling().apply(lambda w: w[-1] - w[0]): w[-1] is the current bar
hour_open_ref	transform("first"): the hour's first close is already known
expanding_max_breakout	expanding().max().shift(1)

Grid strategies: sma_params, momentum_params.

Contribute a trap

Found a way an agent fooled itself? Send it as a trap.

1. Open a trap submission issue, or send a PR directly.
2. Add tests/traps/strategies/<name>.py:
3. The first line is a docstring that starts with TRAP: or HONEST: and says how it lies.
4. Define signal(df) using only pandas and numpy.
5. Keep it minimal: the smallest code that reproduces the mistake.
6. Add a row to TRAPS in tests/traps/test_trap_suite.py: (name, dataset, allowed_verdicts, required_check_statuses). Use random_walk for leaks (no edge exists, so any profit is suspicious) and planted for a real edge. Add honest controls to HONEST too.
7. Add the file to the catalogue above. A test checks that every strategy file is listed here.
8. Run uv run pytest tests/traps tests/unit/test_verify_grid.py.

If the verifier does not catch your trap yet, still send it. Mark the row with the verdict it gets today and describe the miss in the PR. A known miss is a roadmap item.