Polymarket Lab — Statistical Strategy Test-Bench
This is a statistical research bench, not a real-money trading bot. It streams public prediction-market and weather data into a large time-series store, simulates execution conservatively and requires a strategy to clear a bootstrap significance gate before its results are treated as evidence.
01 · Problem
What needed solving
A favourable paper-trading curve is easy to produce by chance, especially when several strategies are tried on small samples. The real problem is building an apparatus that can distinguish a plausible signal from luck, stale data and optimistic simulation.
Learn how prediction-market systems work end to end while imposing an architectural boundary that makes real-money execution impossible.
02 · Constraints
The difficult parts
- Separate apparent profit from sampling luck and multiple-strategy selection effects.
- Simulate latency, fees, depth and stale quotes without turning missing data into favourable assumptions.
- Collect and query tens of millions of time-series rows continuously on one mini-PC.
- Keep the system structurally paper-only, with no wallet, keys, signing code or live-order path.
03 · Approach
How the work was structured
- Built a four-service stack: PostgreSQL, an internal FastAPI service, an asynchronous worker and an eight-page Dash interface.
- Gave each strategy an independent virtual portfolio and ran a do-nothing cash control under the same accounting rules.
- Centred per-trade returns and used 2,000 bootstrap resamples against a no-edge null before allowing a strategy to progress beyond evaluation.
- Applied empirical-Bayes shrinkage, Wilson intervals, proper scoring rules and uncertainty-labelled ensemble forecasts.
- Used pessimistic paper execution with simulated latency, depth-limited fills, fee-aware sizing, rejection reasons and an append-only ledger.
04 · Validation
What can be claimed now
The application contains 167 tests across unit, property, integration, recorded-payload contract and deterministic replay layers. Strict typing and linting run as part of the quality gate. The system refuses to start unless paper-only invariants are satisfied.
The live research bench has accumulated a 16 GB database with more than 40 million market and forecast rows. Most generated signals are rejected before simulated execution; no result is presented as evidence of future profitability, and no real order can be placed.
05 · Learning
What the project clarified
- A benchmark and a rejection funnel are more informative than a strategy count.
- Simulation is useful only when it is pessimistic in the places where real execution fails.
- A result should earn promotion through sample size and significance, not through an attractive equity curve.
- Deleting an unsupported strategy is a successful research outcome.