I Built a Weather Prediction Bot. Then Live Trading Humbled It.

I'm 46 years old. I have a commercial pilot certificate, 391.3 flight hours, and a clear problem: I need 108.7 more hours to reach a 500-hour planning threshold. At roughly $200 an hour, that is about $21,740. Five hundred is not a universal hiring minimum, and paper-market results do not fund flight time.

That number had been sitting on my desk for a while. I tried several angles: an iOS app, KDP workbooks, content work, and weather-market research. At the time, OpenClaw orchestrated parts of that workflow. That runtime is now retired. The legacy weather hypotheses are closed and live trading is off; separate paper research is not a continuation or a revenue claim.

Here's where my head went: I'm a pilot. Weather is not a hobby topic for me. I've been reading METARs, TAFs, area forecasts, and prog charts for two years — the same skills I built while earning my commercial and instrument ratings. I know what NOAA's models are good at and where they fall apart. And every time a weather event rolls through, there are real money markets on Kalshi asking things like "Will the high temperature in Phoenix exceed 105°F today?" or "Will it rain more than 0.25 inches in Seattle this week?"

I thought: I probably have better-than-random intuition here. What if I built a bot to make those trades systematically and found out?

The bot ran paper for 35 days: 217 resolved trades, 76.5% win rate, and $1,773 in simulated profit. On March 28 it went live with a $424 bankroll. Later, after stripping noisy results and auditing calibration, the honest answer was uglier: the weather-only live run was not capital-ready, and subsequent paper research failed to prove a durable market-relative edge.

April 25 update: Kalshi weather is paused. The newest gate audit fixed bogus candidate-flow blockers, but the model still loses to the market baseline and needs more clean paper/live-gated data. I am leaving the original build notes below because the failure mode is the lesson.

August 27 final update: The weather-market hypothesis is closed, not waiting for another threshold adjustment. The preserved forward paper bankroll finished at $175.01, with −$24.99 realized paper P&L, 1 win, 19 losses, and zero open positions. Later versions either lost to the market baseline, produced zero valid opportunities, lacked trustworthy historical provenance, or found zero executable settlement-source crossings. The collectors are paused, the evidence is preserved, and live trading remains off. I am leaving the original build notes below because the failure mode is the useful part.

Why Weather Prediction Markets

Kalshi is a regulated prediction market exchange. You trade on yes/no questions about future events. Unlike sports betting or crypto, the underlying "asset" is something with actual data behind it — government forecasts, historical records, numerical weather models that have been refined for decades.

Weather markets appealed to me for a few specific reasons.

First, the edge is knowable. Temperature forecasts from NOAA and Open-Meteo aren't random. They're probabilistic outputs from physics-based models. If I can extract a probability estimate that's better calibrated than what the market is pricing, I have a real edge.

Second, the events resolve fast. A "will it hit 100°F in Phoenix today?" market resolves by end of day. That's very different from holding a position for weeks. Fast resolution means I learn quickly whether my model is working.

Third, as a pilot, I already had the domain knowledge. I didn't need to learn what a 500mb chart means. I didn't need to understand the difference between ensemble spread and deterministic output. That was already in my head from instrument training — the same skills I built while earning my instrument rating and commercial certificate.

So I started building.

How the Bot Works

Data Sources

The original bot pulled from two forecast APIs: NOAA's National Digital Forecast Database (NDFD) and Open-Meteo. NOAA is the authoritative US source. Open-Meteo provided ensemble model data with uncertainty spreads. Together they produced a probability distribution for temperature, precipitation, and wind at US locations.

For each market, the original bot identified the relevant location and variable, queried both APIs, and built a forecast ensemble. The ensemble spread represented model uncertainty: wide spread meant low confidence; tight spread meant the models agreed.

Probability Estimation

For bracket markets (things like "will the high exceed X?"), I use a Normal CDF approach. I take the ensemble mean as the expected value and the spread as the standard deviation, then calculate the probability that the outcome falls above or below the threshold.

So if the ensemble says Phoenix high temperature tomorrow has a mean of 103°F with a standard deviation of 4°F, I can calculate the exact probability that it exceeds 105°F. That probability becomes my edge estimate against whatever price the market is offering.

Strategies

The original bot ran two strategies, and this is where the real lesson began.

CALIBRATION was the primary strategy. It looked for markets where my probability estimate and the market price disagreed by a meaningful margin. If the model estimated a 70% probability while the market paid 55 cents on the dollar (implying 55%), it labeled the gap as a 15-point edge.

BOUNDARY_FADE was a second strategy I added, looking to fade markets priced near the boundary — extreme events, thin probability ranges. It seemed clever in theory. The data killed it. BOUNDARY_FADE averaged negative $19.38 per trade. It's disabled now.

Position Sizing: Kelly Criterion

This part matters more than most people think. The original bot used a fractional Kelly Criterion to size each position. Kelly estimates a bankroll fraction from an assumed edge and the odds. Full Kelly is aggressive enough to cause significant drawdowns, so the bot used a fraction of it.

The formula used was f = (edge * odds) / (odds - 1), followed by a 25-50% Kelly fraction based on confidence in the estimated edge. Position sizes were capped so no single trade risked more than a defined percentage of the bankroll. Better sizing could limit damage; it could not rescue a bad probability estimate.

Storm Mode

One detail I was proud of: the bot had a storm mode. Normally it scanned for new markets every 30 minutes. When there was a significant weather event in the forecast — a front moving through, a major precipitation event, an extreme heat warning — it shifted to scanning every 10 minutes. More frequent scanning was sound plumbing, but sound plumbing did not create an edge.

What the Original Paper Numbers Said

After 217 resolved paper trades on a $1,000 simulated bankroll, the original diagnostic ledger reported the numbers below. They are historical paper results, not current performance, investable proof, or revenue.

Breaking it down by strategy:

Strategy	Trades	Profit	Avg/Trade
CALIBRATION	124	$907.79	+$7.32
BOUNDARY_FADE	13	−$251.94	−$19.38

CALIBRATION alone: 124 trades, $907.79 profit, $7.32 average per trade. Average win of $21.17 against an average loss of $8.68. That 2.44x win-to-loss ratio is doing a lot of heavy lifting alongside the 52.7% win rate.

BOUNDARY_FADE: 13 trades, -$251.94. Disabled immediately once I saw those numbers.

Monte Carlo Validation

I ran 10,000 Monte Carlo simulations using the CALIBRATION strategy's win rate, average win, average loss, and a realistic position size range. The results:

• 98.3% of simulations were profitable
• Median simulated profit: $873
• CALIBRATION-only run: 100% of simulations profitable

That was never a guarantee of future results. Monte Carlo assumed the future would look like a noisy historical sample. Later market-relative testing showed that this simulation was not evidence of a durable edge.

At the time, the paper ledger looked consistent. Later testing did not confirm that the apparent edge was real or durable.

The Gate That Wasn't Strict Enough

Paper trading is paper trading. I built a gate system: six criteria that had to pass simultaneously before the bot touched real money. On March 28, 2026, the old gate cleared. Those old gates did not establish a durable edge; the August 27 closure supersedes them.

• ✓ 80 paper trades completed — 217 resolved
• ✓ 55% win rate — 76.5%
• ✓ Brier score ≤ 0.20 — 0.1947
• ✓ Positive P&L — $1,773
• ✓ Peak drawdown < 20% — 10.6%
• ✓ 30 consecutive days of trading — 35 days

Update (March 28, 2026): The bot went live with a $424 bankroll, quarter-Kelly sizing, $15 max per position, and a circuit breaker armed.

Update (April 25, 2026): The bot is paused again. A later gate audit showed candidate flow was partially blocked by bad cluster logic, but after fixing that, the remaining block was real: not enough settled live data, not enough trading days, and final decision quality still worse than the market baseline. That is not a green light. That is the safety system doing its job.

Update (August 27, 2026): The evidence loop is complete. V1 lost money, V2 lost to the market baseline or found zero opportunity density, V3 could not produce a trustworthy leakage-safe holdout, and the V4 crossing replay found zero valid settlement-source crossings or executable transitions. The weather hypotheses are killed, collectors are paused, and live remains off.

What I Got Wrong

BOUNDARY_FADE is the obvious one. I was pattern-matching to a trading concept that sounds good in a book and didn't do the work to validate it before trading it. By the time I had 13 trades of data, it had already cost me $252 in simulated profit. That's actually a cheap lesson in paper trading terms. In live trading it would have been real money.

The other thing I underestimated: how much the model spread matters. Early on I was using point forecasts from NOAA without accounting for forecast uncertainty. A mean temperature of 98°F means something very different when the ensemble spread is 2°F versus 8°F. Later versions accounted for spread and abstained more aggressively, but better uncertainty handling still did not prove an executable edge.

What's Next

For this weather thesis: nothing. No threshold heroics, precipitation expansion, wind expansion, collector restart, or live retry. The code and evidence stay preserved so the lesson remains auditable, but the hypothesis is killed.

The August 27 snapshot recorded a separate MLB cross-stat logical-dominance paper collection planned through September 3, with no account or order path and no claimed edge. That is dated history, not confirmation of completion, success, or current operation. Any newer research is distinct from this closed weather thesis and is not funding proof; WeatherNext remains parked.

The longer game is still the same: build income streams that can fund flight hours. But KDP and digital products are cleaner proof right now than prediction-market bravado. I wrote about the full picture of how I'm using AI to build side income as a pilot, and I keep the current numbers on the Flight Funded tracker.

391.3 hours and counting. The weather bot is not funding them, and pretending otherwise would be worse than killing the project.

---