Stochastic Layer

A scoreboard for every maybe — so "I'm 70% sure" means something, and everyone's track record is computed instead of remembered.

A forecast ledger with named forecasters, strict grading, and calibration scoring, in one stdlib-only Python file. Companion to semantic-layer — that one governs facts; this one governs maybes. Built and battle-tested inside the 19Keys empire; take the code, build your own.

The problem, in plain terms

Think about how people grade their own guesses: by memory. And memory cheats. It keeps the hits and quietly drops the misses. That's how someone honestly believes they're "right 95% of the time" while the receipts say 25%. Nobody lied. There was just no notebook keeping score.

Now think about a good weather forecaster. They're not right every time — nobody is. They're something better: when they say 70% chance of rain, it rains about 70% of those times. Their number means something. That's called calibration, and it's the difference between a forecaster and a fortune-teller.

The fix, in four moves

Every maybe goes in the book BEFORE the answer. Exact claim, a deadline, a number for how sure, and the name of who said it. No date, no number, no name — it doesn't count.
Changing your mind gets rewarded, not hidden. New evidence arrives, you update your number — and admitting the odds dropped is counted as a virtue. The book literally tracks downward updates as a score.
Reality grades strictly, and misses stay forever. When the deadline comes: TRUE, FALSE, or PARTIAL — against exactly what you wrote, not what you wish you'd written. Erasing a miss breaks the scoreboard, so nothing gets erased.
The score computes itself. The book tells each person — and each AI agent — their honesty score, and answers the real question: of the times you said 70%, how often did it actually happen?

Everyone on the team — humans and AI helpers alike — carries their own scoreboard, on the same strict rules. When anyone claims a track record, you don't argue. You ask the book.

Quickstart

# put `fcast` on your PATH (one shared ledger for the whole team)
mkdir -p ~/.local/bin
printf '#!/bin/sh\nexec python3 %s/scripts/forecast.py --state "$HOME/.claude/claimgraph/forecasts.jsonl" "$@"\n' "$(pwd)" > ~/.local/bin/fcast
chmod +x ~/.local/bin/fcast

# log a maybe — before the answer, with a number and a name
fcast add "We hit 1000 signups by Sep 1" -p 70 --by 2026-09-01 --who alex

# evidence arrived; the odds dropped; say so (this is the honorable move)
fcast update F0001 -p 55 --note "pace slowed after week 2"

# reality reports; grade it strictly
fcast resolve F0001 -o FALSE --note "reached 640 by the date"

# the scoreboard — computed, never self-reported
fcast score

The one-liner

We used to grade our guesses by memory — hits remembered, misses forgotten. Now every maybe goes in the book before the answer, with a number and a name on it. Reality grades it strictly, misses stay forever, and the book keeps everyone's honesty score.

License