reference-class-forecasting_skill

Overview

This skill helps you anchor forecasts in historical reality by identifying an appropriate reference class and using its base rate as your starting probability. It prevents overconfidence from the "inside view," forces evidence-based adjustments, and gives a clear protocol for multi-stage events and validation of comparisons.

How this skill works

You describe the event you want to forecast and the skill guides you to a correctly scoped reference class (not too broad, not too narrow). It finds or estimates historical frequencies, sets that base rate as the default anchor, and provides tools to test claims of uniqueness, build funnel models for sequential processes, and validate class quality. Adjustments from the base rate require documented, specific evidence proportional to the change.

When to use it

• When starting any forecast and you need a statistical baseline first
• If someone claims "this time is different" and you want to test it
• When predicting success/failure for startups, projects, bills, trials, etc.
• Before detailed inside-view analysis to guard against bias
• When you must combine sequential probabilities (multi-stage funnels)

Best practices

• Always establish the base rate before inside-view arguments
• Choose a reference class that balances specificity with enough data (aim for 20+ cases)
• Use funnel decomposition for events with multiple sequential steps
• Document why your case differs if you move substantially from the base rate
• Validate homogeneity, sample size, and relevance of historical data

Example use cases

• Estimating the probability a seed-stage B2B SaaS startup reaches Series A
• Assessing whether a proposed law will become law using a committee→floor→vote funnel
• Testing a CEO's claim that their project is novel enough to beat typical failure rates
• Building a compound probability model for drug approval across trial phases
• Validating a chosen reference class and getting a confidence level for its anchor