bayesian-reasoning-calibration_skill

Overview

This skill helps you update probability estimates explicitly using Bayesian reasoning so decisions under uncertainty are grounded and auditable. It guides you to set priors, assess evidence likelihoods, compute posteriors, and calibrate confidence. Use it to avoid overconfidence and to show how new data changes beliefs.

How this skill works

The skill walks through a five-step workflow: define a clear hypothesis and timeframe, pick an evidence-based prior, estimate P(E|H) and P(E|¬H), compute the posterior via Bayes' theorem or odds and likelihood ratio, then document and calibrate results. It highlights diagnostic power of evidence (likelihood ratios), supports sensitivity checks, and enforces explicit assumptions so forecasts remain testable and improvable.

When to use it

• Making forecasts or probabilistic predictions under uncertainty
• Updating beliefs when new data or observations arrive
• Calibrating confidence for decisions or forecasts
• Testing hypotheses with imperfect or noisy evidence
• Assessing risk or diagnostic results where base rates matter
• Avoiding anchoring and overconfidence in judgment calls

Best practices

• Start with reference-class base rates and justify any deviation
• State priors before inspecting new evidence to prevent hindsight bias
• Estimate likelihoods explicitly and explain assumptions for P(E|H) and P(E|¬H)
• Use likelihood ratios to gauge evidence strength and run sensitivity analyses
• Record the full update chain and revisit forecasts as more data arrives

Example use cases

• Forecasting product adoption and updating launch decisions after beta results
• Interpreting diagnostic test results by combining base rates with test accuracy
• Assessing the probability of project completion dates as progress updates arrive
• Comparing competing hypotheses about a root cause using likelihood ratios
• Calibrating a team’s probability estimates by tracking past forecasts and recalibrating priors