EXPERIMENT 02 / MODEL RELIABILITY
Can we trust the forecast?
A confidence level is a promise. Test how well prediction intervals keep it when the data changes.
Start with the question
What happens when the future stops looking like the past?
Follow three changes, then test your own assumptions.
Explore the settingsChapter 1 of 3
Learn under stable conditions
An interval gives a range of plausible next returns. In stable conditions, yesterday’s errors can help calibrate tomorrow’s uncertainty.
Compare achieved coverage with the 95% target. A target is not a promise that every finite sample will achieve it.
What stays fixed?
Synthetic heavy-tailed returns, 95% target coverage, a 120-observation calibration window, and seed 42. Each prediction uses only earlier observations.
The terms, in plain language
- Coverage
- The percentage of observed returns inside the predicted interval.
- Interval width
- The distance between the upper and lower bounds. More coverage can come at the cost of less precision.
- Regime shift
- A change in the behavior of the process generating observations.
Loading reproducible evidence…
Your turn
Explore the settings.
Change one assumption at a time. Run a calculation to update the evidence, then inspect or download the result.