Tropical Tidbits is the best source of forecasting of uncertainties and risks associated with major storm events. The lesson learned is that an analysis is only as good as the analyst. At Tropical Tidbits, that analyst is Levi Cowan.
“All models are wrong, but some are useful” is a useless quote provided by bad consultants. The truth is that we pay good money for models and the forecasts associated with them, and most are useful if we listen to the right forecasters.
See Levi Cowan’s analysis of Hurricane Laura as it was forming over Cuba on Monday and compare it to what happened on Thursday morning. Skip to time stamp 10:00 if you want to focus on just the models behind the forecast. https://www.tropicaltidbits.com/blog/2020/08/24/laura-a-serious-threat-to-texas-and-louisiana-marco-quickly-decaying/ A couple of lessons to be underscored from this: Model(s) is plural. A single model using point estimates for input parameters is not sufficient for forecasting any type of uncertain future. Two primary approaches are available. The first develop multiple independent models – preferably from different modelers - and find the commonalities of the predictions (generally described as ensemble models). The second is to use a probabilistic method using statistical distributions for input parameters and Monte Carlo simulations to produce a cone of uncertainty for the forecast.
Post-mortems of our forecasts are essential for learning and improvement. Weather forecasting has become quite good over the past few decades because of the focus on after-event analysis or forecasting post-mortems. The same can be said in the aerospace sector and nuclear sector where time and money are spent to understand the difference between what was forecasted and what actually happened.
Every industry has a range of certainty associated with its forecasts. Forecast accuracy is about two days in weather forecasting for the landfall location of major storms (although the forecasts are not as good for the actual wind speed and still worse for rain volume). Much of the accuracy of the landfall location is related to the physical science of the event and the amount of data that the industry collects on which to base its models. There is a direct relationship between forecast quality and the amount of data collected within an industry, at least when it comes to physical sciences.
These are some things to consider the next time you seek advice in a risky situation, build a model yourself, or hire a consultant to forecast an uncertain future for you. An analysis is only as good as the analyst. Levi Cowan is a reminder of that fact every time you see a tropical storm forming.
