Real-Time Verification and Quality Control of Probabilistic Forecasts

Introduction

The National Weather Service (NWS) is responsible for issuing hundreds of routine river and stream forecasts each day, as well as watches and warnings for the protection of life and property. Through AHPS, the NWS is transitioning from issuing deterministic forecasts to probabilistic products based on ensemble forecast processes. AS key element if this transition is verification, which is essential for purposes of (a) administration, (b) scientific advancements, (c) decision making, and (d) forecast generation. The purpose of this project is to develop methods of validating short-term ensemble stage and flow forecasts that can be used for run-time quality control during the operational forecast process. As such, verification methods must be:

1. Practical to implement in the operational forecast environment, where product schedules are fast-paced, and
2. Based on data readily available throughout the national system.

Furthermore, the links between verification measures and quality control guidance must be well defined to facilitate run-time utility of such measures.

Technical Approach

Our approach focuses on measures and interpretations that can be used nationwide to evaluate ensemble forecasts as they are created in the NWS RFC operational environment. Clearly, our approach does not include the development of operational software or of formal software requirements and design specifications. Rather, this task focuses on research to assess verification measures, develop verification procedures, and design displays that enable forecasters, within the constraints of run-time operations, to examine the verification measures and to connect them to specific elements of the ensemble forecast process.

Because our emphasis is on the development of measures, procedures, and displays that can be implemented operationally, it is important that we consider the experience of RFC forecasters throughout the project. Input from OHD and RFC personnel will not be used to constrain any recommended procedures, but rather to ensure that any procedures also include identification of key requirements for feasible operational verification processes. Input from OHD and the RFCs will primarily be by phone and correspondence, but collaborative workshops with several key RFCs will be critical.

Additionally, to enable efficient analysis of verification measures within an operational setting, it is critical that the team at UCI-CHRS have the capability to run the NWSRFS in-house to produce short-term ensemble forecasts and "hind casts". This requires active support from NWS-OHD and/or selected RFC.

To accomplish the goals of this task, our efforts will be divided into four key stages including: (a) data gathering and preliminary inventory, (b) capacity development, (c) analysis, and (d) implementation.