LINK: Intro PageLINK: Dr. Adrian MarroquinLINK: Cycled Snow VariablesLINK: GFE at NWSLINK: Defining Observed FieldsLINK: WRF Model in NWSLINK: Value of Profiler DataLINK: Evaluating ForecastsLINK: BriefsLINK: Recent Publications  
By Jennifer Mahoney, Joan Hart, and Barbara Brown


A variety of convective weather forecasts are produced operationally and used by the aviation community as decision aids for rerouting air traffic around convective weather. These forecasts, which include the National Weather Service (NWS) Convective Collaborative Forecast Product (CCFP) and Convective Significant Meteorological Advisories (C-SIGMETs), describe convective activity at different spatial and temporal scales and differ slightly in the characteristics that are included in the forecast area.

A critical challenge in evaluating the quality of these forecasts is determining how to appropriately match the forecasts to the observations so that statistical results are representative of the forecast characteristics, the forecast spatial and temporal scales, and the forecast's operational relevance. This process has been particularly difficult for evaluating forecasts from the CCFP and C-SIGMETs that are required to meet minimum size thresholds as well as specific criteria for coverage of convection, cloud top height, and cell movement.

Historically, observations used to evaluate the CCFP were expanded from a 4-km grid to a 40-km grid to approximately match the scale of the forecast. Matching the forecast scale was difficult to determine, since the impact of the convective activity on the operational flow of enroute air traffic was not well defined. Moreover, the coverage attribute was excluded from the verification approach because the application of the attribute was not clearly understood.

This article presents new methods for defining the observation fields used for evaluating the CCFP and C-SIGMET forecasts that consider the effects of convection on the flow of air traffic such as Convective Constraint Areas (CCAs) and incorporate the observed coverage.

Data: Forecasts and Observations

Forecasts – The CCFP forecasts are issued by the NWS Aviation Weather Center (AWC), but are produced through a collaborative process with AWC forecasters, airline and Center Weather Service Unit (CWSU) meteorologists, and MSC (Meteorological Service of Canada) meteorologists. CCFP forecasts are required for areas of intense convection and thunderstorms every 2 hours, with lead times of 2, 4, and 6 hours after the forecast delivery time. The CCFP comprises polygons that are at least 3,000 mi2 in size and contain a coverage of at least 25% convection with echoes of at least 40 dBZ composite reflectivity, and also a coverage of at least 25% with echo tops of 25,000 feet and greater.

The C-SIGMET, generated by forecasters at the AWC, is a text forecast of convective activity that is issued hourly but valid for up to 2 hours (as outlined in the National Weather Service Operations Manual D-22). These forecasts are intended to capture severe or embedded thunderstorms and their hazards (e.g., hail, high winds) that are either occurring or forecasted to occur within 30 minutes of the valid period and cover at least 40% of the 3,000 mi2 or larger forecast area.

Observations The National Convective Weather Forecast Hazard Product (NCWF-H) is used to describe intense convection as it applies to the CCFP that is a threat to aircraft. It is defined by the video integration and processor (which contours radar reflectivity, in dBZ, into 6 VIP levels) values of 3 or greater, and/or 3 or more strokes of lightning in 10 minutes within 8 kilometers of a grid point, on a 4-km grid. For further information see http://cdm.aviation

Techniques for Defining Observations

The techniques for defining the observations for evaluating the CCFP and the C-SIGMET are separated into two parts: 1) developing a definition for Convective Constrained Areas (CCA) and 2) producing observed fields that reflect the attributes of the CCFP, particularly the size and coverage criteria.

Convective Constraint Area (CCA) – This provides the basis for measuring the "scale" of convective activity that impacts the flow of enroute air traffic. Rhoda et al. (2002) determined that pilots tend to deviate around strong precipitation until they get quite close to the arrival airport. However, they were unable to determine the typical distance of the deviations. Therefore, the CCA concept applied here follows guidance provided by the Aeronautical Information Manual (AIM),, which suggests that pilots should remain at least 20 nm away from intense convection in order to minimize safety concerns that are related to convection. However, in practice, this distance is often too large when air space becomes congested. Therefore, to take this operational consideration into account, we defined the CCA here as an area of intense convection (identified by the 4-km NCWF-H grid) plus a 10-nm radius surrounding the convection. The 10-nm radius is measured from the center of each 4-km NCWF-H grid box. Figure 1 shows the raw NCWF-H in which the green areas represent the grid boxes with intense convection. Once the 10-nm radius criterion is applied to the observations in Figure 1, the areas grow slightly (Figure 2) to represent the CCAs. The CCAs in Figure 2 should not be thought of as areas "closed" to enroute air traffic. Rather, they should be considered as areas where the flow of enroute air traffic is reduced because of the influences produced by the intense convection.


Figure 1. Raw NCWF Hazard Product at 4-km resolution at 1900 UTC on 4 July 2003. Green areas indicate VIP values that are 3 and greater and cloud tops are assumed to be 20,000 feet and greater.



Figure 2. Map of convective activity that impacts enroute air traffic at 1900 UTC on 4 July 2003. Green areas indicate 4-km NCWF Hazard +10-nm radius.

Using the CCA as the area of interest, coverage is computed by evaluating the percentage of 4-km CCA boxes meeting the CCA criterion within a larger 92 x 92 km search box. This search box represents the 3,000 mi2 minimum size required before a CCFP or C-SIGMET forecast polygon can be issued. The percent of observed coverage within the search box is assigned to the center 4-km box. The search box is moved one grid square and the coverage is recomputed and assigned to the center 4-km box. This procedure continues until each 4-km box within the forecast domain is assigned an observed coverage value. Figure 3 shows the CCA coverage for the example shown in Figure 1. Increasing coverage represents a decrease in the flow of air traffic, though exactly how much of a decrease is yet to be determined and will be the focus of future research.


Figure 3. Map of convective constraint areas with coverage 3,000 mi2 area that is 25–49% (light green), 50–74% (medium green) and 75% and greater (dark green), at 1900 UTC on 4 July 2003. Yellow areas indicate CCFP forecast.


The application of the technique for two convective cases: a well-organized convective line on 8 June 2003 (Figure 4) and disorganized isolated convection on 5 August 2003 (Figure 5). The observed fields shown in these figures pictorially represent the "perfect" forecast, where the sizes of the fields are greater than 3,000 mi2 and the areas contain a coverage that is greater than the minimum threshold for the CCFP, 25% (Figures 4a, 5a) and the C-SIGMETs, 40% (Figures 4b, 5b).



Figure 4. Organized convective line, 8 June 2003 2-hour forecasts for CCFP (a, left) and C-SIGMET(right) issued 1900 UTC. Observed CCAs, coverage 25% (a) and 40% (b). Forecasts are indicated by hatched areas.



Figures 5. Disorganized convection on 5 August 2003, 2-hour forecasts from the CCFP (a, left) and C-SIGMETs (right) issued at 2300 UTC. Observed CCAs, coverage 25% (a) and 40% (b). Forecasts are indicated by yellow areas.


For the 8 June 2003 case (Figure 4), the forecasts nicely capture the main convective line over the Midwest and large convective area over the Southeast. Convection over the West and Southwest was left out of both forecasts, possibly because the impact on rerouting aircraft due to convection is generally less of a problem over the West than over the eastern half of the U.S.

In the 5 August 2003 case (Figure 5), the larger convective areas over the Northeast, Atlantic States, lower middle half of the U.S., and the upper Northwest were accurately captured by both the CCFP and the C-SIGMETs. However, the smaller convective areas were excluded from both forecasts. These results may suggest that the CCFP and the C-SIGMET forecasts are focused on main areas of convection that are typically much larger than 3,000 mi2 and that the area requirement for the minimum forecast area should be reconsidered.

Conclusions and Future Work

Defining the observed fields for verifying spatial forecasts for convection is key to developing approaches that meet the forecast and user requirements. Here, we build a definition for a Convective Constraint Area (CCA) that is consistent with operational guidelines and is used to characterize the airspace around intense convective weather where the flow of enroute air traffic may be obstructed or reduced. The CCA forms the basis for developing the coverage fields that are used to evaluate the quality of – and characterize the weather requirements for – the CCFP and the C-SIGMETs. Input from the user community is necessary to ensure that the size criterion of 10 nm is operationally relevant. In addition, cloud-top heights need to be added to the CCA techniques presented here to fully incorporate the CCFP weather attributes into the verification approach. Finally, the relationship between the observed coverage and the reduction in the flow of air traffic will be the focus of future research. Defining the observations in this manner sets the stage for the application of object-oriented verification approaches.

Editor's Note: A complete list of references and more information on this and related topics are available at the main FSL Website, by clicking on "Publications" and "Research Articles."

(Jennifer Mahoney is Chief of the Forecast Verification Branch in the Aviation Division. She can be reached at, or 303-497-6514.)