By Brent Shaw, Mike Kay, Jennifer Mahoney, John McGinley, and John Smart
Under the auspices of a nationwide effort led by NOAA known as the Coastal Storms Initiative (CSI), a locally run version of the new Weather Research and Forecast (WRF) mesoscale numerical weather prediction model has been installed at the Jacksonville (JAX), Florida, National Weather Service Weather Forecast Office (WFO). CSI is a collaborative effort of various local, state, and federal organizations to lessen the impacts of storms on coastal communities. The effort to install WRF at the Jacksonville WFO is but one component of the initiative, designed to improve accuracy and detail of forecasts of coastal winds, precipitation, and visibility. This local modeling effort represents collaboration between the NWS Office of Science and Technology, the Jacksonville WFO, FSL, and the Florida State University (FSU) Department of Meteorology.
Three pertinent issues related to local modeling within the NWS WFO environment are addressed. First, can public forecast services provided by a WFO be enhanced through the use of a locally run mesoscale modeling system? Second, does the use of a data assimilation component improve local model forecasts compared to simply initializing a local model directly from the NCEP national forecast models? Third, can the new WRF model serve as the local model component in the WFO environment in a similar manner as the workstation Eta system has in other WFOs?
To address these questions, the group of collaborators designed a local configuration that would meet the operational needs while providing data and a verification method that could provide insight into these issues. In this article we provide an overview of the CSI WRF modeling system as installed at the Jacksonville WFO, including the data assimilation component, postprocessing, and limited quantitative results. Information on the perspective of the operational forecasters regarding value added by this system is contained in Welsh et al. (2004). A verification study of the operational WRF forecasts is provided in Bogenschutz et al. (2004) and in Mahoney et al. (2003).
WRF Configuration - Version 1.3 of WRF in use at the Jacksonville WFO is available to the general community for download at http://www.wrf-model.org. The dynamic core used for the CSI system is the third-order Runge-Kutta solver (Wicker and Skamarock 2002) formulated for the mass-based vertical coordinate. No explicit numerical filters are used during model integration (diffusion constants are set to zero).
The horizontal model domain (Figure 1) grid uses a Lambert Conformal map projection with grid spacing of 5 kilometers. This was chosen to match the resolution of the grids used to populate the National Digital Forecast Database (NDFD) via the Interactive Forecast Preparation System (IFPS; see the LeFebvre article Using GFE in NWS Forecast Operations). The analysis grid consists of 145 points in each direction. Use of an Arakawa-C stagger in the WRF results in 144 mass points in each direction, allowing an equal number of points in the grid to be distributed across the 16 processors available on the computational platform. The Runge-Kutta solver allows a long time step of 30 seconds to be used despite the 5-km grid spacing.
Figure 1. Horizontal model domain for the CSI JAX WRF simulations. The domain consists of 145x145 points on a 5-km non-staggered grid. Image is USGS 24-category land use class as provided via the WRF Standard Initialization package.
In the vertical, 42 full levels (41 computational layers for the mass variables) are used, with a minimum vertical increment of approximately 20 meters at the lowest levels, increasing to approximately 1000 meters at the model top, which is set at 100 millibars.
Physics options employed include the NCEP 5-class version 1.3 of WRF microphysics, Dudhia shortwave radiation, RRTM longwave radiation, the MRF (Hong-Pan) PBL scheme, and the OSU Land Surface Model. No cumulus parameterization is employed.
The model initial and lateral boundary conditions are provided via the WRF Standard Initialization (WRFSI) package, version 1.3.2, also available to the public at the above Website. The WRFSI is configured to read analysis grids from the Local Analysis and Prediction System (LAPS) for the initial atmospheric conditions, and NCEP Eta grids on the 12-km NCEP grid 218 for the initial soil and sea conditions, and for the lateral boundary conditions.
Data Assimilation – For the WRF simulations used
by the operational forecasters, the initial conditions
are provided by LAPS, using the diabatic
initialization technique (Shaw et al. 2001). LAPS is able to use a
wide variety of observational data, including GOES
imagery, GOES soundings, WSR-88D reflectivity and
radial velocity data, wind profilers, RASS temperature
profiles, METAR and maritime surface observations,
mesonet observations, GPS-Met total precipitable water,
and ACARS data. In the Jacksonville WFO
implementation, the availability on the NOAAPORT data feed and
what is available via the Local Data Acquisition and
Distribution (LDAD) feed. For now, this consists primarily of
surface observations, satellite imagery, and single-level
radar reflectivity data. For the surface observations, there
are typically between 100 and 150 surface
observations available from the combination of METAR
reports, mesonet sites, and marine observations. As
communication and LDAD issues are addressed, we
anticipate additional datasets becoming readily available, such
as ACARS, GPSMet, and wideband WSR-88D
radial velocity and full-volume reflectivity.
The unique diabatic initialization relies on the LAPS three-dimensional cloud analysis, which includes a cloud model to partition the condensate into the various species and determine cloud-type information. Using the cloud-type information, a vertical motion profile is derived, and these profiles are used as "observations" in a final three-dimensional variational (3DVAR) adjustment to ensure that the mass and momentum fields are in balance with the analyzed cloud field. The 3DVAR balance step is fully described in McGinley and Smart (2001), and the details of the cloud analysis and vertical profile assignment are discussed in Schultz and Albers (2001). Note that LAPS is under continuous development, and the version described here is much newer than the versions currently fielded within the AWIPS platform. The forecasters at Jacksonville have reconfigured the standard AWIPS version to match the CSI domain, and they are able to view the analyses from the standard AWIPS LAPS as well as the advanced CSI LAPS.
Hardware Platform and WFO Integration – The computer used for LAPS and WRF is a Linux
cluster consisting of 9 nodes. Each node contains dual
Athlon 2-GHz processors, and interprocessor
communication is handled via the Gigabit Ethernet interface.
LAPS and all model pre- and postprocessing are
performed on the master node. The WRF model runs on
16 processors, spanning the remaining 8 nodes, using
the MPI version of the model. The model
configuration described earlier is able to complete a 24-hour
forecast in approximately 2.5 hours.
The cluster interacts with the Advanced Weather Interactive Processing System (AWIPS) via the LDAD system. LDAD is used to transfer observational data and national model grids to the cluster for ingest by LAPS. Output from LAPS and WRF is transferred back to AWIPS via the same LDAD exchange mechanism. The WRF model is post-processed incrementally by a model postprocessor that is provided with LAPS. The output is destaggered onto the analysis grid, vertically interpolated to isobaric levels, and written into GRIB files that are sent to AWIPS as the model is running. Thus, forecasters are able to view each output hour of the forecast as they are produced rather than waiting for the entire simulation to be completed. The postprocessor also provides tabular text forecasts for a list of points specified by the Jacksonville WFO. By sending the grids to AWIPS, forecasters are able to use and evaluate the forecasts on their operational workstations, which allow overlay of other data such as observations, satellite and radar imagery, and other grids. Additionally, sending the data to AWIPS provides the opportunity to import the WRF grids into the Interactive Forecast Preparation System.
Experiment Design and Verification – The computing capacity at the WFO and the model configuration allow for multiple model cycles per day. The schedule of runs was configured to best meet the needs of the local forecast operations while providing meaningful data to study the impact of adding a local data assimilation component and/or the value of local modeling compared to the national products. All simulations discussed below are run out to a 24-hour output increment (i.e., 25 frames per forecast cycle).
Each day, two cycles with a 0600 UTC initial time
are produced. Both simulations are identical in
every aspect except for the initial conditions. Both
simulations utilize the 0000 UTC NCEP Eta on a 12-km
Lambert Conformal grid for lower and lateral
boundary conditions. The "operational" run (hereafter
referred to as WRF-LAPS) is initialized with LAPS and
is started at 0645 UTC and completed by 0815 UTC. The second "comparison" simulation (hereafter
referred to as WRF-Eta) begins when the operational run
is complete, and uses the 6-hour forecast from the
0000 UTC Eta as the initial condition instead of LAPS.
Since the first-guess used for LAPS in the operational run is also the 6-hour forecast from the 0000 UTC Eta, these two runs serve the purpose of determining the value of adding additional local data and performing a reanalysis in the context of the LAPS diabatic initialization. Furthermore, since they have a 0600 UTC initial time, they can be directly compared to the 0600 UTC Eta run from NCEP to see what if any value is added by local models compared to the national guidance.
In addition to the 0600 UTC runs, two more WRF-LAPS runs are performed each day at 1500 UTC and 2100 UTC to meet the needs of the Jacksonville WFO. These two runs provide updated, high-resolution model output between the national Eta and GFS runs using the LAPS diabatic initialization. The 1200 UTC and 1800 UTC runs of the operational NCEP Eta model provide the
boundary conditions for these runs.
For all four runs each day, a subset of the postprocessed model output in GRIB is transferred back to FSL for processing through the Real-Time Verification System (RTVS, Mahoney et al. 2002). RTVS verifies the forecasts of surface temperature, humidity, wind speed and direction, and precipitation against surface observations within the domain. Standard statistical measures of accuracy for continuous variables such as bias and root mean square error are provided for temperature, humidity, and wind parameters.
In addition to the CSI WRF runs, the 12-km national Eta model is also processed through RTVS using the same algorithms and observations. When comparing more than one model run, RTVS provides "equalization" to ensure that only those model cycles for which all models being compared were available are used in the statistics. Finally, RTVS provides a Web-based interface to view the results of the verification interactively at http://www-ad.fsl.noaa.gov/fvb/rtvs/csi/. A detailed analysis of the RTVS results is contained in Mahoney et al. (2003).
In addition to the quantitative validation being provided via RTVS, the GRIB data retrieved by FSL is also provided to Florida State University, where detailed case studies and mesoscale feature-based assessments are being performed (Bogenschutz et al. 2004).
Successes – The most important measure of
success when testing a new application in a WFO environment
is whether or not the forecasters find the application useful. Making the WRF grid available on AWIPS
provided the incentive for the forecasters to look at the
model forecasts, and over time, more and more of the
forecasters have become comfortable with the WRF model and
have begun to rely on it in various situations. In particular,
early in the experiment, forecasters discovered that
WRF forecasts of visibility reductions due to fog were
very accurate. One of the first area forecast
discussions issued by the Jacksonville WFO referencing the
WRF actually indicated a change in their thinking for
the visibility forecast based solely on the WRF forecast
and its previous performance in similar situations.
The surface winds forecast parameter is also important within the Jacksonville WFO area of responsibility. The CSI project specifically calls for improved forecasts of wind speed and direction for input into a new wave model being developed under CSI.
Quantitative verification of the WRF wind speed
forecasts using the RTVS show that the WRF
forecasts significantly outperformed the NCEP Eta forecasts at all hours of the 24-hour
forecast period. Figure 2 shows the root mean
square error (RMSE) and bias of the surface wind forecasts for the 0600 UTC run of Eta,
WRF-LAPS, and WRF-Eta.
Figure 2. RMSE and bias for all 0600 UTC Eta, WRF-Eta, and WRF-LAPS forecasts of wind speed.
The southeast U.S. and adjacent coastal areas are dominated by convective activity during much of the year. Quantitative precipitation forecasts (QPFs) via numerical methods are traditionally poor for these types of events due to lack of model resolution and the inherent chaotic nature of air mass thunderstorm development and evolution. The LAPS diabatic initialization attempts to improve explicit short-range QPFs by initializing the NWP models with active clouds and precipitation. This experiment provides further evidence that finer scale models coupled with advanced techniques using satellite and radar information can provide improvements. Figure 3 depicts the ESS and frequency bias scores for the 0–6 hour QPF for various thresholds of precipitation from the 0600 UTC run of the NCEP Eta, the WRF-Eta, and the WRF-LAPS.
Figure 3. ESS and bias for the 0–6 hour forecast period from the NCEP Eta, CSI WRF-Eta, and CSI WRF-LAPS. Statistics are from RTVS for 1 June–19 October 2003.
The WRF-LAPS demonstrates better ESS and a more consistent bias across all thresholds of precipitation than either the NCEP Eta or the WRF-Eta run. This figure also shows the benefit of adding local data to the initialization using the LAPS diabatic method, as the WRF-Eta forecasts had a low bias for all thresholds, indicative of the typical model "spin-up" problem for precipitation processes. The Eta suffers much less from the spin-up problem, likely due to its advanced 3DVAR data assimilation cycle, but is still outperformed in the 0-6 hour forecast period by WRF-LAPS.
Challenges – Several challenges presented
themselves during this project. First and foremost, network
security requirements and lack of bandwidth between
the Jacksonville WFO and the rest of the NWS
network made it difficult to engineer an optimum solution
to ensure timely availability of all required input data.
The first-guess grids and observational data, including
radar and satellite, are made available via the LDAD
system, whereas the Eta tiles for the lateral and lower
boundary conditions are obtained via FTP from either the
NWS Southern Region Headquarters in Fort Worth, Texas,
or from the NCEP anonymous FTP server. Many of the
run failures during the experiment were due to slow
or incomplete data transfers, either due to
network performance or unanticipated impacts when
router configurations were changed while applying
Network bandwidth available to a WFO varies by location, and JAX happens to be more limited than most in the Southern Region. Plans for the LAPS analysis included the acquisition of multiple wideband WSR-88D radar feeds from within the region by making use of
the CRAFT network. Unfortunately, at the time of writing, this was still not possible. To mitigate
this, FSL has been performing the radar data ingest
at FSL using narrowband radar from multiple sites and transferring the LAPS intermediate file to
the JAX cluster. It is expected that the LAPS
diabatic initialization will benefit greatly from
multiple wideband radar sites as demonstrated in
the International H2O Project (Shaw et al. 2004).
Planned upgrades to AWIPS during the experiment provided additional challenges, as various changes and additions made to allow ingest of the local model, as well as custom scripts to provide data to the cluster via LDAD, were overwritten during the upgrades and had to be recovered. Support for custom configurations on AWIPS will likely improve as local modeling within the WFOs becomes more prevalent.
Difficulties arose in the initial integration of the Linux cluster because of how it was configured by the vendor, which was more suited to "high availability" computing rather than "high performance" computing. This required some time to learn and reconfigure the system for use with parallelized software. Lessons learned from the CSI project are valuable in preventing this in future WFO offices. Additionally, minor hardware failures, including a failed network card and a failed main processor, were responsible for a few mode failures during the project. These were generally discovered and repaired quickly by the Jacksonville WFO Information Technology Officer.
Meteorologically, the WRF forecasts did not perform as well as the Eta model and other national guidance for surface temperature (Figure 4). Both the WRF-LAPS and WRF-Eta runs exhibit a negative temperature bias (too cool) during the afternoon hours (at peak heating) and a positive temperature bias (too warm) at night. This is fairly typical for many models, including the Eta, but was much more exaggerated for the WRF forecasts.
Figure 4. RMSE and bias for the Eta, WRF-Eta, and WRF-LAPS surface temperature forecasts for all forecast hours of the 06 UTC runs from 1 June through 13 October 2003.
However, it is important to remember that the Eta model and its associated postprocessed fields (e.g., 2-m temperature) has undergone extensive tuning since its implementation several years ago, whereas the WRF model is new and was used in an "off-the-shelf" configuration. Officially, WRF is not yet considered to even be a research-grade model. Despite the deficiencies in forecasting surface temperatures, its performance in other categories is still quite encouraging given the state of its development. The problems with the temperature forecasts warrant some investigation into the implementation of the PBL, land surface, and radiation schemes and their interactions within the WRF model.
Conclusions and Future Work
Despite the challenges faced in implementing the WRF model as a quasi-operational tool within the Weather Forecast Office, this project has made progress in answering the questions posed in the Introduction. The quantitative statistics and anecdotal evidence show that local models can and do add value in
the local forecast process, particularly in the area of
QPF and wind forecasts. For short-term forecasts (0–6
hours), initializing these models using additional local data
appears to provide even more value. For longer
term forecasts, lateral boundary conditions tend to
dominate the source of forecast error for such small domains
as the Coastal Storms Initiative area, but in some
cases (e.g., wind speed), the additional resolution of the
model appears to still provide advantages.
Finally, even though it is in the early stages of development, the performance of the WRF model is very encouraging. Groundwork laid by the CSI project may serve as a foundation for developing a standardized WRF-based local numerical weather prediction package suitable for use in all NWS WFOs. We hope to continue upgrades to the Jacksonville WFO system, including the addition of the wideband WSR-88D reflectivity and radial velocity data from Jacksonville and surrounding offices, GPS total precipitable water retrievals, ACARS data, and local wind profilers, all of which are currently supported by the version of LAPS being used but are unavailable to the cluster at the time of writing. A second evaluation period during the winter may also provide useful verification data to assess WRF performance in a different weather regime.
Acknowledgments - Special thanks to the following
collaborators in this project: P. Welsch and A. Wildman,
NOAA NWS Weather Forecast Office, Jacksonville, FL.; J.
Savadel, NOAA NWS Office of Science and Technology, Silver
Spring, MD; and P. Bogenschutz and P. Ruscher, Florida
State University, Department of Meteorology, Tallahassee, FL.
Editor's Note: A complete list of references and more information on this and related topics are available at the main FSL Website www.fsl.noaa.gov, by clicking on "Publications" and "Research Articles."
(Brent Shaw is a researcher in the Local Analysis and Prediction Branch headed by Dr. John McGinley. He can be reached at Brent.Shaw@noaa.gov, or 303-497-6100.)