Systems Development Division

A major activity within the division is supporting the National Weather Service (NWS) AWIPS development. This division works closely with the Modernization Division to ensure that the appropriate expertise is available for all new AWIPS development and system maintenance.

The division comprises two branches and one science advisory group:

Advanced Display Systems Branch – Designs and develops interactive display systems for operational use and prototype systems for operational demonstration.

Data Acquisition and Dissemination Systems Branch – Utilizes the latest software and hardware technology to develop local data acquisition and advanced weather dissemination for modernized Weather Forecast Offices. Dissemination decision support systems offer easy-to-interpret information of rapidly evolving weather events for emergency preparedness agencies that typically lack meteorological expertise.

Scientific Applications Group – Develops and implements scientific software systems designed to improve weather forecasting by taking advantage of opportunities offered by recent advances in meteorological observations and information systems.

The main focus in Fiscal Year 2000 was work on the meteorological display (D2D) and text components of the AWIPS Weather Forecast Office (WFO) system for the NWS. Build 4.2 developments allowed NWS to decommission the 20-year-old legacy communications and display system. Emphasis then turned to streamlining and enhancing user services, including system configuration and customization. Along with Modernization Division staff, branch personnel supported the fielding of AWIPS build 4.3. This phase included enhancements to local data acquisition and processing, which provide an easier way to interrogate dial-up gauges.

In the second half of the fiscal year, attention turned toward developing and testing build 5.0 capabilities for the winter 2000/2001 release, and designing and developing build 5.1.1, targeted for mid-2001. A key feature of 5.0 is augmentation of the text interface, which makes data retrieval possible using World Meteorological Organization (WMO) standard product identifiers, in addition to the legacy identifiers used by NWS for the last 20 years. This allows Alaska and Pacific Region NWS offices to make more effective use of the AWIPS text capabilities. FSL completed other features in build 5.0, including:

• More efficient use of high-resolution radar data.
• A change in the way point data are stored and accessed, providing a means for sites to customize and augment display formats.
• A graphical interface that enables WFO sites to move their LAPS analysis domain.
• An expanded suite of hurricane monitoring displays.
• Improvements in MSAS analyses and display; numerous additional diagnostic fields include significantly more quality control information.
• Radar functions mimicking WSR-88D Principal User Position (PUP) capabilities, working toward eventual decommissioning of the PUP.

In collaboration with NWS and the AWIPS contractor, Litton PRC Inc., the branch continued efforts to transfer system-level knowledge of D2D (display in two dimensions) to enhance AWIPS maintenance and field support.

Product development continued in support of operations at the NWS Aviation Forecast Center and on a D3D system for field office use. Internal support includes systems administration, testing, and configuration management.

The branch worked with other FSL staff in the coordination of collecting, transporting, and setting up NOAA exhibits at the American Meteorological Society Annual Meeting.

Other FX-Advanced Projects

Linux – For many years, FSL has been actively pursuing and developing meteorological workstation capabilities for a low-cost computer platform. A Unix PC workstation developed over ten years ago is still being used, with some moderate enhancements, by the Central Weather Bureau in Taiwan to support their daily forecast operations. Factors such as the development of FX-Advanced, the maturing of Linux, the availability of a good GNU C++ compiler, and the increasingly high performance of personal computers have motivated FSL to pursue a Linux workstation development. Linux has become a viable alternative to expensive Unix operating systems for software development and networking, and as an end-user platform. Our long-term goal is to develop an FX-Linux workstation system that performs all acquisition and processing functions of the current FX-Advanced and incorporates architectural improvements to accommodate changes in technology and user requirements.

The first phase of this ambitious multiyear effort involved porting existing FX-Advanced software to a PC/Linux platform. This entailed modifying and standardizing the code so that it could be compiled with the GNU compilers, changing Unix scripts, and modifying software so that it can accept the little-endian byte order of the PC. A significant task was implementation of changes required to properly display information on a true-color graphics display for a PC. A nearly complete display system has been in use at FSL since early 2000, with plans to place such a demonstration system in the Boulder WFO early in 2001.

Other FX-Advanced components that have been developed for or ported to a Linux environment are the LDAD system and D3D (display in three dimensions). Concurrent with efforts to port much of the existing software are studies to redesign major portions of the FX-Advanced software to optimize it for Linux, and to incorporate new requirements and technology.

D3D – The primary objective last year was to verify the implementation of D3D on Linux and to obtain feedback from operational forecasters on the utility of three-dimensional data displays for operational forecasting. An exercise was conducted with participants from the NWS Regions, Headquarters, and National Centers. The exercise was divided into two phases to accommodate the schedules of some participants. Each phase consisted of a training session to familiarize users with the basic D3D user interface and capabilities, and an exercise to give users an opportunity to inspect real-time and selected data cases. Since the Linux version of D3D had not been fully tested, HP workstations were used for the duration of the exercise. Subsequently, the D3D/Linux code was extensively tested at FSL, and also used frequently in demonstrations and weather briefings. The performance of D3D on a PC has been impressive, even without the use of special graphics accelerators to manipulate three-dimensional displays. The latest version of the Vis5D software released by the University of Wisconsin has been received and is being integrated with D3D. It supports multiple data and display contexts, features often requested by users.

FX-Connect – The goal of the FX-Connect (FXC) project is to explore advanced system architectures that support collaboration among users, distributed processing, and distributed databases. Plans are to address some of the operational issues associated with each of these system capabilities. Interactive collaboration has only recently become practical, with the availability of higher communications bandwidths on the Internet and other communications networks. Whiteboards, chat rooms, and video cameras are now used regularly by many different groups. FXC provides the capability for multiple users to view real-time meteorological data, manipulate the information and displays, share local data, and create manual graphical products in a collaborative session. A chat room capability (to be supplemented by telephony) is available to exchange messages between collaborators. Since meteorological displays are exchanged as metafiles, the system supports such features as progressive disclosure of information as the display is magnified and color changes for images and graphics. FXC can be used as an independent workstation or in a collaborative mode, in which all requests and display interactions are shared with every user participating in a session. Figure 41a illustrates the different types of data sources available to an FXC user during collaboration; Figure 41b illustrates a typical FXC display composed of a satellite image, model graphics, and manual graphics annotation. Any one of a number of AWIPS databases can be used as the primary source of data for FXC. However, only one database can be used at any one time to create a display. AWIPS data can be supplemented with images from a Web server, manually generated graphics, or graphics created from local datasets.

The FXC code has been significantly redesigned based on experiences and lessons learned from the initial design and implementation. The new code makes much greater use of processing threads so that several functions can be performed concurrently (from a user perspective) instead of sequentially. Also, the capability to access and display text products (such as watches, warnings, and forecasts) was added.

Continuing support will be provided to the National Weather Service during the fielding of AWIPS build 5.0, testing and deployment of builds 5.1.1 and 5.1.2, and development work on build 5.2.1. Key tasks of these builds will include improvements in radar mosaic generation and display, several features to make data access and display more efficient for users, a proximity alarm feature for text products, software enhancements to support the NOAA National Centers, addition of model and satellite soundings, and more precipitation guidance from River Forecast Centers. System performance issues will continue to be addressed.

Other FX-Advanced Projects

Linux – FSL has begun a major effort to prototype a transition of AWIPS to Linux machines. Work during the coming year will include tests of new high-speed network connections and communications, and development of a Linux file server. The branch will continue to support the NWS Linux demonstration, and test various configurations of single and dual-headed workstations.

D3D – The D3D system will be installed on PC/Linux machines in several operational forecast offices. A basic infrastructure will be defined for supporting these users and for receiving their comments on all aspects of the system. Additional training will be scheduled for users who missed any training sessions during the exercises. The system will be evaluated, and enhancements will be made based on user feedback. The interactive Skew-T will be redesigned to include a hodograph and allow computation of additional parameters. New Vis5D features will be added to D3D that will enable users to display data in several windows, or display several forecast models in a single window.

FX-Connect – The name of this project will be changed to FX-Collaborate because of the increased emphasis on intersite communication. Considerable effort will be spent on solidifying existing capabilities and adding new features, such as a slide briefing capability, a vastly extended drawing tool, many new meteorological products, and the creation and execution of procedures. More emphasis will also be placed on accessing data on Web servers, including images publicly available on Websites and some unprocessed meteorological data. It is anticipated that FXC, as with D3D, will be installed at a limited number of sites for evaluation purposes.

RSA – The U.S. Air Force is upgrading its launch support systems provided through the Range Standardization and Automation program. FSL plans to assist in this work by building on the current Linux AWIPS software (build 5.1.2) to provide weather support for launch operations. A limited demonstration is proposed at the Western Range (Vandenberg Air Force Base), with possible expansion to the Eastern Range (Cape Canaveral Air Force Station).

Emergency Management Dissemination System – The Emergency Management Dissemination System (EMDS) is an experimental dissemination system (Figure 42) that integrates gridded weather data, text and graphical information containing severe weather warnings, watches and advisories, point observations, and radar precipitation data to provide state and local government emergency management agencies with advanced weather information. The same communications systems (Internet, Intranet, ftp, facsimile, etc.) that make it possible to gather observations from the community are also utilized for disseminating weather information to community users. Providing advanced weather information to state and local government emergency preparedness agencies can result in more timely and appropriate mitigation procedures by emergency managers. User feedback received from the LDAD test sites in Tulsa, Atlanta, Miami, Des Moines, and Kansas City is being considered in software/hardware upgrades to prototype dissemination systems. Some of these include the integration of Graphical Forecast Editor (GFE) meteorological gridded data and the ability for emergency managers to integrate their own Graphical Information System (GIS) shape files into the dissemination system to reflect new roads, schools, and construction within their local communities.

Another accomplishment involved the technology transfer of operational support and maintenance for the Local Data Acquisition component of LDAD to the NWS Systems Engineering Center (SEC). Meteorologists and software engineers at SEC are continuing to add local data acquisition enhancements and support for NWS field site operations. Thus, as AWIPS continues to evolve, FSL is able to focus on new dissemination functions of LDAD. For instance, a prototype EMDS system was developed that will make it easier for state and local government agencies to use advanced weather information from AWIPS. The Web-based system provides data relating to temperature, dewpoint, relative humidity, fire danger, visibility, and radar, as well as other weather variables such as images and graphics. Critical weather warnings, watches, and advisories can be displayed graphically or as text, offering emergency managers a precise picture of the impact of severe weather. Graphical time series of weather parameters can be displayed to provide users with weather tendency or trend information.

The EMDS can be tailored by individual NWS field offices to distribute AWIPS weather information that is relevant to their local community. At FSL, the experimental system uses datasets from the Local Analysis and Prediction System (LAPS), the MAPS (Mesoscale Analysis and Prediction System) Surface Analysis System (MSAS), and the GFE; however, the system can be configured to use a variety of gridded datasets and point observations. The software has been developed in Java, thus can be used on the Internet or Intranet as an application or applet. While much of the NWS modernization has dealt with improving the ability of NWS weather forecasters to improve the timeliness and accuracy of severe weather warnings, the Web-based dissemination system is targeted to state and local government users of weather information. Once the AWIPS Web-based dissemination system is deployed nationally, state and local emergency preparedness agencies will have access to very high-resolution weather information for their own specific mitigation plans.

With training provided by the Scientific Applications Group, FSL assisted the NWS Western Region headquarters at Salt Lake City, Utah, with the integration of MesoWest observations (including quality control information) into AWIPS at the Western Region sites (about 30 field offices). Other local observations that may be available at Weather Forecast Offices (WFOS) are profiler data in the Alaska Region, flood inundation maps from a local government flash flood office, or other unique local weather observations.

Experiments in the branch led to the implementation of Web-based mirroring technology to provide emergency managers with weather information through the NWS Regional headquarters. AWIPS weather information from a Web server at a local WFO is mirrored at the regional headquarters, providing another level of security for WFO operations.

MSAS/RSAS

MSAS and RSAS exploit the resolution of surface data to provide timely and detailed gridded fields, or analyses, of current surface data. Surface analyses are critical to weather forecasting because they provide direct measurements of surface conditions, permit inference of conditions aloft, and often give crucial indication of the potential for severe weather. MSAS runs operationally at modernized National Weather Service (NWS) Forecast Offices as part of the AWIPS workstation, and RSAS runs operationally at the National Centers for Environmental Prediction (NCEP). Two advantages of both systems are speed and closer fit to the observations, since they produce a one-level, analysis-only grid and therefore require very few compute resources. Also, because the systems do not initialize a forecast model, their respective analyses are performed on the actual surface terrain and not along a model topography; hence no model surface-to-station elevation extrapolations are required, all surface observations may be used, and the fit to the observations is maximized. MSAS and RSAS incorporate elevation and potential temperature differences in the correlation functions used to model the spatial correlation of the surface. The resulting functions help take into account the physical blocking by mountainous terrain, and improve the representation of surface gradients. Data typically ingested by MSAS and RSAS include standard meteorological aviation reports (METARs), Coastal Marine Automated Network (C-MAN) observations, surface reports from fixed and drifting buoys, ships, and the NOAA Profiler and Ground-Based GPS-Met networks, as well as surface observations from any available local mesonets. Sophisticated quality control techniques are employed to help screen the surface observations. On AWIPS, the results of these techniques are passed to the AWIPS Quality Control and Monitoring System (QCMS).

Meteorological Assimilation Data Ingest System (MADIS)

MADIS, formerly named FAME (FSL's American Mesoscale Experiment), provides value-added data from FSL's Central Facility to outside organizations and agencies for the purpose of improving weather prediction. These available observations are useful in the development and verification of meteorological data assimilation systems. MADIS provides access to real-time and archived datasets, observational quality control results, and an Application Program Interface (API) to read and sort the observational data. Quality control of the observations is important because considerable evidence has shown that the retention of erroneous data or the rejection of too many good data can substantially distort data assimilation grids and verification results. Observations in the MADIS database are stored with a series of flags indicating the quality of the observation from a variety of perspectives (e.g., temporal consistency and spatial consistency), or more precisely, a series of flags indicating the results of various quality control checks. Users of MADIS can then inspect the flags and decide whether to use the observation. The MADIS API allows easy access to the data and quality control information, hides underlying data formats from the users, and automatically performs much of the data preparation required for meteorological data assimilation.

Staff continued to support the operational MSAS and RSAS versions on AWIPS and at NCEP. The main MSAS accomplishment was the operational fielding of AWIPS build 5.0, which includes:

• MSAS quality control messages encoded into the Standard Hydrometeorological Exchange Format (SHEF).
• Displays of all three MSAS sea-level pressure analyses (NWS sea-level pressure, MSAS sea-level pressure, and altimeter).
• Two new derived fields (potential temperature advection and equivalent potential temperature advection).
• Displays of the observations used in each analyzed field (along with quality control results).
• Substantial performance enhancements, especially for those programs writing netCDF files across the AWIPS network.
• Improved fallover scripts which allow MSAS to continue running after certain types of AWIPS failovers.

The main RSAS accomplishment was the implementation of subhourly updates for the hourly surface analyses. Multiple runs per hour allow RSAS to first run earlier in the hour to provide more timely analyses (e.g., 5 minutes past the hour), and to subsequently incorporate later-arriving observations and improve analysis accuracy. RSAS is currently the only data assimilation system at NCEP providing subhourly updates to its gridded output. In addition to these accomplishments, the operational versions of both MSAS and RSAS were updated to 1) utilize Eta, instead of NGM forecasts, as the background, or "first guess" grid; 2) widen the observation-time window to allow for the ingest of more Mexican and mesonet observations; and 3) add logic to assign elevations to maritime data in the Great Lakes areas with missing elevations. The Scientific Applications Group also worked closely with NCEP Central Operations and FSL Facility Division staff to set up an RSAS operational backup in FSL's Central Facility.

New functionality for the systems continues to be developed. A major new version of RSAS incorporates a 15-km grid from Alaska to Central America, and covers significantly more ocean areas than were covered in the previous domain. Other upgrades include a new topography grid that better matches observation elevations and better treatment of the model backgrounds. Figure 43 shows an AWIPS D2D display of MSAS observations and grids; Figure 44 shows an AWIPS display of the new 15-km RSAS grid.

MADIS

During Fiscal Year 2000, the initial "alpha" version of MADIS was completed and tested by FSL users. The observational datasets that make up this initial version are shown in Table 1. The observations are acquired by the FSL Central Facility from a variety of sources, including NOAAPORT, Aeronautical Radio INCorporated (ARINC), and FSL's Demonstration Division NOAA Profiler Network (NPN) and ground-based Global Positioning System (GPSMet) data hubs. Mesonet data, decoded and stored with software originally developed for the NWS Local Data Acquisition and Dissemination (LDAD) system, are provided at over 2,000 stations from local, state, and federal agencies and private firms. Major contributors to the mesonet datastream are the NOAA Cooperative Institute for Regional Prediction (CIRP) at the University of Utah, which provides "MesoWest" data from the cooperative mesonets in the western United States, and the Boulder NWS Forecast Office, which provides mesonet data from the local Denver/Boulder area, and also data from the Remote Automated Weather System (RAWS) network run by the National Interagency Fire Center (NIFC).

The current geographic coverage of the data extends from Alaska into Central America, and plans include extending the coverage, in some cases, to global domains. MADIS quality control checks were implemented on three levels. Level 1 quality control checks are considered the least sophisticated, level 3 the most sophisticated checks, with level 2 in between.

Level 1 checks include validity checks, which compare the observed values to specified tolerance limits, and position consistency checks, which compare the current location and time report to previous reports to ensure that a moving platform's position is consistent with its reported movement. Inconsistent positions are identified as unreal speeds or unlikely course changes from the last reported position.

Level 2 consists of internal and temporal consistency checks for all observation types, as well as time-height consistency checks for wind profiler data and hydrostatic, super adiabatic lapse rate, and wind shear checks for the radiosonde data. In general, temporal consistency checks restrict the temporal rate of change of each observation to a set of prespecified tolerance limits, and internal consistency checks enforce reasonable meteorological relationships among observations measured at a single station. For surface data, temporal consistency checks are performed on all pressure, humidity, temperature, and wind data; for aircraft data, they are applied to temperature and altitude reports only. A common example of an internal consistency check is the comparison of dewpoint temperature to temperature: the dewpoint observation must not exceed the temperature observation made at the same station or both observations are flagged as failing. Internal consistency checks for wind profiler data include a bird contamination check that inspects and combines profiler measurements of wind direction, velocity variance, and vertical velocity to detect the presence of bird migration. If birds are determined to be present, the corresponding wind data are flagged as failing the internal consistency check. The time-height check for profiler data then ensures consistency in the time and height dimensions of the winds by using pattern recognition techniques to quality control the current hour's data with past data in a 6-hour sliding window. For radiosonde data, the hydrostatic, superadiabatic lapse rate, and wind shear checks ensure hydrostatic consistency between vertical layers, and also reasonable vertical consistency for the temperature and wind data.

MADIS level 3 quality control checks use surrounding observations to check the spatial consistency of the observation being quality controlled. The only level 3 check supported in the initial version of MADIS is a spatial consistency check applied to surface observations by the RSAS system running at FSL. Results of the RSAS check are included with all MADIS surface observations.

The MADIS API, a library of subroutines callable from Fortran, provides access to all of the MADIS observation and quality control information. In general, the API is very easy to use, and allows each user to specify station and observation types, as well as quality control choices and domain and time boundaries. With the API, the underlying format of the datasets remains completely invisible to the user, and many implementation details that arise in data ingest programs are automatically performed. MADIS API users, for example, can choose to have their wind data automatically rotated to a specified grid projection, and/or choose to have mandatory and significant levels from radiosonde data interleaved, sorted by ascending height, and corrected for hydrostatic consistency. Figure 45 shows the MADIS data available at 1200 UTC on 18 September 2000.

The MSAS and RSAS systems will continue to be improved, including upgrades required to increase grid resolution and vary domain boundaries. MSAS upgrades on AWIPS will include incorporation of configuration scripts that allow each NWS Forecast Office to specify the domain and resolution of its local MSAS systems, and also to specify the analysis grids desired by its forecasters. RSAS upgrades at NCEP will include implementation of a 15-km grid that will cover the full North American domain.

MADIS

Observations and capabilities will continue to be added to MADIS. The initial version of MADIS described here is expected to be available to university and government researchers in Fiscal Year 2001. Access will be through a Web interface that will provide the forms necessary to request real-time and archived data, and allow users to download the MADIS API, a "READ ME" installation guide, sample programs and data, and a complete User's Guide. The Web pages will also provide access to an e-mail forum for users to submit questions and obtain answers from MADIS developers and/or other MADIS users. Access to additional data, including satellite- and radar-based datasets, is also planned.