The Facility Division (FD) manages the computers, communications networks, and associated peripherals that FSL staff use to accomplish their research and systems-development mission. The FSL Central Facility comprises 69 computers ranging from workstations and servers to a supercomputer-class Intel Paragon Massively Parallel Processor (MPP). In addition, mass-storage devices, data-ingest interfaces, local- and wide-area networks, communications links to external networks, and display devices are part of the facility. Over 500 Internet Protocol (IP)-capable hosts and network devices serve the other five FSL divisions and the International Program. These hardware and associated software facilitate the development, testing, and evaluation of advanced weather information systems and new forecasting techniques.
The division designs, develops, upgrades, administers, operates, and maintains the FSL Central Computer Facility. For the past 17 years, the facility has undergone continual enhancements and upgrades in response to changing and expanding FSL project requirements and new advances in computer and communications technology. In addition, FD lends technical support and expertise to other federal agencies and research laboratories in meteorological data processing, telecommunications, and weather radar development.
The Central Facility acquires and stores a large variety of conventional (operational) and advanced (experimental) meteorological observations in real time. The ingested data encompass almost all available meteorological observations in the Front Range of Colorado and much of the available data in the entire United States. Some data are also received from Canada, Mexico, and the Pacific Ocean. The richness of this meteorological database is illustrated by such diverse datasets as hourly surface observations, advanced automated aircraft, wind profiler, satellite Global Positioning System (GPS) moisture, and Doppler radar measurements. The Central Facility computer systems are used to analyze and process these data into meteorological products in real time, store the results, and make the data and products available to researchers, systems developers, and forecasters. The resultant meteorological products cover a broad range of complexity, from simple plots of surface observations to meteorological analyses and model prognoses generated by sophisticated mesoscale computer models. Figure 24 shows the functions and services provided by the FSL Central Facility.
Project and Research Support
In response to FSL user and project requirements, upgrading of the FSL Central Facility with new systems and capabilities will continue during Fiscal Year 1997. In addition to procuring and installing new Open Systems computers, essentially all remaining DEC VAX processors will be decommissioned.
Figure 31. Archive 1 Data Analyzer with enhanced capabilities for the new NWS Operational Support Facility (Norman, Oklahoma) testbed radar.
The next major step in the development of the Facility Information and Control System will be to incorporate the various Central Facility system status information into an Oracle database. This system-type information will include the status of various system resources such as disk space usage, device availability, process slot usage, and memory and CPU usage. The intent is to provide FD Operations staff and developers with more useful and consistent information. The system-level information will be added to the existing monitoring capabilities to provide data that the operators can use when diagnosing reported problems. In turn, this will allow further streamlining of problem notifications and system status reporting. The above FICS upgrades will enable FD operators to better monitor the Central Facility and provide more effective and timely first-line troubleshooting and problem resolution capabilities.
FD Systems Administration staff will develop an in-house suite of computer performance reporting tools with emphasis on Sun Solaris 2.5, HP-UX 10.x, and SGI IRIX 6.2. Also, staff will be writing and consolidating a number of utilities used to report on system performance. This suite of tools will be used to provide reports to developers showing the effects of various pieces of code added to a system. Also, reports will be made available to systems administrators to help pinpoint abnormalities on a particular system, and to management to help in projecting the remaining capacity of existing systems. These reports will make it possible to troubleshoot existing problems more efficiently, help determine where to add additional processes, and/or how to better utilize a system.
For years, FSL employed a common backup strategy for its distributed network of UNIX hosts. A program developed by FD Systems Administration, called FSL_backup, has been running on FSL divisional systems, created 40 sets of backups on 40 tape drives, centrally logged all backups, and provided a common interface for notifying operators when a tape needed to be changed, or when a particular backup failed and the reason for the failure. Because of the substantial increase in backup data volume and the continual need for modifications to FSL_backup due to new operating systems and file system architectures, the laboratory has outgrown the current system. In response, staff is developing a design for a centralized backup solution to meet the current and projected needs of the laboratory. A dedicated, expandable backup system with several DLT tape drives and stackers using a third-party software package will be implemented. FD Systems Administration staff will work with all laboratory systems administrators to determine what types of data need to be backed up and when. The goal is to provide the best and most reliable backup and recovery of user and system files in case of a catastrophic failure.
FD Systems Administration staff will develop a consolidated information repository containing UNIX operating systems and FSL-specific configurations and policies. The repository will be browsable through the World Wide Web and will be located on FSL's Central Server for easy access. Users needing more information about particular systems-related subjects will be able to look here for help. Of course this will be a dynamic repository accommodating the changing interests and requirements of FSL.
The FSL network adequately supports the laboratory's bandwidth needs. Due to the division network upgrades already completed, each FSL division's bandwidth usage is well within installed capabilities. When a division's bandwidth requirements change, the installed division subnet can be expanded very quickly to meet new or increased bandwidth demands without modifying the current design.
However, at the present rate of data and volume increase, the FSL backbone itself will become saturated in about two years. Because almost all traffic external to any division (including access to Central Facility data, e-mail, Internet, and DNS) is passed over the shared 100 megabit per second (Mbps) Fiber Distributed Data Interface (FDDI) FSL backbone ring, the FSL backbone is arguably the most critical part of FSL connectivity. Saturation of this central backbone would adversely affect almost all aspects of FSL communications. In anticipation of this problem, the FD Network Management Team will begin a two-year phased replacement of the current FSL backbone with higher bandwidth capabilities. An Asynchronous Transfer Mode (ATM) switch fabric interconnecting all FSL routers and service nodes will be procured and deployed. Because this switch fabric will provide dedicated 155 Mbps connectivity to each connected node, the aggregate bandwidth in the ATM upgrade will be increased by an order of magnitude. To ensure minimum interruption to current network services while integrating ATM technology into the laboratory backbone, backbone nodes will be connected via the current FDDI ring and also the new ATM fabric in parallel. The recently implemented OSPF protocol in each backbone node will ensure a transparent fallback to the current FDDI backbone when the ATM path is not available. Another advantage of the planned ATM implementation is that current FDDI network cards and concentrators will eventually be placed in FSL divisions to increase bandwidth available to nodes there. This hand-me-down philosophy of network connectivity upgrades ensures no waste of acquired equipment while placing bandwidth where it is most needed.
The Network team will supplement the current FSL dial-in access through 28.8/33.6 kbps analog modems with Integrated Services Digital Network (ISDN) technology providing up to 128 kbps bandwidth to home and travel nodes. An 8-port Basic Rate Interface (BRI) card with associated router will be acquired and installed in the existing FSL Xyplex 9000 Hub. An additional advantage to this approach is that remote access via modem or ISDN will be handled seamlessly by the current operational Dial-In Automated Support programs.
The Network staff will focus their software development efforts on automated network monitoring, bandwidth utilization, line utilization, and connection tools. The goal is to implement these tools for widespread use by making data easily available to systems administration personnel and FSL management using Web technology. This information will be utilized for making informed decisions on needed network connections, system sizing and usage, and system reliability enhancements. In the past, this information was attainable only on a case-by-case basis and required specific programming efforts. Another software development endeavor will address the creation of an automated configuration control system. Employing an Oracle database and Simple Network Management Protocol (SNMP) polling, the system will provide automatic configuration update information for each node on the FSL network.
Implementation of a capability to ingest, store, process, and distribute WSR-88D wideband radar data within NIMBUS will be completed during Fiscal Year 1997. The handling and preprocessing of GOES-9 satellite data will be integrated into the FSL GOES Local Groundstation System (GLGS). The GLGS monitoring and data quality tracking capabilities will be significantly enhanced. Alternate options will be investigated for acquiring real-time TIROS data to replace the outdated VAX-based FSL TIROS data-ingest system. An analysis will be performed to implement an FSL standard image distribution format for transport of locally created radar and satellite image data. In addition, the following additional data-acquisition tasks will be performed:
Networked Information Management client-Based User Service
FD Systems Administration staff and NIMBUS developers will work together to develop and implement a plan for reconfiguring NIMBUS processors into sets of "triads" for enhanced efficiency and reliability. Each triad will consist of three similar machines from the same manufacturer (i.e., Sun, HP, or SGI), and will run the same level of operating system. Two processors in each triad will be configured to perform primary real-time NIMBUS functions. The third machine will serve as an integration system, and also will be configured to act as a failover system should one of the two primary systems in the triad malfunction.
During the first quarter of Fiscal Year 1997, the Auspex NFS file server will be fully integrated into the Central Facility infrastructure. All NIMBUS /public data serving will be transferred to the new file server. As a result, the FSL user community will see much better throughput and reliability of the data made available on /public. By centralizing all the data on the NFS server, failovers from one NIMBUS compute engine to another become transparent to the users because these data will remain in the same location regardless of the system used to generate them. Another advantage to the new file server is that its compute engine architecture can be changed as dictated by laboratory requirements while the data storage area will remain stable and independent of these changes.
FSL's LDM servers will be upgraded to use Version 5 of Unidata's LDM software. NIMBUS itself will be converted to use native LDM software as its distribution router. As a result, all FSL data will have to be assigned more specific LDM feed types for better routing flexibility. FSL will also begin to send data to Unidata for distribution, over their Internet Data Distribution system, to a large number of universities.
The Process Manager will be upgraded to simplify managed-process configurations and to more efficiently perform concurrent processing. The following managed processes will be integrated into NIMBUS: 40-km MAPS using the FSL MPP, LAPS data transfer from the Forecast Research Division to NIMBUS /public, ACARS translators, and netCDF makers for several new data sources. A new method for storing station information and other metadata will be developed by using the Oracle database management system and netCDF. New procedures for metadata table updates will also be integrated.
Efforts will continue to provide more Central Facility information to users by employing the widely available World Wide Web technology. For example, information about the location of all available data and how they can be accessed will be made available to both internal and external users. FSL users will have access to an up-to-date Central Facility Task List on the Web showing ongoing development projects, their priority, and estimated completion dates. Also, staff will continue the development of Web-based NIMBUS documentation.
The aging Sun 670 MP central NIMBUS servers will need to be replaced during the coming year. Provided that management approval and necessary funding are received, a Sun Enterprise 4000 server will be acquired and installed. To complete the SGI triad configuration, a third SGI processor also will have to be procured and integrated.
During Fiscal Year 1997, FD will continue providing data to the WFO-Advanced workstation to support its development at FSL. In support of the WFO-Advanced development effort, staff will provide code and advice as needed for such projects as the implementation of radar data handling applications. Advanced meteorological products will be sent to the Denver NWS Weather Forecast Office. In addition, FD operators will monitor the WFO-Advanced workstations deployed at the Denver WFO 16 hours a day, seven days a week. FD will also support researchers with real-time and retrospective data at ERLs, UCAR/NCAR, and at universities through the UCAR Unidata-developed Internet Data Distribution (IDD) system.
FD will continue to operate and provide network support for the Web-based GLOBE data servers located at FSL to collect, process, and organize environmental data from several thousand schools. The development of a system monitor for GLOBE will continue, and Operations staff will start monitoring the designated GLOBE systems.
In continuation of the joint effort with NCAR Research Applications Program to support the NWS Operational Support Facility (OSF), FD staff will lead a team to advise the OSF on sun-source network calibration and the incorporation of clutter processing upgrades. The team will also install testbed radar instrumentation for the OSF and will perform technical data-acquisition tasks.
Collaboration will continue with Taiwan Central Weather Bureau and Hungarian Meteorological Service scientists and developers in transferring FSL technology to these organizations.
FD technician staff will continue providing audio and visual support for the FSL technical reviews, special presentations and demonstrations, and will support the deployment of FSL computer equipment at the American Meteorological Society Annual Meeting. FD staff hosted the fall 1996 meeting of the ERL Technical Committee on Computing Resources.
This page maintained by: Wilfred von Dauster Last modified 5 August 1997