14.      CONCLUSION

14.1     Introduction

          How accurate is the Dobson instrument? What variance may be ascribed to instrument uncertainties in the ozone data? These basic questions are simple, but their answers are not. As has been shown in preceeding sections, there is a great variety of error sources, there are large variations in the effect of the error sources from instrument to instrument, and the many factors upon which errors are dependent are often not well known. The combining of the error estimates into a single "representative" value is difficult for present day measurements, and is very difficult for measurements made in the past. For most of the errors, especially the larger ones, there is no a priori basis for their representation as variances. Thus there are significant difficulties in assessing the reduction of the errors in data from a time series or from an ensemble of instruments.

          Despite the difficulties, some answers must be given, and it is the purpose of this last section to attempt the generalisations required. It should be understood that these involve a certain amount of subjective judgement. It is recommended that the numerical error estimates given be used with caution and only after reference has been made to the accompanying text.

          Finally, at the end of the section, there is given a summary and discussion of ways of improving the accuracy of both the instrument and the network as a whole.

14.2     Summary and discussion of errors

          Most of the individual error estimates have been given in the summaries to be found at the end of each section. A grand summary of them, for just the AD direct sun ozone estimation, is given in Table 14.1 below. In the table, estimates are made for two loosely defined categories, "typical good instrument or situation", and "typical bad case". The first category describes the errors expected when proper procedures of maintenance and operation are faithfully carried out and when adverse factors, such as stray light, heavy haze, high pollution or high airmass, are not significantly affecting the measurements. The second category describes the typical errors expected on occasion, or at some sites, or with some instruments, in situations when the error concerned is large.

          The proportions of instruments falling in and between these categories depend on the particular error source. A very rough overall estimate is that at present half the network's instruments will meet most of the "typical good" category estimates all or most of the time and that perhaps ten to twenty percent of the network will be affected by some of the larger "typical bad case" category estimates all or most of the time. In the past, the proportion meeting most of the "typical good"

          Three broad generalisations can be made on the basis of Table 14.1. Firstly, the aggregate of the instrument-related error sources is greater than the aggregate of all the other error sources. Secondly, the largest of the instrument-related error sources concern the extra-terrestrial constants, stray light and optical adjustment. Thirdly, atmosphere-related error sources (excluding the absolute accuracy of the ozone absorption coefficient scale) are small except for a few sites affected by severe air pollution. These generalisations have been largely understood and accepted, if not explicitly demonstrated, by instrument specialists for some time.

          The error estimates in each column of Table 14.1 cannot be simply added or RMS added, but some form of aggregation is called for. It seems as if suitable measures of accuracy for present day daily AD direct sun ozone estimations, relative to the current ozone absorption coefficient absolute scale, are about 3% for instruments mostly meeting the "typical good" category requirements, and about 5% to 10% for instruments which suffer from the larger errors listed in the "typical bad" category. Overall, the author risks the guess that two thirds of recent years' AD direct sun ozone estimations have an accuracy of 3% or better relative to the current absolute scale. This is reasonably consistent with the 1979 comparison of Dobson stations with the BUV-TOMS satellite instrument reported in WMO (1982).

TABLE 14. 1 Summary of error sources for individual daily standard AD direct sun ozone estimates.
Error source Error estimates Comments Typical good Typical bad instrument or case situation Instrument sources: Temperature dependence <1% 2% Depends on climate and operator care. Mechanical deformation <1% ? Inadequate information. Stray light <1%? 5%? Inadequate information. Very variable. Rapid increase for airmasses > 3. Wavelength adjustment 1% 2% Better results in recent years generally. Other optical problems <1% 10%? Inadequate information. Some may be accounted for in other error source entries. Wedge calibration 0.5% 2% Modern equipment needed for best results. Bandwidth effect <0.5% <0.5% Larger values for other band combinations. ETC determination 1% 5% Best results from direct intercomparison. Langley method error often about 5%. ETC stability 1.5% 4% Drift over 2 to 5 years between intercomparisons. Electronics <0.5% 1%? Inherently small. Operational precision <1% RMS <2% RMS Depends on skill, atmospheric stability, airmass. Atmospheric sources: Ozone absorption (i) absolute coefficient 3%? 3%? Good prospects for 1 to 2% in near future. (ii) relative coefficient 1.5% 1.5% Due to stratospheric temperature variation. Aerosol extinction <0.2% 3% Worst case requires extremely turbid sky. Interfering absorption <0.5% 5% Only bad near cities, volcanoes etc., on occasion. Other sources: Airmass calculation <0.2% <5% Only bad at high latitudes with "22 km" assumption. Solar variability <1%? <1%? Inadequate information. Is zero when ETC is set. Sampling biases <0.5% 3% Proportional to temporal and spatial variability.

 

          Analyses of archived Dobson column ozone data often use only the AD direct sun observations. However, if all the ozone data are used, then the accuracies of the other observation types must be considered. What little is known of these is summarised for the main types in Table 14.2. All observation types suffer from the error sources listed for the AD direct sun observation in Table 14.1, and usually so to a greater extent. The error estimates listed in Table 14.2 represent approximate accuracies relative to the standard AD estimation. The usefulness of the non-standard observations depends on their information contribution relative to the otherwise available data and to the expected natural variation. In some instances an observation of 10% accuracy may be very useful.

 

TABLE 14.2 Summary of errors for the main non-standard estimation methods. These errors are additional to those in Table 14.1 for the standard AD direct sun method, and are rather approximate.
Estimation method Error estimate Comments Typical good Typical bad instrument or case situation AD zenith blue 2% RMS 5% RMS? Depends on accuracy of empirical method and charts. Little information. AD zenith cloud 3% RMS 10% RMS? As above. Also depends on cloud's type, density and temporal variation. CD direct sun 2% 5%? Errors generally two to three times those for AD direct sun. C direct sun 3% 10%? Additional errors mainly from aerosols and inter- fering absorption. Focussed image 4%? 20%? Very little information. Depends on instrument and on operator skill.

 

          The question of the detectability of trend in ozone data is of great topical interest. Table 14.3 isolates those error sources which could conceivably contribute trend-like error to archived data, and it gives for each source an estimate of the possible trend in the daily AD direct sun observation of an individual instrument. This shows that erroneous trends of up to about 10% per decade can be expected for some instruments. This sets a significant limitation to the trend detection capability of the data. It is particularly ironic that the considerable efforts which have been made over the past decade to upgrade the adjustment and calibration of instruments are themselves a source of some of the largest potential erroneous trends.

TABLE 14. 3 Summary of error sources which could exhibit trend-like behvaiour, with estimates of potential erroneous trend for an individual instrument's AD direct sun observations.
Error source Likely maximum trend Comment percent per decade 1973-1983 Optical adjustments and their historical improvement 5% Due to changes to instruments and periodic calibrations especially upgradings over last ten years. ETC determinations 5% Due to changes to instruments and periodic calibrations especially upgradings over last ten years. Instrument drift 2% Effect is reduced by frequent calibrations. Ozone layer temperature 0.5%? No substantial evidence for this. Aerosol extinction 1% Requires heavy and changing haze. Interfering absorption 2% Only significant near sources. Airmass calculation 1% Due to change from fixed 22 km layer to climatological mean heights. Solar variability <1%? Inadequate information.

 

          It is not clear to what extent the nett erroneous trends of instruments in a group will be correlated. In Reinsel's (1981) statistical study, the trends at individual stations for the one to two decades before 1979 were found to range from about -4% per decade to about +8% per decade. These values are consistent with the estimated magnitudes of possible erroneous trends given in Table 14.3. Although the detected trends may reflect some real ozone changes, their scatter within a region, especially the -2 to +8% per decade scatter found for Europe, indicates that other factors, probably instrument?related errors, are the main cause.

          Any systematic reduction of the effects of stray light (see Section 4) over the last decade or so would have contributed an upward erroneous trend with a total change of possibly up to 10% for the particular instruments concerned. The results of the instrument intercomparisons should provide some information on any predominance of positive or negative corrections and therefore any nett erroneous trends. The calibration histories reported in WMO (1982) for the instruments (mostly United States instruments) which have undergone repeated intercomparisons, show a tendency for successive corrections to be of the same sign. This indicates systematic long term drifts in each instrument's response. Interestingly, seven of the nine instruments show predominantly negative corrections, i.e., positive drifts. The corrections are generally in the range of 1 to 3%, and give nominal drifts generally in the range of 1.5 to 5% per decade. The significance of the drifts will depend on the time intervals elapsing between intercomparisons and hence between corrections. A cursory examination of the international intercomparison results given in Section 7 shows no predominance of positive or negative corrections, though this deserves further study.

          The possible errors for an individual daily ozone estimation listed in Table 14.1 can be large, but the nett errors in statistical means of estimations will be less, of course. In the daily mean for a large number of instruments, or for a large region, many of the errors will reduce as if part of a random distribution. Systematic error components will be present though, the largest being those due to the determination and stability of extraterrestrial constants, to stray light effects and to other optical adjustments or problems, and altogether these might amount to an error of up to say 3%. The combined effect of aerosol extinction and interfering absorption may amount to a bias of 1 to 2% under extensive urban haze and pollution, such as is common, for example, over large areas of Europe and north eastern United States during the summer, or following large volcanic events, but otherwise it should be less than about 0.5%. The combination of biases due to the bandwidth effect, the airmass calculation, solar irradiance variability and sampling biases are likely to be negligible.

          At an individual station the errors in time averages, viz, monthly, seasonal and yearly means, will also reduce, to the extent that they are random over time. However, there will remain some significant relative biases, in particular, those from errors of adjustment and calibration, which may cause seasonally dependent errors of up to 10% and which may show step changes due to periodic re-adjustment or re-calibration. Interfering absorption can also cause biases at some sites which might amount to say 2% for some monthly data. The total relative bias in yearly mean data, at a guess, might lie at 3% or less for most individual instruments at present, but would have been greater in the past. This is also moderately consistent with the results of the Dobson-BUV TOMS comparison results (WMO, 1982). The ultimate and more hazardous guess is that the yearly mean global ozone column data might have relative accuracies of about 1 to 2%. This guess is obviously open to debate.

          An important point is that statistical trend analyses of historical data series, if properly done, will show up any step changes in the data record, together with any systematic variations or excesses in the quasi-random components of the record. Indeed, such analyses may in fact be more useful for the detection of data irregularities than for the original purpose of trend detection.

          Clearly, in order to extract the most information from any Dobson ozone data set, it is essential for statisticians and instrument specialists to jointly examine the data set, preferably on a station by station, or instrument by instrument basis. To simply use assessed typical error estimates, such as those given here, may result in large overestimates or underestimates of errors for a particular instrument, as well as a missed opportunity to detect, and hence correct, obvious data errors.

          An instructive example of the joint approach is the re-evaluation of the 1963 to 1979 Dobson record for Bismark, North Dakota by Komhyr et al. (1981b). They carefully examined the calibration and test histories of the instrument and determined corrections for various factors for the whole record of daily data. The most significant correction component arose from changes to the extraterrestrial constants. The corrections showed a seasonal cycle superimposed on a long term drift, with magnitudes typically in the 1 to 3% range. Only about 2% of the data needed a correction greater than 5%. The largest errors, from 10-18%, occurred either in the early few years, or as the result of two instances of data processing error. This particular instrument is regarded by Komhyr et al. as a very stable one. The authors subsequently carried out trend analyses on both the uncorrected and corrected sets, and found the resulting trend estimates to be different by about 1% per decade, though this was smaller than the statistical uncertainty in the estimates.

14.3     Summary and discussion of recommendations

          Various suggestions and recommendations have been made throughout this review with a view to improving the accuracy of the Dobson instrument and the observational network. Many of the recommendations have been made before by others, but have yet to be implemented satisfactorily. Most have already been listed in the section summaries, but they are drawn together here, under three categories, according to whether they are principally physical factors, instrument factors, or operational factors.

          The physical factors concern ozone absorption coefficients, interfering absorption, and solar irradiance variability. A thorough re-evaluation of the Dobson ozone absorption coefficients based on the work of Bass and Paur (1981) is already being undertaken by an ad hoc committee of the International Ozone Commission and should be completed by the end of 1983. The real significance of interfering absorption by volcanic SO2 and by other less well known trace species, especially those of industrial origin, will require further research. Some surveillance of the literature on atmospheric chemistry for new or newly discovered potentially interfering species would be desirable. The question of the effect of any solar UV variations will be difficult to resolve owing to the requirements of spectral resolution and high accuracy over periods of years. Satellite mesurements will be essential, though there is some prospect of using Dobson instrument calibration records to assess past variability.

          Instrument factors form potentially large error sources. The principal items needing attention are stray light and certain aspects of the optical design. Stray light is undoubtedly a serious problem for some instruments and should be subject to further detailed research. The fact that its effect varies noticeably among instruments suggests that its solution should not be difficult. There are other intrinsic optical characteristics, in particular internal reflections and lens aberrations, whose effects need further careful examination, possibly by means of ray tracing techniques. There are likely to be some associations between stray light, optical problems and inadequate adjustment of the instrument. The dependence of photomultiplier relative spectral sensitivity on temperature and applied voltage deserves further study. Further research into the instrument causes of the larger of the drifts in extraterrestrial constants would also be very desirable.

          Recommendations about operational factors are the most numerous of the three categories considered. This is partly due to the nature of operational observational programmes and their need for continual attention to operational detail. The most important, essential recommendation is that the intercomparison of instruments, both by multiple instrument international intercomparisons and by travelling reference instruments or reference standard lamps, be continued on a regular scheduled basis, and be extended as quickly as possible to include all of the instruments in the network. Only in this way will the large errors due to inaccurate extraterrestrial constants be avoided. It would also be very prudent to strengthen the World Dobson Spectrophotometer Standard by representing it as the result of not just one, but three primary reference instruments.

          The current efforts to extend to all instruments the quality of adjustments and operations being achieved at the major laboratories should also be continued. Serious consideration should be given to establishing international and regional training programmes for station observers. The possibility of a newsletter to disseminate, in timely fashion, views and techniques relating to the network and its accuracy should also be encouraged. A newsletter could also provide a vehicle for the many useful small studies, for example studies similar to the papers of G.M.B. Dobson (Walshaw, 1975) which otherwise might not be published.

          There is a need to further explore the accuracy of the non-standard observation types, e.g., the cloudy sky method, and to seek improvements to these observation types. The development and exploitation of automated Dobson measurement systems should be given every encouragement. Further studies of sampling errors and their impact should be done, an important issue being to better define the ratio of scientific benefit to financial cost for the establishment of any new stations. There are regular calls for new stations in the remote oceanic and southern hemisphere regions, but so far few countries have shown any willingness to financially support them.

          Overall, it seems that in order to have the world's Dobson instruments function optimally as a network, there is a need for further coordination and leadership. Some of the additional responsibilities could be shared throughout the network, but probably most would be best done by, or under the umbrella of, the Central Dobson Spectrophotometer Laboratory. Ideally the management of the network should be based on an agreed-to quality assurance plan which defines the procedures required to actively ensure that the desired standards are reached in ail aspects of the network's operation. The instrument's Operations Handbook (Komhyr, 1980b) already goes a long way to meeting the objectives of quality assurance. Serious consideration will need to be given to increasing the resources available to implement the various recommended components of any quality assurance plan, especially the relatively costly tasks of international intercomparisons and international staff training exercises.

          Error analyses and error reduction efforts are usually very worthwhile, but their success inevitably advances the point of diminishing returns, where only small improvements in accuracy are gained for large efforts. At present there remain several large potential error sources, of the order of 5%, which definitely require further attention. However, once these are generally under control, our efforts should then increasingly concentrate on ensuring the successful routine operation of the network at this sustainable minimum error level, which for individual instruments in the Dobson network might eventually be, overall, about 2% relative uncertainty, plus the 1 to 2% absolute uncertainty due to the uncertainty in any new standard ozone absorption coefficients.


Return to Table of Contents

Forward to References