|Brief Review of Problems and Issues with Integrated Surface Data (ISD)
Following is a brief summary of problems and issues documented thus far with the historical ISD database. These are often the result of problems in the originally reported/transmitted data or with the metadata. Corrections will be applied as time and resources permit.
1) Global surface hourly data prior to 2006 have a sporadic problem where 6-hour precipitation amounts (i.e., for preceding 6-hour period) are sometimes = 0 when the 6-hourly amount wasn't reported but precipitation did occur.
2) For 1996-present, some 3-hourly and 6-hourly precipitation fields have the condition code = 2 (indicating trace) when the precipitation amount is greater than 0 -- in this situation, the condition code should be ignored and the precipitation amount shown is correct. These problems are restricted mainly to the U.S., and are found occasionally during precipitation events. Most of these instances have been corrected in the data. But, please keep this in mind when using these particular data fields. Other precipitation reports, such as hourly and 24-hourly amounts, are not affected by this problem.
3) For users of the abbreviated format in the Climate Data Online web system (space-delimited format in English units), a problem was recently corrected, which occasionally caused database wind gust values of missing' to be converted to a data value of over 200 mph on the abbreviated form. Yes, those speeds are a tad high, so if you received any data such as this, simply ignore those extremely high values. The problem has now been corrected.
4) Some systematic problems have been found in data prior to 1950 for selected U.S. stations. These data were very recently key-entered from original forms, and some problems occurred in the processing which affected some of the data. The data will be replaced in the near future.
5) Auto METAR observations (origin - DATSAV3) have some negative temperatures incorrectly decoded as positive between August 2002 and January 2003.
6) U.S. station (USAF ID) 722053 (Orlando, FL) has around 10 incorrect temperature values surrounded by missing temperature data in each case between March and May 1990.
7) U.S. station (USAF ID) 722051 (Orlando FL) has thousands of incorrect temperature values between 1976 and 1988.
8) U.S. station (USAF ID) 726885 (Meacham OR) has scattered invalid temperatures during January 1973, June - August 1973, and January 1974.
9) Temperature and dewpoint values for U.S. station (USAF ID) 727476 (Baudette MN) have been determined to be too high from November 1983 through October 1984.
10) The Douglas Army Air Field AZ station (WBAN 23001, USAF ID 999999) has been assigned the civilian WBAN number for (USAF ID) 722735 Douglas Bisbee Int'l AP AZ (WBAN 93026) during parts of 1942 and 1943.
11) Data for Sioux Falls Municipal AP (WBAN 14944) and Sioux Falls Army Air Field (WBAN 14908) have been combined as one station (ie – in one data file) from June through August 1951. (USAF ID = 726510).
12) U.S. station (USAF ID) 726776 Lewistown MT should have WBAN number 24010 instead of 24036 for 1942 through 1944.
13) U.S. station (USAF ID) 722687 Cavern City Air Terminal NM should have WBAN number 23006 instead of 93033 for 1942 through 1945.
14) U.S. station (USAF ID) 726460 Wausau WI has visibility values which were occasionally offset by one digit in the original/incoming data, resulting in decoding errors and incorrect values in ISD. This happens occasionally in the 1983 through 2000 period, and possibly other periods. For example, one occurrence is 11/25/1983 at 1400 UTC and another is 04/25/1986 at 0200 UTC.
15) U.S. station (USAF ID 999999) Dalhart Army Air Field TX is assigned WBAN ID 93042, but should be 23010 from 1942 through 1945.
16) U.S. station (USAF ID) 722560 Waco, TX is assigned WBAN ID 13928, but the stations’ data are duplicated as U.S station 999999 with WBAN ID 13959 from 1942 through 1945.
17) Katmandu, Nepal (USAF ID 444540) has invalid precipitation data (by a factor of 10) due to a data reporting problem (whole millimeters vs tenths of millimeters in synoptic code). Period affected is not yet determined.
18) a) All data from 07/1941 - 05/1947 for U.S. station (USAF ID) 722630/San Angelo Mathis Fld TX with 99999 as WBAN ID are actually San Angelo Goodfellow AAB (USAF ID 999999, WBAN ID 23017).
b) Some data in 1945 for U.S. station (USAF ID) 722630 San Angelo Mathis Fld TX may include observations from San Angelo Concho Army Air Field.
c) All data from 02/1948 - 06/1948 for U.S. (USAF ID) station 722630 San Angelo Mathis Fld TX with 23034 as WBAN ID are actually San Angelo Goodfellow AAB (USAF ID 999999, WBAN ID 23017).
d) For 07/1948 - 08/1958, it appears that data for WBAN ID 23034/San Angelo Mathis Field and WBAN ID 23017/San Angelo Goodfellow AFB are combined as (USAF ID) 722630 San Angelo Mathis Fld TX (WBAN ID 23034). The number of observations are roughly double the norm for these months, and 07/1948 is historically the beginning of the DSI-3280 airport observations.
19) U.S. station (USAF ID) 723965 Paso Robles CA is assigned WBAN ID 93209 from November 1948 through March 21, 1952 but should be WBAN ID 23231.
20) U.S. station (USAF ID) 725765 Riverton WY is assigned a missing code for WBAN ID (99999) from 2000 – 2005 but should be WBAN ID = 24061. After 2005 this station is listed as (USAF ID) 726720 with a missing code for WBAN ID (99999). The data for this period should be under USAF ID 725765 and WBAN ID 24061.
21) During the cold months at the beginning and ending of 1948, the dew points at Bozeman MT (726797 24132) are all 2 or 3 degrees too low. It seems like the lowest ones (around 0 deg F, and thereabouts) are all 3 degrees low, and when it gets a little higher (like in the teens and 20's instead of single digits), they are 2 degrees low. This is confirmed by comparing downloaded data with original observation forms.
22) From January 2006 through March 2010 the temperature and dewpoint values for Newark Intl AP, NJ (72502 14734) from METAR sourced (FM-15, FM-16) records are given only to the nearest whole degree Celsius, instead of to the nearest tenths (i.e. all tenths digits are zero).
23) U.S. station (USAF ID) 722010 (Key West, FL) has incorrect temperature values between 0053 and 0353 on July 12, 2009 (in the negative 20 Celsius range).
24) 39 Canadian sites have missing temperature conversion units in the master station catalog causing the decoder to treat the temperature and dewpoints from these sites as Fahrenheit values and thus requiring conversion to Celsius, when in fact, the temperature and dewpoint values are already in Celsius. This causes these locations to have much lower temperature and dewpoint values than what is actually correct. The period affected for these locations is typically mid-to-late 2000’s and the correction for this discrepancy is well underway.
25) Various, mostly random errors, are present in the overall dataset, and these are being documented and scheduled for correction. Quality control of the data (which has already taken place and reflected in the online data) has corrected well over 99% of the errors present in the original data prior to the quality control. Additional quality control and corrections will be applied as time and resources permit.
26) The following east-coast U.S. stations are experiencing a problem with sea level pressures reported in the mandatory section of each record during some observations in cases where the sea level pressures drop below 960 HPa. When pressures go below this threshold they are reported as 100 HPa higher than they should be (e.g. 958.0 HPa reported as 1058.0 HPa). This occurred during the final 2 days in October 2012 as a result of the passage of Hurricane Sandy. Ref: Oct. 29, 2012 from 2142 UTC through Oct. 30, 2012 at 0155 UTC at 724088 13707. It also occurred on September 22, 1989 between 0200 – 0500 UTC at Charleston, SC as a result of Hurricane Hugo. List of affected stations (there are likely others):
KWRI - 724096 14706 McGuire AFB, NJ
KILG - 724180 13781 Wilmington/New Castle, DE
KNEL - 724090 14780 Lakehurst NAS, NJ
KVAY - 724074 93780 Mt. Holly/South Jersey Regional, NJ
KCHS - 7222080 13880 Charleston Muni, SC
KPNE - 724085 94732 Philadelphia NE, PA
KPHL - 724080 13739 Philadelphia INTL, PA
KDOV - 724088 13707 Dover AFB, DE
KMIV - 724075 13735 Millville, NJ
KACY - 724070 93730 Atlantic City, NJ
27) A gap occurs for numerous non-U.S. stations for the first 8, or so, months of 1999.