Inconsistencies in NCDC Historical Temperature Analysis?

By: Christopher C. Burt , 6:46 PM GMT on June 06, 2012

Share this Blog
5
+

Inconsistencies in NCDC Historical Temperature Analysis?

Jeff Masters and I recently received an interesting email from Ken Towe who has been researching the NCDC historical temperature database and came across what appeared to be some startling inconsistencies. Namely that the average state temperature records used in the current trends analysis by the NCDC (National Climate Data Center) do not reflect the actual published records of such as they appeared in the Monthly Weather Reviews and Climatological Data Summaries of years past. Here is why.

An Example of the Inconsistency

Here is a typical example of what Ken uncovered. Below is a copy of the national weather data summary for February 1934. If we look at, say Arizona, for the month we see that the state average temperature for that month was 52.0°F.



The state-by-state climate summary for the U.S. in February 1934. It may be hard to read, but the average temperature for the state of Arizona is listed as 52.0°F From Monthly Weather Review.

However, if we look at the current NCDC temperature analysis (which runs from 1895-present) we see that for Arizona in February 1934 they have a state average of 48.9°F, not the 52.0°F that was originally published:



Here we see a screen capture of the current NCDC long-term temperature analysis for Arizona during Februaries. Note in the bar at the bottom that for 1934 they use a figure of 48.9°.

Ken looked at entire years of data from the 1920s and 1930s for numerous different states and found that this ‘cooling’ of the old data was fairly consistent across the board. In fact he produced some charts showing such. Here is an example for the entire year of 1934 for Arizona:



The chart above shows how many degrees cooler each monthly average temperature for the entire state of Arizona for each month in 1934 was compared to the current NCDC database (i.e. versus what the actual monthly temperatures were in the original Climatological Data Summaries published in 1934 by the USWB (U.S. Weather Bureau). Note, for instance, how February is 3.1°F cooler in the current database compared to the historical record. Table created by Ken Towe.

So Why the Difference in current NCDC records and past USWB records?

The basic reason for the difference is that the NCDC has begun to switch over from using what they call the ‘Traditional Climate Division Data Set’ (TCDD) to a new ‘Gridded Divisional Dataset’ (GrDD) that takes into account inconsistencies in the TCDD. Here is a summary of what was wrong with using the TCDD data sets to determine temperature averages and trends. This is a quote from ‘Transitioning from the traditional divisional dataset to the Global Historical Climatology Network-Daily gridded divisional dataset’ by Chris Fenimore, Derek Arndt, Karin Gleason, and Richard R. Heim Jr., NOAA/NESDIS/NCDC, Asheville, NC:

“The GrDD is designed to address the following general issues inherent in the TCDD:

1. For the TCDD, each divisional value from 1931-present is simply the arithmetic average of the station data within it, a computational practice that results in a bias when a division is spatially under sampled in a month (e.g., because some stations did not report) or is climatologically inhomogeneous in general (e.g., due to large variations in topography).

2. For the TCDD, all divisional values before 1931 stem from state averages published by the U.S. Department of Agriculture (USDA) rather than from actual station observations, producing an artificial discontinuity in both the mean and variance for 1895-1930 (Guttman and Quayle, 1996).

3. In the TCDD, many divisions experienced a systematic change in average station location and elevation during the 20th Century, resulting in spurious historical trends in some regions (Keim et al., 2003; Keim et al., 2005; Allard et al., 2009).

4. Finally, none of the TCDD’s station-based temperature records contain adjustments for historical changes in observation time, station location, or temperature instrumentation, inhomogeneities which further bias temporal trends (Peterson et al., 1998).”


The new GrDD is based upon 25 square kilometer (about 9.6 square miles) gridded divisions rather than the traditional regional climate divisions used in the TCDD. The original raw data of specific weather stations has not been changed, but for the purpose of the GrDD analysis the temperature assigned to a grid that happens to include a specific weather station will not match the actual station measurement since the gridded area may include several different weather stations and will also be weighted to take local topography and urbanization (heat island) issues into account. The complete transformation to the GrDD from the TCDD is slated to be complete in 2013 but has been used to some degree already since about 2007.

The impact on the long-term climate record is summarized here (from the same document cited above):

IMPACTS (PRELIMINARY FINDINGS)

Using the methods mentioned above, we identified the climate divisions, states and regions for which the data are impacted the most by this transition. In the aggregate, the GrDD dataset is slightly cooler and wetter when compared to the TCDD, but significant regional differences are apparent (Fig. 1). Typically, the greatest deviations occur in areas with large elevation differences, divisions which border Canada and Mexico, and divisions lacking a large network of stations early in the record.”

Here are two maps illustrating how the change has affected the long-term means for temperature and precipitation by traditional divisional units in the U.S.:





The report concludes with the following comment:

CONCLUSION

”The GrDD is a modern, quality-assured database that improves upon the historical monthly temperature and precipitation data that are currently available with the TCDD. Use of these data will improve our understanding of observed changes in climate across the contiguous U.S. (CONUS). Regression techniques used to derive pre-1930 divisional data in the TCDD have been replaced by real station data in the GrDD, improving comparisons made to early 20th Century data. Because of the different algorithms used, slight variances in temperature and precipitation averages may be seen throughout the data record. The average change in trend was about 0.06°F per century. The annual temperature trend in each division is between -0.3°F and +0.3°F per century and only three climate divisions had differences in their mean larger than 0.3°F per century. In terms of precipitation, the mean change in slope is slightly negative for the annual period”.

A Short History of U.S. Weather Data Collection

The first collection of weather data statistics published in the U.S. appeared in 1857 when a book titled ‘Climatology of the United States and of the Temperate Latitudes of the North American Continent’ written by Lorin Blodget. The book contains detailed monthly and annual temperature and precipitation statistics for about 200 or so locations in the U.S. collected from a variety of different sources (mostly military and the Smithsonian Institute) going back in some cases to the 1780s (and made by various individuals).



Title page of Blodget’s book that included the first published U.S. climate statistics for the U.S.

In 1874 the Smithsonian Institute relinquished its weather data collection responsibilities and passed them on to the Signal Service of the Army. The U.S. Weather Bureau (USWB) was established in 1890 under the umbrella of the U.S. Dept. of Agriculture in 1890 and assumed most of the weather data collection responsibilities. In 1895 the Weather Bureau instituted a homogenization of data collection (meaning it adopted a uniform plan of observations for all it weather sites across the country). In 1906 the USWB published its first comprehensive volume of U.S. weather data in ‘Bulletin Q: Climatology of the United States’ edited by Alfred Judson Henry. This book contains complete weather statistics for every site in the U.S. that had both temperature and precipitation (about 650 stations in all plus about another 1000 that collected precipitation data only). In 1970 the USWB became the National Weather Service (NWS).

Here is a short timeline summary of the above (note the overlapping of various agencies over time):

1820-1890: Medical Dept. of the Army military posts

1849-1874: Smithsonian Institute cooperating observers

1870-1905 (approx): Signal Service of the U.S. Army

1890-1970: U.S. Weather Bureau, Dept. of Agriculture

1970-present: National Weather Service (NOTE: the NCDC was established in the 1950s and is part of NOAA not the NWS).

There are currently about 5700 COOP, ASOS, and NWS sites that collect both temperature and precipitation data and an additional 5-6000 that collect precipitation data only.

It will be interesting to see how the changes in weather data analysis will affect the temperature and precipitation long-term averages across the contiguous U.S.

KUDOS: Ken Stowe for his diligent research in comparing the historical weather data records.

Christopher C. Burt
Weather Historian

Reader Comments

Comments will take a few seconds to appear.

Post Your Comments

Please sign in to post comments.

or Join

Not only will you be able to leave comments on this blog, but you'll also have the ability to upload and share your photos in our Wunder Photos section.

Display: 0, 50, 100, 200 Sort: Newest First - Order Posted

Viewing: 8 - 1

Page: 1 — Blog Index

8. SevereClear
12:49 PM GMT on June 23, 2014
Christopher Burt,

In this writing, please correct the errors of "Smithsonian Institute" with "Smithsonian Institution".

Member Since: June 23, 2014 Posts: 0 Comments: 0
7. Christopher C. Burt , Weather Historian
9:10 PM GMT on June 08, 2012
Quoting BaltimoreBrian:
Very good explanation Mr Burt. It looks like your revision page on the web for Weather Extremes will be getting a big rewrite. Unfortunately.


Actually not, since I only follow extreme weather records for individual cities or sites. The original raw data for these sites has not been changed, only the regional averages. So even my monthly temperature records for such and such a place remain valid (whew!).
Member Since: February 15, 2006 Posts: 320 Comments: 297
6. BaltimoreBrian
6:52 PM GMT on June 08, 2012
Very good explanation Mr Burt. It looks like your revision page on the web for Weather Extremes will be getting a big rewrite. Unfortunately.
Member Since: August 9, 2011 Posts: 26 Comments: 8880
5. Christopher C. Burt , Weather Historian
6:19 AM GMT on June 07, 2012
Here is my response to the 'Watts Up With that post' that you refer to:

Mr. Watts,

Christopher C. Burt here, author of the Weather Underground blog you have quoted herein.

First, just for the record, I would like to correct your assumption that my blog was written by Dr. Jeff Masters. Solely myself wrote the blog with no input from others in the Weather Underground organization.

Second, you have quoted only the first half of my blog on the subject of the NCDC changes so far as evaluating the changes in long-term means (LTM%u2019s) of temperature averages in the contiguous U.S. (CONUS). This gives a false impression that I disagree with the new methodology the NCDC is now using. That is not the case, as would be obvious if the 2nd half of my blog had been published in your piece today (June 6) titled %u201CNOAA%u2019s National Data Center caught cooling the past-modern records don%u2019t match paper records%u201D.

There are very good reasons for %u2018massaging%u2019 the areal temperature (and precipitation) data for the use in ascertaining trends in climate change.

For instance in the example I used of Arizona in 1934: the USWB (U.S. Weather Bureau, Dept. of Agriculture) based their 52.0 state average on data from 78 sites that reported from around the state that particular month of February 1934. Of these 78 sites 3 were in the city of Phoenix (Airport, USWB site, and Indian School), 3 were in Yuma (Citrus Station, USWB, and Valley site), and 2 were in Tucson (Airport and Univ. of Arizona campus). So 8 (more than 10%) of the 78 sites for the entire state were located in three of the warmest cities in the state. Furthermore, 27 of the 78 sites were in Maricopa and Yuma counties, the two warmest counties in the state that comprise 12.6% of the state's landmass yet account for 34.6% of the observation sites.

It does not take a genius to see this leads to a problem when trying to ascertain a %u2018state average%u2019 temperature. You might argue, well why not just stick to these same sites that reported in 1934 and compare to what they now observe in 2012? That is not possible because many of the sites that reported in 1934 have long since stopped supplying data, so it is therefore impossible to keep that timeline continuous. Plus, even the city or town sites that STILL report data now have (since 1934) relocated within their municipalities and/or effected changes in instrumentation.

This is why, for the sake of determining long-term trends, it is not possible to simply use the same raw data from 1934 as in, say, 2012.

The NCDC has thus necessarily come up with a better way of trying to address these issues. So long as they apply the same parameters to the new GrDD (grid system) for ALL the sections for the whole POR (period of record 1895-present) then the actual raw data for trend purposes is irrelevant. And, in fact, the original raw data from all the INDIVIDUAL sites used HAS NOT been changed, it is only the way their record has been interpreted that has (for the above reasons I outlined) been changed.

Yours,

Christopher C. Burt
Weather Historian
Weather Underground

Quoting ClimateChange:
Good post, Chris. Of course, the changes are designed to make comparisons possible by homogenizing the method of computing areal means. And, in any event, have had a negligible effect on the overall national warming trend (0.06F/century). This can be verified easily by looking at the old data. The Weather Bureau used to merely average all stations in a climate division to compute a divisional temperature. Since some stations no longer exist and other new stations have come into existence, this doesn't allow direct comparisons in the absence of homogenization. Unfortunately, climate denialists have latched on to this blog post in order to cast doubt on the climate record. I won't link to the offending blog, but I just wanted to let you know of the situation.
Member Since: February 15, 2006 Posts: 320 Comments: 297
4. spbloom
2:03 AM GMT on June 07, 2012
Quoting RogerMaestri:
If they modify the data that is printed and saved in the Library of Congress, imagine what they are doing with the data from satellites!


Your brain is surely not safe, Roger. Quick, to the tinfoil hat!

Or, you know, if after reading and analyzing the cited material you have some actual problem with the changes, perhaps you could enlighten us.

Maybe start with the fact that data itself has not been changed.
Member Since: May 12, 2010 Posts: 0 Comments: 430
3. RogerMaestri
11:51 PM GMT on June 06, 2012
If they modify the data that is printed and saved in the Library of Congress, imagine what they are doing with the data from satellites!
2. ClimateChange
10:36 PM GMT on June 06, 2012
Good post, Chris. Of course, the changes are designed to make comparisons possible by homogenizing the method of computing areal means. And, in any event, have had a negligible effect on the overall national warming trend (0.06F/century). This can be verified easily by looking at the old data. The Weather Bureau used to merely average all stations in a climate division to compute a divisional temperature. Since some stations no longer exist and other new stations have come into existence, this doesn't allow direct comparisons in the absence of homogenization. Unfortunately, climate denialists have latched on to this blog post in order to cast doubt on the climate record. I won't link to the offending blog, but I just wanted to let you know of the situation.
Member Since: September 8, 2011 Posts: 8 Comments: 245
1. Angela Fritz , Atmospheric Scientist (Admin)
8:17 PM GMT on June 06, 2012
Nice explanation of the new QCed product. Although it makes your job a little more difficult, with so many different record sources, I think it's important to address issues like these in the historical temperature record so we can paint an accurate picture of how our climate is changing.

Viewing: 8 - 1

Page: 1 — Blog Index

Top of Page

About weatherhistorian

Christopher C. Burt is the author of 'Extreme Weather; A Guide and Record Book'. He studied meteorology at the Univ. of Wisconsin-Madison.