Study Ranks Esri US Demographic Data Most Accurate

Many people assume that using accurate data is not a particularly important element of an overall project; however, incorrect data can negatively impact the results of any analysis and have dramatic consequences for affected populations. Data inaccuracies may occur either by overestimating or underestimating populations or households.

Population and household data. Lowest score reflects highest accuracy.

Faulty population data could cause health care providers to underestimate the population and miss vaccinating vulnerable people or waste doses by overestimating numbers. Retailers that locate a new store based on overestimated population data risk losing significant revenue and market share. Local government officials could lose grant funds if poor data is used to calculate the area population. Public safety and risk management agencies could overlook special needs populations before, during, and after a disaster. Does data accuracy matter? Indeed it does—in terms of dollars, health, grants, service—or perhaps life itself.

How Do Data Providers Verify Accuracy?

The release of each US decennial census enables data vendors to evaluate the accuracy of their annual demographic estimates, because these estimates are benchmarked against census results. Data vendors can also learn how their data compares to those of other providers. In 2011, Esri took advantage of this once-a-decade opportunity and commissioned an independent study to obtain an unbiased answer to the question of its data accuracy.

Research Team

The eminent research team of Matthew Cropper, GISP, Cropper GIS; Jerome N. McKibben, PhD, McKibben Demographic Research; David A. Swanson, PhD, University of California, Riverside; and Jeff Tayman, PhD, University of California, San Diego, conducted the study. Cropper has a broad background in GIS and demographic analysis. McKibben, Swanson, and Tayman are noted authorities on small-area forecasts and measures of forecast accuracy. They have written and presented papers extensively on these subjects.

The data was provided to the research team without identifying the individual vendors, enabling a completely blind study. The researchers had no idea which vendor data was included or which methodologies were used by the respective vendors. Esri's motivation for the study was twofold: to test the accuracy of Esri's demographic data to identify areas for improvement in its update methodologies, and to ensure that Esri is providing the most accurate data to its users.

How the Study Was Conducted

The researchers compared the total population and total households data variables from Esri and four other major data vendors. The team conducted the study for the entire United States at the state, county, census tract, and block group geographies. All the vendors, including Esri, had created their forecasts using 2000 Census geography. To analyze the accuracy of the vendor forecasts without modifying their data or compromising the original results, the 2010 Census counts were assigned to 2000 Census geography.

The research team investigated and evaluated a range of direct and supporting measures to assess vendor accuracy and reported the results as a scorecard. The scorecard was then used to measure forecast accuracy across three dimensions of accuracy—Precision, Bias, and Allocation—to obtain a total (unweighted) score. The lowest score denoted the highest accuracy.

Esri data was confirmed to be the most accurate overall.

Precision measures the percent difference between a forecast and a census count and is particularly accurate at measuring small-area forecasts. Therefore, because Precision is the best single measurement of accuracy, it is discussed in the excerpted study report located at esri.com/accuracy.

Esri Ranked First for Precision

The results are in: Esri ranked first for Precision among the five vendors at each geography level. Esri's estimates were also the closest to the Census 2010 results.

The researchers note in their study summary, "After reviewing the results for all quartiles at all levels of geography, it is concluded that Esri had the lowest Precision error total for both population and households. The results also show that at smaller levels of geography, which are more difficult to forecast, Esri tends to perform even better, particularly for households."

What Does This Mean to Data Users?

This study proves that Esri's demographic update methodologies produce the industry's most accurate demographic data. Users can be confident that Esri's Updated Demographics data will provide them with the best possible analysis results.

About Esri's Data Development Team

With more than a century of combined experience, Esri's data development team is composed of geographers, demographers, statisticians, economists, and programmers. This team has compiled a distinguished record of producing innovations to Esri's demographic update methodology to continually improve the accuracy of the data.

How to Access Updated Demographics Data

Esri's Updated Demographics data is available as a database in a variety of formats, including shapefile, file geodatabase, and Microsoft Excel. Updated Demographics is also available in Business Analyst Online, Community Analyst, Business Analyst Desktop, Business Analyst Server, the Business Analyst Online APIs, and the Community Analyst APIs.

To learn more about Esri's Updated Demographics data, visit esri.com/demographicdata.

Related Podcast

Urban Policy and the Census
Heather MacDonald and Alan Peters, authors of the Esri Press book Urban Policy and the Census, discuss the use of GIS in analyzing demographic trends in urban planning. Listen to the podcast. [11:00 | 10 MB]