Case Study
Cleaning Up Data in South Adams County
The South Adams County Water and Sanitation District (SACWSD) was formed in 1953 under the State of Colorado Special District provisions to serve the area now known as Commerce City. Although the district serves the Commerce City population, it is a separate entity and presently the largest combined water and sanitation district in the state of Colorado, covering 65 square miles and serving nearly 50,000 customers.
Challenge
The geographic information system (GIS) team at SACWSD draws on multiple data sources, including their own fieldwork, to compile three separate datasets covering drinking water, irrigation, and sewage systems. The team uses ArcGIS to support expansion, maintenance, replacement, and improvement of the systems. In addition, the data is shared with third-party applications, such as the district's asset management system (for daily service requests and completed work orders) and the water modeling software, which has its own requirements. So, having reliable data is important.
In fall 2016, the water and sanitation district had just completed collecting data using GPS. GIS supervisor Cliff Sullivan and GIS technician Mark Dickman soon realized that data quality varied widely. Inaccurate data can potentially reduce efficiency by delivering incorrect asset locations, and it makes integrating with other business systems difficult. Better data also helps improve customer service with accurate work assignments and enhanced emergency response.
The job of cleaning up the data started out as a long, tedious effort. Sullivan, Dickman, and an intern had to review each feature manually for spatial accuracy and attribute correctness. This type of quality control (QC) process was limited in that it was exploratory and not comprehensive.
Left: Field crews use a metal detector to locate a missing valve by tracing the metal lid of the valve box. Once located, its spatial location using GPS coordinates is updated in the GIS. Right: GIS Supervisor, Cliff Sullivan (right), providing clarification about the direction of a water line and the location of other valves for that line.
Solution
Sullivan noticed an Esri Data Health Check on the agenda for an upcoming conference. He learned that ArcGIS Data Reviewer, an extension to ArcGIS, could make his job go much faster. In addition to the ArcGIS extension, the district purchased on-site consulting services and worked with an Esri geodata engineer to implement Data Reviewer and provide knowledge transfer.
Results
The GIS team members at SACWSD used Data Reviewer initially to get an overall snapshot of the data to help them decide which out of the hundreds of possible data checks that would perform first.
Sullivan said, "Our Esri consultant was patient and very thorough in showing [us] how to use the product. We covered a lot in the two days that consultant was on-site. Our experience with Esri Professional Services was stellar."
"Working with Esri allowed SACWSD staff to make rapid progress. While some checks are self-explanatory, staff needed expert assistance to understand the logic behind configuring the more advanced ones and to get them to run correctly.” Dickman said. The best practices are now captured in a template and made available to anyone.
ArcGIS Data Reviewer’s Batch Job Manager Tool allows for easy creation or modification of automated checks. When the checks are done running, the total number of errors found is displayed and stored in the Reviewer Table.
Data Reviewer helped standardize the quality control process of prioritizing and finding errors and tracking the corrections. Within three months, one dataset was completely cleaned up. Spatial edits on the remaining two larger datasets were soon completed. The initial focus was finding disconnected features. Connectivity checks ensured that all features were connected properly for the water modeling software to successfully run and provide accurate results. Next, for service requests and work orders to be correctly created within the asset management software, each asset's ID must be populated and must not have duplicate values. These were determined using the Unique ID check.
Esri was also instrumental in helping the district build its databases and providing advice on data modeling such as using domains instead of subtypes. The district worked collaboratively with Esri to build a data schema that is now self-explanatory to users in the organization.
Before they acquired Data Reviewer and engaged Esri's help, Sullivan and Dickman were looking at years of manually checking features as they built their database. Now, only Dickman is working on the effort, and he is already six months ahead of schedule. By the time all three datasets are complete, they expect to have saved more than one man-year.
As new infrastructure development continues, the district plans to use ArcGIS Data Reviewer to perform QC on the new data as well as to maintain their existing data.
I highly recommend this approach for small shops like ours [whose staff] are not able to dedicate 100 percent of their time to doing QC.
