Developing a Quality Assurance Plan

For our next blog topic, I’d like to discuss how a quality assurance (QA) plan can be used to ensure the production of quality GIS data. A QA plan is a document, usually created by a project manager, which identifies quality related objectives, standards and procedures for your dataset. These guidelines are used to maintain consistent quality control (QC) processes throughout the duration of a project and helps determine the success (or failure) of a project, and whether the deliverables meet customer expectations. In essence a QA plan is designed to ensure everyone involved in the project is on the same page about what quality means for your data and how you are going to measure and ensure compliance throughout the project life cycle.

The establishment of a QA plan has many benefits including less data rework because quality requirements have been identified ahead of time and are being measured and monitored throughout the project. Additionally, project teams often experience greater productivity since a QA plan drives the examination of the production processes for efficiency and effectiveness. For project managers, the QA plan supports quality policy guidelines that your organization may have already established for your projects. For customers, satisfaction is increased because deliverables of the project will meet or exceed their expectations as a result of taking the time to measure and meet quality standards.

Components of a QA Plan
Similar to the way books are organized into chapters, QA plans are easier to understand when they are divided into sections. The design of a QA plan varies widely based on your organization’s requirements, but typically there are four commonly used sections: Project Overview, Project Design, Project Data Assessment, and Project Reporting and Oversight. It is important to understand that this is just one example of the structure of a QA plan; you have the freedom to design and implement one that suits your project’s needs. Regardless of whether you use a fixed template, it is beneficial to think of these key aspects of quality before you begin the project, because planning up front reduces rework later on. Each of these sections has one or more separate components, but I’ll give you a general overview, including some examples of each.

Project Overview
The overview section of a QA plan is where you would bring together important information that introduces project participants to the big picture. By establishing the scope of the project and the tasks that will be performed, you should begin to see areas where quality control measures can be put into place. The overview can be used to reference related documents that define the project objectives and identify QA criteria and procedures such as request for proposals, contracts, budgets, database designs, industry standards, and product specifications.

The project overview can also be used to describe the process that will be used to approve the QA plan. One of the common phrases used in QA is Plan, Do, Check, Act. Before finalizing a QA plan, you may want to test the plan for problems by running a pilot project before initiating full-scale data production. For example, if your organization is contracting out the development of key base layers (i.e. streets, parcels, hydrants, etc), the project delivery team would first be responsible for ensuring the data deliveries adhere to the project’s data creation guidelines. They may be required to test the data with the help of different stakeholders (i.e. police, fire departments). Before full production begins, a pilot project would need to be completed on a small geographic area. If the QA plan is successful against the pilot database, full production would begin.

Project Design
You can think of this section as a way to manage project logistics, in other words, how will you organize and control the project environment and the materials involved. Quality assurance is about more than just running validation tests on your data; QA should be involved in every aspect of the process and can include processes for how to manage materials. Over the course of a project, data may be received from an outside source and it is necessary to organize the data and correspondence in an effective manner. For example, are you planning to organize deliveries and correspondence through file folders (hard copy and digital), email threads, spreadsheets, and/or a shared website? By establishing a mechanism for handling materials you can ensure that everyone has access to the correct information and that they are not validating the wrong version of the database or making changes to data that they should not have access to.

One of the key aspects of this section is the acceptance criteria. This would include data quality aspects such as positional accuracy, attribute accuracy and completeness. If you ask a group of people to define data quality (i.e. what makes a quality road dataset), you will likely get many answers. By defining the scope and purpose of the data in the project overview, everyone has a common understanding of how the data will be used. This drives the requirements for what is an acceptable level of quality. For example, if the purpose of the road dataset is vehicle navigation, then you may have stringent acceptance criteria to ensure that the road name and address range attributes are correctly populated. Whereas you may not have the need to define criteria to ensure that other road attributes such as the pavement type are populated.

Project Data Assessment
This section is all about defining what methods you will use to ensure the data meets the defined acceptance criteria. There are many types of validation you can choose to use. For example, you can use the various ArcGIS Data Reviewer automated checks that will be run on the data deliverables. The Data Reviewer checks can be used to assess spatial aspects of features such as ensuring they meet minimum size requirements, that attributes are correctly populated, or to validate the relationship between features such as finding overlapping polygons. Visual review is also a commonly used validation method that allows you to inspect the data for issues such as missing features or misalignments.

Sometimes, the dataset being validated is large, which makes a comprehensive analysis of the data impossible. In this case, the ArcGIS Data Reviewer Sampling Check can be used to investigate a subset of the entire dataset. The sampling check allows you to save time by not validating each feature. It automatically determines a sample size that is statistically significant at a certain confidence interval, plus or minus an acceptable margin of error. As you inspect the sample set Data Reviewer can report if the entire dataset passes or fails validation.

In this section of the QA plan you would define the workflow to be used to validate your data. Here you can define when you will perform the chosen types of validation. You can also define points in the process when you should stop validation because the data fails and wait for the data to be corrected. The following QA/QC Overview Flowchart is an example that represents the process of using ArcGIS Data Reviewer to perform automated and visual QC of data received from an outside contractor.

Project Reporting and Oversight
Use this section to define how you want to report the results of your QC review. The purpose of a QA report is to disclose errors found during the initial inspection of the deliverable(s) and the automated and visual checks which identified those errors. If automated and visual review were performed using ArcGIS Data Reviewer, one benefit is that a report of errors can be generated using the built-in automated reporting tools. This section should also outline procedures for resolving data problems, timeframes for reporting issues, and approving changes.

It is also important to do oversight on your QA plan. You can start by reviewing it in detail and considering whether or not the results are as expected. If they are, that’s great and you’re ready to do full production, but if not then you need to determine where complications exist. Do your QC procedures need to be modified or does the wording of your QA plan need to be changed?

Final Thoughts
As a GIS or project manager, it is important to identify the details of how QA/QC procedures should be implemented throughout your project. This can be accomplished through a comprehensive quality assurance plan. Establishment of a QA plan supports all members of a production team, their organization and, most importantly, the customer. Teams who leverage a QA plan minimize risk to their project and their reputation by identifying data quality requirements early in a project delivery process and mechanisms to ensure quality production throughout the project life cycle. Finally, customers who drive quality requirements minimize risk to their stakeholders by ensuring data quality is being measured and monitored throughout their project.

For additional information on creating a Quality Assurance Plan, please listen to the Esri podcast ”QA/QC for GIS Data: Creating a Quality Assurance Plan” found on the Esri Instructional Series Podcasts page. I hope you see the benefits of using this powerful planning tool to maintain consistency in the creation of a quality GIS product.

Contributed by Sherry Smith & Jay Cary