Best Practices for Pipeline Inspection Data with GIS

Utilities that have adopted a geographic information system (GIS) to support the management and inspection of pipes have significantly increased efficiency. Many utilities store pipeline inspection data and GIS data in the same geodatabase. Unfortunately, further down the road, the geodatabase and multiple associated files become more complex and increasingly vulnerable to serious management complications. An alternative solution that has long-term sustainability involves keeping the majority of pipeline inspection data in a separate database. The question, then, is how the pipeline inspection data can be used in conjunction with the GIS data. The answer is to use innovative, next generation pipeline inspection software. Such software can link or launch the data from the two databases with ease, allowing engineers, field staff, planners, and schedulers to have a real-time picture of the state of the pipeline system. This is a critical step to scale operations in the future, especially as various technologies—with even more data associated—become more commonly used in pipeline inspection.

Pipes are crucial infrastructure in cities. Pipes deliver commodities like water, natural gas, and petroleum and remove refuse like wastewater, all of which is important in daily life. Since the pipes are underground, proper methods must be used to monitor, inspect, and maintain them. GIS has become crucial for governments and companies to support pipeline management. Ways in which the industry has integrated pipeline management and GIS include the following:

Once a pipeline has been put into place, continual monitoring is necessary to ensure that no structural or operational problems, leaks, or geohazards could arise. This is critical not only for the petroleum industry but also in the management of municipal services such as water distribution and wastewater collection. Proper inspection of pipelines—using remote hardware for closed-circuit television (CCTV), sonar, laser, acoustic, Electro Scan,^[1] IBAK PANORAMO,^[2] and other inspection technologies—can enable engineers to monitor hazards and determine operational, maintenance, or structural needs.

^[1] Electro Scan is a registered trademark of Electro Scan Inc.

^[2] IBAK PANORAMO is a registered trademark of IBAK

GIS data has an invaluable role in identifying and mitigating emergencies that can occur in pipelines. This is crucial, especially in pipelines that manage wastewater and water distribution. The majority of US cities cannot function without water distribution and will quickly have major health and safety issues if water or wastewater collection systems are not functioning. Continuously monitoring for structural and maintenance problems is a proactive approach and enables fast response to any problems with the pipeline before the situation becomes worse. Swift response to pipeline emergencies can protect people and crucial resources. 

Field operations are among the more expensive aspects of pipeline management. However, geographic information systems and inspection data can be used to significantly reduce these costs. One example is the use of GIS to locate pipes with maintenance or structural needs for rehabilitation and repair; GIS can be used to identify critical assets and ensure that they are a higher priority when repairs and/or rehabilitation is needed. 

Inspection data—when it is easily accessible through GIS and includes comprehensive detail—can be used for planning and prioritizing projects outside pipeline maintenance such as street paving, sidewalk repair, and fiber installation. Failing to review the condition of a pipe prior to an asset being installed above it can be costly. Imagine having to tear up your newly installed highway because of a failing pipe that needs to be dug up and replaced. By checking the status of pipelines prior to construction projects, repairs and rehabilitation can be performed prior to an asset installation, potentially saving millions of dollars.

ITpipes understands the benefits of integration with ArcGIS and provides effective software that supports pipeline management. Having collaborated with Esri for many years, ITpipes has created intelligent software that brings innovative features for pipeline inspection management to your fingertips. The specific features and benefits of ITpipes include the following:

• ITpipes software is integrated with Esri ArcGIS Online, Esri published feature services, ArcGIS Desktop, and other GIS-based services. This integration allows engineers and inspectors to create configured go-to solutions using maps with pipeline inspection information.
• The ITpipes software allows the collected data to be transmitted to the field workforce. Fieldworkers are able to use GPS and location awareness to get access to existing-condition assessment information, with comprehensive detail that can help them with their routine inspections. This includes making streaming video, graphs, and charts instantly accessible on any device.
• The ITpipes intuitive workspace helps scheduling by integrating activity tools with a web map that can be used to provide crew assignments, simplify routing, and organize routine and emergency inspections. This is all done in the software, making it easier for all field crews and supervisory staff to organize scheduling, planning, prioritizing, and thus monitoring activities.
• ITpipes has advanced data management that enables reviewers to group, sort, display, and filter report data. As the user adjusts the data view, the map view is dynamically updated as well. This can be based on scheduling; planning operations, maintenance, and structural repairs; or performing other predefined reporting. Easy-to-use tools can display coordinated map reports from inspection filters to provide real-time visual analysis of the data collected. All this can be achieved with the simple setup of ITpipes with Esri's ArcGIS.
• An easily accessible inspection view includes detailed information and allows users to compare old and current inspections for the same pipe. This view can be launched, with a single click, from any ArcGIS web map or enterprise application such as Cartegraph, Cityworks, or Lucity products; Tyler Technology's Munis; Novotx; and Maximo.
• ITpipes software provides GIS administrators with comparison and discrepancy reporting tools that are invaluable. These tools can notify administrators when discrepancies exist between the GIS attribute data and field survey information. The tools can be used with a few clicks to apply updated information to the geodatabase. This keeps maps up-to-date with information collected in the field and minimizes GIS administrative efforts.
• ITpipes is configurable and includes several specific national and regional standards. These include SPICAP, PACP, LACP, MACP, SCREAM, and WRC.ITpipes administration tools for IT staff are invaluable. ITpipes provides tools that can be used to easily manage data from the multiple inspections of a pipeline.
• Data from the different inspection methods can be integrated to provide a multidimensional assessment that includes CCTV, laser, sonar, multisensor, inclination, and grout data. This provides a more realistic picture of sewer mainlines, storm drains, manholes, culverts, laterals, wells, dams, levees, and more.

Ultimately, all these features work effectively in providing inspection intelligence with collaborative mapping. This intelligence enables inspectors and engineers to schedule, plan, collect, and analyze pipe inspections, thereby ensuring that the system is fully functional without compromising residents’ way of life.

GIS data storage practices depend on the size of the organization. For instance, smaller organizations tend to have limited staff that use local GIS files and ArcGIS Online, in combination with desktop GIS, COTS product-based browser apps, and native device apps to distribute spatial data.

Larger organizations have more complex GIS support, with staff in different interdependent branch offices that are responsible for collecting, monitoring, and inspecting segments of the pipeline. For these organizations, a simple, local, software-as-a-service (SAAS) solution does not provide adequate benefits. It is hard to minimize redundancy while ensuring that the right kind of data is accessible to the right individuals. There are numerous considerations that need to be kept in mind. These include ensuring that:

• The most recent GIS data is accessible to the stakeholders. The multiple data updates collected in different departments need to be coalesced and presented as a single data view for the entire pipeline system or an individual pipe.
• There is an acceptable level of performance of the system in terms of reliability, speed, and accessibility.
• The pipeline attribute data is valuable to users by keeping this information up-to-date.
• All inspection data—including current and historical inspection information and various inspection types of data such as CCTV, sonar, and laser—is available to users, and that all users have the ability to easily review and compare inspections.
• The system facilitates interdepartmental sharing of data. GIS staff should be able to coordinate efforts with field staff to aid in achieving more accurate and efficient pipeline monitoring.
• There are seamless system updates and upgrades with minimal impact on end-user effectiveness of use. There should be no downtime—in case of emergency, this information needs to be readily accessible.

In larger organizations, GIS data is generally stored in Enterprise Geodatabases (EGDBs) that run on top of a common relational database management system (RDBMS) such as SQL Server, Oracle, and PostgreSQL. Storing data in an EGDB running on an RDBMS provides an enhanced level of data integrity and security because of the inherent advantages of a RDBMS, and it allows data to be easily accessed from desktop, browser, and native device applications in the office or the field. For desktop GIS, this access is via direct database connections, but for browser and native device applications, these datasets are accessed through RESTful web services provided through ArcGIS Enterprise. By using EGDBs coupled with ArcGIS Enterprise, organizations can access, share, and use this geographic content from any device, anywhere, anytime.

There is a misconception that all data collected from inspections should be seamlessly added to the same database—this is not very effective for a couple of reasons. First, the integration of an asset's data—based on multiple inspections and hundreds of observations per inspection, as well as hundreds of associated files—pushes the envelope for easy administration. Second, pipeline inspection information changes constantly, and there are associated files that need to be stored. This can easily cause challenges with the management of inspections and make it difficult to access the right inspection information through GIS, when the data is stored directly inside the geodatabase. We propose that it is better for minimal overall pipeline asset data to be updated in the geodatabase quickly and regularly, and that the majority of the pipeline inspection data be stored in a different database. This data must be easily accessible and consumable by GIS users.

Typically, there are three levels of data pertaining to pipeline inspection:

• The first level is the asset detail header that is similar to attribute data that exists only once per asset. The asset detail header can be used to store information such as the pipe asset identifier; pipe size, shape, material, length, and status; and the date the pipe was built.
• The second is the inspection header detail that exists multiple times for a specific asset in the pipeline. These details are recorded for every inspection performed. Examples include the inspection date, the reason for inspection, the inspector's name, and the flow.  The information may be recorded several times during inspection.
• The third and final level includes observation details that are recorded multiple times during each individual inspection. This includes the logged defects and the severity of the defects. Inspections should always monitor and record the extent of the damage of a specific asset as well as any upcoming structural or maintenance needs, along with the severity of the overall need. 

These three levels of data are continually updated. In addition to these details obtained with every inspection and observation, you have associated files. These files include PDF reports (potentially multiple PDF files for each inspection), snapshot image files of defects (sometimes with several snapshots per defect), video files associated with inspections, laser and sonar imagery, inclination and grout data, and Acoustic and Electro Scan data and graphs—the list continues to grow.

The most common approach initially requested by agencies is to connect the enterprise geodatabase to thousands of associated files (such as videos) that are referenced after pipeline inspections are complete. The problem with this approach is that it becomes quite difficult to manage these files in the long term. There are always issues with missing files, directory movements, and location changes influenced by the new data that is obtained from the pipeline inspection.

The most effective solution is connecting your enterprise GIS to a pipeline inspection app that fully manages all of the related reference files, while the inspection data is kept in a different database but is linked with the software. The application will provide the necessary tools required for management and archiving without burdening the geodatabase with the pipeline inspection data. Using separate databases for inspection and GIS data allows the inspection software to use a single callout from your Esri application.

The following are some of the data sharing options that you can implement:

• Using the pipeline inspection app allows you to group pipes by asset, inspection, or observation details and visualize this information on a map. With subgrouping on any data field—such as overall or observation pipe ranking, automated color coding for unlimited grouping, and linear plot tools to visualize actual defects via color coding on the map—you have all associated details for planning and prioritizing operations and maintenance or structural repairs. Plus these tools are easy for any operator, supervisor, or engineer to use and do not require setup by a GIS professional.
• Setting up an app to launch from your enterprise GIS for creating full condition assessment and sharing inspection details is recommended. You can easily send a hyperlink to access comprehensive inspection detail via such an app. From enterprise GIS, you could send attribute details or hyperlinks.
• Deploying bid packages for contractors, engineering packages, and more, via an app like this is by far simpler than pushing info out of a geodatabase and trying to reconnect associated files and pieces for sharing.

Using a native pipeline inspection app provides the best practice for analyzing, reporting, reviewing, and sharing inspection information. Storing pipeline inspection information separately from the geodatabase will ensure that data can be easily managed while producing results that are more successful in the long run. 

GIS has become a crucial part of pipeline monitoring and maintenance. The advancement in pipeline inspection techniques and the addition of new multisensor technology have increased the detail and complexity of the data that is collected. Typically, most companies want to put all the pipeline inspection data with the GIS data into one geodatabase. Unfortunately, this can lead to complications and mismanagement in the long term due to the multiple tiers of inspection data and associated files related to inspections. Pipeline inspection data requires different methods for storing and curating it. ITpipes suggests that instead of using the rudimentary approach highlighted, companies create a separate database for storing all the data on pipeline inspections. The software will be able to link the data from the two systems, update enterprise GIS attributes of the desired data, and provide management tools for associated inspection files and imagery. Keeping the databases separate, yet integrating specific overall pipe inspection attributes, ensures that the data stored in the EGDB does not get complicated or convoluted with the minutiae of pipe inspections. This helps reduce management efforts further down the road, keeps your data up-to-date, and provides easily accessible real-time analysis to all users.