1-Maintaining Currency, Minimizing Disruptions

Maintaining Currency, Minimizing Disruptions

Utility uses phased migration from CAD to GIS

By Faye Hall, Enspiria Solutions, Inc.

Editor's note: A large utility employed some innovative strategies that minimized cost and disruption when it migrated from an existing CAD system to a GIS workflow.

The DWG In application runs without user interaction. It compares the latest version of an AutoCAD file to its previous version to determine which features have been inserted, updated, or deleted.

When Georgia Power Company and its parent Southern Company weighed deployment options for a new enterprise GIS, they decided on a phased implementation. A phased approach meant maintaining two systems. However, adequate resources were not available to maintain two systems in a timely fashion, so in addition to migrating data and implementing a new enterprise GIS with new tools and interfaces, they embarked on the development of a utility application that would maintain the currency of the GIS data without additional resource requirements. This utility was DWG In.

Georgia Power Company is affiliated with Southern Company, a regional energy company in the southeastern region of the United States. Georgia Power provides energy distribution services in 153 of Georgia's 159 counties to more than 2.2 million customers.

Southern Company decided to implement an enterprise GIS based on ArcGIS and ArcSDE with additional functionality provided through ArcFM and Designer 9.1, two tools from Esri business partner Miner & Miner. A new data model would be implemented to address core business needs and facilitate the implementation of new functionality provided by Miner & Miner's Designer product, Esri Schematics, as well as additional feeder management and tracing tools.

Prior to the enterprise GIS, Georgia Power maintained its spatial data in a customized AutoCAD Map 6 environment and had developed an extensive library of tools, interfaces, and programs based on this platform to meet its operational requirements. However, these software tools were reaching the end of their life. At the same time, Southern Company was focusing on a single common platform to reduce costs, provide a more robust environment, and support a move to a mobile environment. The need to move to a new platform was evident.

Southern Company and Georgia Power enlisted the services of Enspiria Solutions, Inc., a consulting and systems integration firm and Esri business partner based in Denver, Colorado, to aid in the planning, migration, development, and deployment of an enterprise GIS environment. For Georgia Power, this project included the migration of approximately 120,000 AutoCAD drawings to ArcSDE; the deployment of the populated ArcSDE enterprise database, an ArcGIS environment with custom tools; and the deployment of Miner & Miner's ArcFM and Designer solution. Interfaces for supporting outage management, system analysis, customer linking, and a Web-based map viewing platform would be developed.

The data migration component of the project was to be performed over a 12-month period. Resource levels and the complexity of the environments, interfaces, and tools further extended the amount of time that multiple systems and interfaces would have to be maintained. Once the schedule was finalized, and assuming no delays, there would be more than 18 months between the beginning of data migration and the completion of the rollout of the GIS.

Georgia Power had to address three concerns. Data migration would take several months during which data updates would continue in the AutoCAD environment. Regions would not be transitioned to the GIS environment until several months after deployment when the system interfaces were complete. During the intervening months, updates would continue in the current AutoCAD environment. The data migration investment needed to be protected and the data not allowed to become stale.

To facilitate the transition and address its concerns, Georgia Power analyzed the following alternatives for maintaining the currency of its data:

Double posting or posting updates to the existing AutoCAD system and the new GIS system
Posting updates to the GIS, then exporting the changes back to AutoCAD
Posting changes to AutoCAD and importing the changes into the GIS using a custom tool

After deliberating and carefully considering the impacts of each strategy, it was decided that posting the changes to the AutoCAD environment, then importing the changes into the ArcSDE database was the best approach. A data synchronization tool called DWG In was developed to maintain the data in the GIS based on edits made to the AutoCAD files. Data editors in the 54 regional operating headquarters could maintain the data in the existing environment until they were trained and transitioned to the new GIS tools.

DWG In was developed as an application that could be scheduled and run without user interaction. It compares the latest version of an AutoCAD file to its previous version to determine which features have been inserted, updated, or deleted. To limit the impact on the existing processes and systems, DWG In would be a separate, stand-alone utility that would run at a scheduled frequency. An additional utility was developed that identifies and copies any files that have been modified to a specific directory. Using the changed files, DWG In can compare them to another set of files. Once the differences between the two files were detected, the remaining translations between the AutoCAD features and ArcSDE features could be performed. The edits are written to a version so users can verify the edits and post to default.

DWG In was developed in Microsoft's C# programming language using the ArcObjects API, Miner & Miner API, and the Feature Manipulation Engine (FME) from Esri business partner Safe Software. The C# application manages the translation of each AutoCAD file and handles any errors raised by the translation process. Due to the relationships between the features, both spatial and attribute data, separate translations are performed so relationships are created correctly. For example, when anchor features that snap to poles are translated, the pole is written to the ArcSDE database first so an anchor can then snap to it. Based on the features and relationships, DWG In performs four separate translations.

Another obstacle in developing the DWG In application was that Safe Software's FME product did not write ArcSDE network features. The Georgia Power data model contained an electric network with many network features maintained in the AutoCAD environment. To work around this, these features were postprocessed using FME to transfer them to a personal geodatabase. A custom tool was then used to move them from the personal geodatabase to the ArcSDE database.

Similarly, the Miner & Miner ArcFM product added extensions to each feature and object class in the data model that allowed custom code and AutoUpdaters to be executed whenever a feature or object was modified. The features and objects were written to ArcSDE through the FME translator using ArcObjects API, rather than through the ArcCatalog or ArcMap interface. The Miner & Miner extensions failed because a Miner & Miner license had not been appropriately checked out. Initially, a custom FME Reader was written to check out a Miner & Miner license so the reading and writing of the features from the ArcSDE database could occur, invoking an error dialog. However, a Microsoft patch released in July 2006 resulted in the FME translation displaying a memory reference error at the end of every translation. Since DWG In ran as a service, there was no way for a user to respond to the dialog boxes so DWG In would hang until someone cleared the dialog boxes.

In addition, it was not possible to control custom AutoUpdaters from within FME Readers—they couldn't be disabled but failed to consistently fire. The custom AutoUpdaters were not written for execution from a stand-alone application. In many cases, it was assumed that ArcMap would be running when the custom code was executed but this assumption resulted in, among other issues, error message boxes being displayed. As with the previous license issues, with no user to respond to these error dialog boxes, DWG In would hang until someone was able to clear the dialog boxes.

It then was decided that using a postprocessor to move the simple features from a temporary personal geodatabase to the ArcSDE database would help solve many of the issues. An empty personal geodatabase was developed with the same data model as the ArcSDE database except it did not have a network and none of the Miner & Miner extensions were applied to it. More columns were added to each feature and object class so the postprocessor was aware of the type of transaction to be performed with the feature.

By using the personal geodatabase with a postprocessor, the FME translations no longer required a geodatabase writer to access the database. This allows the geodatabase reader to be changed to an ArcSDE reader that uses ArcSDE to access the features rather than the ArcObjects API. Because features were now being read from the database, a custom FME Reader was not needed and any problems with Miner & Miner licenses were avoided. As an added bonus, this change resulted in a 66 percent improvement in the performance of DWG In. Files that had been taking 20 minutes to process were being processed in less than 7 minutes.

Another important design consideration was the ability to log inconsistencies or issues so that corrections could be made. With more than 100 feature classes being updated by DWG In based on edits in a relatively forgiving environment, it was possible that a translation could fail due to an invalid data type. Other possible issues arose when people moved or deleted features in the ArcSDE database without realizing the impact of these changes on DWG In.

Fortunately, the new data model provided for the ability to determine if and when the feature was modified so that potential conflicts could be logged for review. Log files, potential conflict files, failed features, and unmatched shapefiles were created to provide users and administrators with the ability to investigate potential problems and correct issues efficiently.

DWG In was deployed in October 2006 in the Georgia Power production environment only after stringent testing. More than 1,700 versions have been posted to the production database through DWG In. New processes, based on the number of edits, validation tools, and spot checks, have been implemented to ensure data quality.

The DWG In project has been a success. It enabled Georgia Power to maintain the currency of its data in the GIS during the 18-month migration process without the expense of additional resources. Further, DWG In reduces cost and resource impacts that would result with a delay in schedule. For more information regarding DWG In, contact Faye Hall at fhall@enspiria.com or visit www.enspiria.com.

About the Author

Faye Hall researches, designs, and develops software applications to integrate legacy systems with GIS for the utility industry and local governments. She performs requirements gathering and application design and works all phases of the project life cycle. Her nine years of consulting and system integration experience have included utilities and public sector work. Her background also includes engineering experience in the automotive manufacturing industry as well as general software design and programming experience in the telecommunications and railway transportation industries. She holds a bachelor of applied science degree in systems design engineering from the University of Waterloo, Canada.