Dirty area management with the utility network

Dirty areas don’t have to be a dirty secret.  In this blog we want to shine a light on dirty area management and introduce some changes that have been made for ArcGIS Pro 2.5.  

Before we get too far, it’s important to understand why dirty areas exist and their purpose. All utility networks have a network topology that is responsible for managing connectivity and network attribute information for features. The network topology caches this information, which is then used by tracing and diagramming operations. The network topology engine also maintains connectivity between features and ensures features comply with network rules and feature restrictions for a utility network.  

What exactly are dirty areas?  

Dirty areas are used as a visual marker to detect where edits to features have taken place that require incorporation into the network topology. They are created when an edit is made to feature geometry, terminal configurations, associations, the subtype field, and attributes that are defined as network attributes. Dirty areas are also created for error features. Also, keep in mind that currently dirty areas are created during reconcile and post operations for any edit made in a user version.

Since dirty areas can be accessed as a sublayer of the utility network, you can work with these in a similar manner as any other layer: label them, symbolize them, or simply turn them on or off. Editor tracking is enabled for dirty areas and can be used to further understand which users performed specific edits at a specific date and time. 

UpdateType is a system-provided and managed field that can be used when working with symbology to help you understand the source of dirty areas in your network.  

UpdateType is a coded value domain that is used internally to track the operation that created the dirty area. Let’s take a look at the Electric_42_Updatetype domain and how it is assigned.

Don’t trust it if it’s dirty

Since network diagrams and tracing (analytics) rely on the network topology, unexpected results may be returned if dirty areas exist. If a dirty area exists, that means the current representation of the feature is not reflected in the network topology.

ArcGIS Pro has mechanisms in place to help ensure that your results can be trusted when using analytics:

• Trace operations have a Validate Consistency option that is enabled by default. This will return an error if the trace results encounter features with associated dirty areas.
• Network diagrams cannot be created when dirty areas are present that intersect with the selected features. Existing diagrams use consistency warnings and have an update process that can be followed to ensure diagrams are always up to date with the edits that are made to the underlying features.

While dirty areas aren’t going to act as barriers or prevent running a trace, given that the state of the network in the area covered by a dirty area is unknown, the results may not be reliable.

Check out the following video that helps to explain the Validate Consistency trace configuration option and how it can impact trace results:

Let’s clean things up a bit

Validating the network topology is important to maintain the consistency between features and what is stored in the network topology. The validation process can be performed in several different ways:

• Use the Validate command on the Utility Network Data tab (on the ribbon) while working in an active map
• Use the Validate Network Topology geoprocessing tool via the utility network web feature service
• Use the Validate Network Topology REST API endpoint

What does validation check?

When dirty areas are validated, the features are evaluated to ensure they comply with the feature restrictions and network rules of a utility network. If the feature passes the check, the dirty area is removed and the network topology is updated. If the feature fails the checks, an error feature is created.

Dirty areas generated from update subnetwork error features are also evaluated during validate due to changes that may have been made in the subnetwork.  These include the following  Error feature dirty areas generated from validating the network topology are skipped during the validate network topology operation. The dirty areas for these features will be reevaluated when edits are made to create new dirty areas for insert, update, or delete operations.

Validating the network topology also marks any subnetworks as dirty. This is an important piece to understand in subnetwork management.

New at 2.5 – Changes to the Validate Network Topology process with partial extent

When you validate the network topology based on a specific extent, the dirty areas that intersect the validation extent are now clipped. Previously, the entire extent for dirty area that intersected the validation extent was included in the validate process.

There are a few exceptions to this – If there are dirty areas from the following that are partially contained within the validate extent, the evaluation extent is adjusted to include the entire dirty area:

• Association dirty area
• An error feature dirty area from the update subnetwork operation

Check out the following video that helps explain changes at ArcGIS Pro 2.5 to the Validate Network Topology operation:

Putting it all to use

Now that you know more about dirty areas and how they come into play when working with a utility network, here are a few tips that may come in handy for dirty area management.

When validating the network topology based on a specific extent, it is important to be aware of the following:

• • After you validate a topology, dirty areas may still be present if the extent validated did not encompass all of the dirty areas in the network. This is important to be aware of when working with analytics such as tracing and network diagrams.
  • A feature is not guaranteed to be incorporated into the network topology until the full extent of the feature’s dirty area is validated. If there are any dirty areas associated with a feature, this will impact tracing and update subnetwork operations that use the validate consistency trace configuration option.

For longer running validate operations, use the Validate Network Topology geoprocessing tool. This tool takes advantage of asynchronous processing by using the UtilityNetworkTools geoprocessing service. This service is located in the System folder and is dedicated to utility network processes; since it runs separate from the feature service, it has its own configurable timeout (default 2 hours).

While edits are continuously being posted to the DEFAULT version, it may be handy to automate the Validate Network Topology tool at certain intervals throughout the day. You can automate this by scheduling a Python script or model to run at a specific time or interval. There is even a new way to schedule geoprocessing tools at ArcGIS 2.5 directly from the Geoprocessing pane!

Learn about a new feature at ArcGIS Pro 2.5 to schedule geoprocessing tools