Part Two: What’s new in ArcGIS Data Interoperability at Pro 3.2

There were so many things to cover in this new release, new functionality and optimizations for your no-code integrations that we needed two blogs.  In the first blog we covered new visuals, formats, and format support improvements.  In this blog we will spend time on transformation enhancements. Here we will cover:

• OpenAPI Specification
• AttributeManager
• DateTimeRounder
• UniqueIdentifierGenerator
• Data Inspector
• Compare Workspaces

Let’s take look at some highlights!

Transformations Enhancements

1. Data Interoperability is no-code technology, but you can still work with web API calls using the HTTPCaller transformer.  The transformer now supports the import of OpenAPI Specification (aka ‘Swagger’) files (JSON or YAML), allowing the transformer to pre-populate request parameter and authentication method settings.

Here is an example using the swagger file for ArcGIS Online Workflow Manager; first import the specification file using the Import button (you need to obtain the file from Esri):

Then choose an endpoint, in my case to GET the location of a job: 

Then the transformer knows I need to supply a token plus URL path values for orgId, itemId and jobId: 

I would also need to build the base domain part for my hosted Workflow Manager instance, but you get the idea.  If there was a more complex authentication and header requirement for the API the automation would stub these out for me, but in this case it’s just a token.

2. AttributeManager now supports explicit casting of field type, for example see below I have set incoming datetime data to date (without time) in three fields.  Commonly encountered formats like CSV and Excel can be tricky to infer a schema from, so this feature lets you fix the problem of “schema drift” that you might get if you let a schema be automatically handled with changing input data.

3. DateTimeRounder is new transformer now available.  Its need is explained in a few workflows.  First, binning data into time intervals, and secondly when doing change detection.  The ChangeDetector transformer is brutally strict about comparisons, so when you’re comparing streams from different systems of record, like geodatabase and feature service, and one stores datetimes in milliseconds while the other uses microseconds, then even with identical datetimes you’ll get ‘fake change’ detected, caused by precision, unless you round to a common interval.  Here is a simple example of rounding a datetime to the nearest whole day, expressed in ISO format: 2022-01-01T08:00:00 rounds to 2022-01-01T00:00:00 and overwrites the subject fields with these transformer settings:

4. UniqueIdentifierGenerator is a small but vital transformer, which in addition to its classic UUID and GUID creation capability can now supply ULID values!

ULIDs (unique lexicographically sortable identifiers) are good through AD 10889 for those long running projects!  A ULID is a 128-bit (26 characters) identifier that consists of two components: a 48-bit timestamp and an 80-bit random component. The timestamp component represents the number of milliseconds since a specific epoch (usually January 1, 1970, or “Unix time”).

The timestamp component in a ULID allows for lexicographic sorting, meaning that ULIDs generated at different times can be sorted in chronological order simply by comparing their values. This makes ULIDs useful for scenarios where time-based sorting or ordering is required, such as in distributed systems or databases.   By incorporating the timestamp component, ULIDs provide a balance between uniqueness, lexicographic sorting, and time-based information.

Here are five I made in a few milliseconds, hence the common prefix containing the time component:

01HCFNNTF5Z9N8ZRRF0ACSC6YZ

01HCFNNTF6384F8QTSDNEHB5T4

01HCFNNTF63H8604Z9Q7BBKARK

01HCFNNTF6756ZWKH72YR0CR30

01HCFNNTF6AAGKFY8AMBN5J878

5. Data Inspector now supports display of unexposed fields, such as format attributes or arrays generated from JSON queries, plus tooltip links to help documentation.

6. Compare Workspaces is a new tool.  From Workbench, in the Tools menu, select Compare Workspaces and select your editable and read-only FMW files:

Then an experience opens where you can migrate differences from the read-only to the editable workspace.

7. One of my favorite transformers is the Geocoder.  Strictly speaking as it’s a package it isn’t part of the 3.2 release, but I want to call it out.  Geocoder now supports locator views, where you can specify the geocode match type and allowed extent for Online or Enterprise geocode services, a huge quality improvement.  Here is the extent setting for one of mine, limiting World geocode service matches to Vancouver, Canada city limits:

Lastly, for coordinate system sticklers like myself, EsriReprojector now supports multi-step geographic transformations.  Enjoy ArcGIS Data Interoperability for Pro 3.2!

Check out the ArcGIS Data Interoperability page for latest updates and announcements.