With the release of ArcGIS Pro at version 3.4 and ArcGIS Enterprise at version 11.4, we are pleased to announce several new features that will improve the functionality and usability of ArcGIS Data Interoperability. In this article, we will look at the following enhancements:
- Web Tool Scheduling
- ArcGIS OnlineConnector & Metadata
- Supported DuckDB database
- Forward (Batch) geocoding
- Improvement on User Experience/User Interaction
Web Tool Scheduling
The introduction of web tool scheduling marks a significant milestone for ArcGIS Data Interoperability. This feature, which has been highly requested, enhances the functionality and efficiency of data management. The constant need to synchronize data between systems of record is a common usage pattern for ETL (Extract, Transform, Load). This process can now be moved from Pro machines to Enterprise servers by scheduling the synchronization using this new tool.
Not only does this new tool save time by allowing the user to schedule synchronization tasks, but it also ensures that these tasks are executed both consistently and reliably, relieving the user of the burden of manually synchronizing their data. For additional information including a demonstration on how to use the tool, please refer to our ETL Pattern: Scheduling Web Tools blog post on the Esri Community website.
ArcGIS OnlineConnector & Metadata
The ArcGISOnlineConnector transformer now supports getting and setting metadata from portal items. When working with portal items, you can now bring metadata along with the actual data using your ETL workflows. This feature is available for use with ArcGIS Pro at version 3.4, but for users still working with version 3.3, the process for using this valuable capability is outlined in our ArcGIS Blog post, Metadata – Data Interoperability’s Hidden Talent (Part Two)
This new feature improves the quality of your data by allowing the inclusion and setting of metadata, which improves the depth and breadth of data analysis. Also, ArcGIS feature services now support timestamp offset and big integer field types, bringing them into line with the geodatabase formats, ensuring that your data can flow between the two environments without issue.
Supported DuckDB database
The popular DuckDB database is now supported as a SQL-compliant database that includes reader/writer capabilities as well as the ability to create SQL queries (SQLCreator) and to execute SQL queries (SQLExecutor). Now that this database is supported at version 1.0.0, it allows the user to dynamically load extensions, like the spatial extension, into their user profile. Since DuckDB is a database and supports virtual filesystems (URL paths to cloud object stores), users can virtualize data access in real-time, making remote files work like database objects.
As demonstrated in the Esri Community blog post, ETL Pattern: Working With Cloud Well-Known Files, users can script DuckDB to read a virtualized view of data in an object store. That workflow can now be automated using ArcGIS Data Interoperability and a SQL Executor. Adding this capability to ArcGIS Data Interoperability increases the ease and accessibility of this product, making it a versatile and valuable asset for anyone’s toolbox.
Forward (Batch) Geocoding
The Geocoder transformer, which allows the conversion of addresses to latitude/longitude coordinates, now offers a Forward (Batch) geocoding method when using ArcGIS Online or ArcGIS Server geocoding services. This feature is scalable, allowing the user to improve throughput compared to processing single addresses per call. This ability to customize the batch size helps improve efficiency and eliminates time-consuming processes. Moreover, when used in combination with a custom locator view service URL you can reliably and efficiently create geometry from location information, like place names or addresses.
Improvement on User Experience/User Interaction
In an effort to improve the user experience and interaction with ArcGIS Data Interoperability, several updates have been made to the interface and apps.
In the Workbench app, typing into the canvas has an autocomplete experience that shows format reader/writers, transformers and even Hub transformers.
We have also updated the system to allow the user to filter based on the category buttons. By simply double clicking on an entry, it can be added to the canvas.
Enhancements have also been made to the HTTPCaller. It now includes a test preview in the dialog box while authoring the configuration.
The Import OpenAPI feature in previous releases of the HTTPCaller is now officially a standalone transformer called OpenAPICaller. You can select your Endpoints directly from your API Specification in YAML or JSON files. A query test control is offered in both the HTTPCaller and OpenAPICaller, so users can view returned JSON files in a tree structure for better understanding.
Conclusion
These updates enhance the capabilities and workflows of ArcGIS Data Interoperability, empowering users to explore, capture and visualize their data with ease. We are constantly working to improve this product so stay tuned for more exciting features and enhancements in the future! For more information and to get started, visit our website or contact your account representative.
Article Discussion: