Introducing Cloud Data Warehouse Support

With the release of ArcGIS Pro 2.9 and ArcGIS Enterprise 10.9.1, you can now establish database connections to the following cloud data warehouses to explore, visualize, and share data within your organization.

Cloud data warehouses are databases architected to handle large volumes of structured data that are accessible in public clouds as managed services. Cloud data warehouses are being used across a variety of departments and industries to make use of the following advantages:

• Low total cost of ownership
• Improved speed and performance
• Improved data access and integration
• Scalability and elasticity

Cloud Data Warehouse support in ArcGIS Pro

Now, GIS users can take advantage of all the benefits of a CDW with the ability to access and visualize the data on a map and perform analysis.

With this release, we’ve added database access support for BigQuery, Snowflake, and Redshift. This allows you to access these cloud data warehouses in ArcGIS to visualize and analyze your data using advanced rendering techniques.  You can explore data for spatial and non-spatial patterns through many tools.

Connect, explore, and visualize your CDW data

The Database Connection dialog now supports establishing connections to these cloud data warehouses. Create a database connection to view and explore the objects in a cloud data warehouse from the Catalog pane in ArcGIS Pro.

Create a Query Layer

You can perform analysis against CDW data using native DBMS SQL capabilities using query layers. The New Query Layer dialog has several enhancements that you’ll notice when working with cloud data warehouses.

• • List of Tables – By default, the list of tables is not displayed in this dialog. It is assumed that most queries will be constructed in the console for the cloud data warehouse. If you need access to view tables, you can check this on.
  • Create materialized view – You have the option to create a materialized view for the query layer to reference. This is created during the query layer creation process. Using materialized views improves map interaction performance. For BigQuery, you can decide how long will the materialized view is stored by setting an expiration. After it expires, the materialized view is automatically deleted.

Enable Feature Binning (BigQuery and Snowflake)

Feature binning is a method of feature reduction that can help visualize data trends while vastly improving the drawing performance of layers that contain thousands or millions of point features. You can aggregate features into bins using the Enable Feature Binning tool in ArcGIS Pro. After bin-enabled, you can visualize these features in the map view and interact with the bins by zooming in and out using different scales. You can also adjust the bin size and scale threshold. Since BigQuery and Snowflake use static feature binning it is important to ensure the bin cache is kept up to date as the underlying data changes.

Share data from cloud data warehouses to your organization

You can share data from your cloud data warehouse to your organization as a map image layer. Let’s review some of the important settings to be mindful of during the publishing process.

• On the General tab of the Share As Web Layer pane confirm that Map Image is checked under Reference registered data.
• Use the Configuration tab of the Share as Web Layer pane and click on Configure Parameters and review the Data Source section. When cloud data warehouse datasources are detected in the map, additional options are available for how the service will access the data.

Data Source options

The first radio option allows you to pick whether the data will be accessed directly or by using a materialized view.

• Access data directly – Use this option when the query can’t be materialized or in cases where the result set is relatively small. The service references data in CDW that was used in query layer SQL expression.
• Materialized view – Use this option to create a materialized view during publishing for the service to reference. The materialized view is managed through the life of the service and is deleted when the service is deleted.

The second radio option will create a snapshot for the query. A cache of query layer results are copied to the relational datastore on the hosting server for the service to reference. If your ArcGIS Enterprise isn’t running in the same cloud environment as the cloud data warehouse, creating a snapshot can improve rendering performance. Snapshots are also recommended with continuously updated data is not needed. Snapshot can be refreshed on demand through the Enterprise portal.

Take it for a spin

This is the initial release introducing support for Cloud Data Warehouses, so be on the lookout as we continue to improve functionality and add more capabilities with upcoming releases. We look forward to all your feedback, suggestions, and questions. If you are using CDW within your organization, this is the perfect time to make the most out of it with ArcGIS Pro 2.9 and ArcGIS Enterprise 10.9.1.