The 1.8.3 release of the ArcGIS API for Python adds a slew of new features and capabilities. Over the last few years, it has expanded and now includes several different modules and more than 2,300 methods and functions.
The ArcGIS API for Python is designed to serve a wide spectrum of users, including GIS administrators, Python developers, and data scientists. The API can be used for tasks like administering web GIS; publishing, updating, and cloning a wide variety of GIS data; building sophisticated analytical models; and automating mission critical workflows.
Framework Improvements
One of the biggest improvements performed behind the scenes was migrating the HTTP communication modules to use Requests (a popular third-party module) from a built-in urllib module. There is no change to your scripts, except that Esri can now better support some custom and complex authentication systems. For JSON parsing, Esri switched to UltraJSON, a faster JSON decoder and encoder. These changes improve support for fundamental features, such as multipart uploads, while extracting performance improvements. In addition, support for Python 3.8, JupyterLab 1x and 2x, and Pandas v1.0 was added.
GIS Administration
ArcGIS Notebooks and the Notebook Server play a key role in enabling the spatial data science capabilities of ArcGIS. The ArcGIS.gis.nb module has been enhanced, allowing you to manage instances of Notebook Server that are running and inspect their properties. This module also enables powerful workflows such as executing notebooks on demand and scheduling notebooks. Plans include support for managing different versions of these items.
There are numerous small updates and fixes to other parts of the GIS administration API. When creating users, administrators can now use smart defaults they have defined for their organizations. The new ArcGIS Online Usage Reports class allows better compilation of usage reports in ArcGIS Online, something users have been asking for.
On the security front, the API now supports authentication using API keys. This new login mechanism targets GIS developers; more information will be released in the coming months.
Content Management
The new ArcGIS.apps.dashboard module lets users create ArcGIS Dashboards apps using Python. Many data scientists want a medium that is simpler than notebooks to communicate results. This module can help them quickly compose such apps.
Content migration and cloning are important workflows for users. The new GroupMigrationManager workflow allows group administrators to migrate content from one group to another across organizations.
Long-term users of the API who are familiar with the clone_items function that is native to the Python API will find that it has been enhanced to support additional app types along with various bug fixes.
This release also adds new publishing workflows such as publishing directly from content deposited into a data store and the ability to set desired item IDs for certain workflows and environments.
On the Open Geospatial Consortium (OGC), Inc. and Free and Open Source Software for Geospatial (FOSS4G) fronts, Esri has added the ArcGIS.mapping.ogc sub-module to work with OGC layers. Another exciting development is the ability to directly read in-memory GeoPandas GeoDataFrame objects into ArcGIS. The API now supports reading GeoPackage files and newer columnar file formats such as feather. Esri is researching new patterns to increase the performance of Spatially Enabled DataFrame (SeDF) objects.
Spatial Analysis
You can now pass SeDF objects as inputs to spatial analysis tools, allowing you to easily enable data engineering workflows that use the powerful ready-to-use tools for spatial analysis in your Web GIS. Another feature is the ability to sanitize column names (i.e., cleaning column names by converting them to strings, removing special characters, and renaming columns) before saving the DataFrame.
SeDF objects now support a new renderer property to store symbology or drawing information. When reading a feature layer, the renderer will pick up the symbology defined on that web layer and utilize it to visualize the layer on a map. You can override it with custom symbols at any time.
Now almost all web tools in the Python API support asynchronous processing. Async processing allows you to submit big and long-running tasks to the server and get back a GPJob object. This essentially frees up the Python kernel so subsequent commands can run while the server keeps working on the submitted job. The job object can be polled anytime to know its status or cancel the job.
Raster Analysis
Esri added two new classes—Raster and RasterCollection—that allow you to work with a wide variety of imagery data in a seamless manner. You can read and work with local file-based raster data and publish it as imagery layers (services). When you read local raster data, the API chooses ArcPy as the processing engine. When you read an image service, it uses the ArcGIS Image Server as the back-end engine. This allows you to work with the same API, no matter where the data is stored and/or processed on the back end. New raster functions and distributed raster analytics tools for applications, such as trend analysis, multidimensional analysis, and change detection, have been added.
Deep Learning
The ability to train deep learning models for geospatial applications using the ArcGIS.learn module is a significant feature of the API. It supports more than 30 models for various advanced workflows. These models can work with a wide variety of data such as planimetric and oriented imagery, point cloud, feature, tabular, time-series data and unstructured text. This release adds models for key workflows such as edge detection, change detection, road extraction, and image translation.
A new computer-vision model allows users to go from scanned paper maps to georeferenced vector layers. This release challenges the common myth in the geospatial industry that deep learning is suitable only for imagery data by innovating with models that work on tabular and time-series data.
The FullyConnectedNetwork model allows users to feed tabular and raster layers into a deep neural network, letting it learn and model even the most complex of patterns. The TimeSeriesModel allows you to perform time-series forecasting.
The new MLModel class allows you to integrate machine learning models from the popular scikit-learn library and the new ArcGIS.learn.text submodule. which has models for natural language processing (NLP).
What does NLP have to do with GIS? For starters, you can extract addresses and location information from unstructured text, predict missing parts of an incomplete address, and autocorrect errors in addresses, making them suitable for geocoding.
To simplify the installation of the libraries and frameworks needed for deep learning, Esri has added new deep learning installers for ArcGIS Pro and ArcGIS Enterprise and a new conda metapackage for independent installation.
A significant addition around this release is the availability of ready-to-use deep learning models in the ArcGIS Living Atlas of the World. [To learn more about these models, see “Ready-to-Use Geospatial Deep Learning Models,” also in this issue.] Since these models have been previously trained by Esri, they eliminate the tedious and data-intensive task of cleaning and preparing the large volumes of data necessary to train models or procuring the hardware necessary for such tasks. Instead, you can simply download the models and perform inference.
Mapping and Visualization
The map widget in Jupyter Notebook can now link with other map widgets in the same notebook. This allows you to synchronize navigation between two or more widgets to load and visualize different layers side by side for the same extent. Once linked, any interaction in one of the linked maps will cause other maps to update. The synced navigation works with 2D and 3D modes of widgets, so both perspective and planimetric rendering of the same study area can be viewed.
In addition to static map images, live, interactive versions of a map can be embedded in a notebook. When exporting a notebook as an HTML file, maps can be embedded as static images or dynamic web maps. If a map in a notebook is not big enough, use the export to HTML function to generate a full-size web page of your map.
Local file-based raster files can be loaded on the map widget by adding the new Raster class as a layer to a map and easily visualize imagery and raster data.
JupyterLab IDE versions 1x and 2x are supported. Search for the ArcGIS extension within the JupyterLab IDE and easily install it.
Documentation
Older guides were replaced in 2020 with comprehensive, tutorial-style Notebook guides for network analysis, geoenrichment, orthomapping, and the geometry modules. These guides provide foundational information followed by a layout diagram that describes the parts of the module and how they work with the rest of the API. These guides are numbered to aid new users.
Esri has also embarked on a long-term project to improve the API reference by adding contextual snippets for parameters, cross-referencing keywords, schematic images for certain tools, and example Python scripts.
The Try-it-Live samples experience for the API has now been migrated to ArcGIS Online. These sample notebooks run in ArcGIS Online—simply log in and test-drive them.
Conclusion
The members of the Python API team are passionate about what they do and are determined to bring you a first-class Python API for geospatial analysis.
Ready to try out the new API? You can upgrade by typing “conda upgrade-c esri ArcGIS” into your anaconda prompt.
Thirsty for more details? Check out the release notes.
Ready to try out the new API? You can upgrade by typing “conda upgrade-c esri ArcGIS” into your anaconda prompt.
About the author
