Digital transformation is critical for organizations, companies, and governments. It’s how they’re adapting to a world full of new data, new challenges, and new opportunities.
One of the most important tools in an organization’s digital transformation is the ability to automate tasks. Especially when something needs to be done repeatedly, using technology to automate it saves time and resources.
At the plenary session of Esri Developer Summit 2020, the ArcGIS Enterprise team showed a number of tools that allow GIS users to automate their workflows – helping their organizations along in the journey of digital transformation.
Three levels of automation
Jay Theodore, CTO of ArcGIS Enterprise, gave an overview of automation in the ArcGIS Enterprise platform.
He spoke of three levels of automation:
- Simple automation, where tasks previously done manually are now automated without significant change in the process. This might involve automating your data validation workflow or setting up a remote check of your GIS users’ activity.
- Advanced automation ratchets up the scale and complexity of the tasks being automated. Here, you may be chaining together multiple workflows to derive greater insight or efficiency. Jay discussed distributed collaboration, which enables trusted sharing pipelines between GIS systems, as an example.
- Intelligent automation, the most complex level, feeds into decision making. Here, you are using software to inform and drive your processes. Perhaps you’re deploying a deep learning model to determine where your field workers should go, or which parts of your utility network to expand.
Each of these tiers is important for organizations to understand and consider. They feed into decisions about how you scale your system, where you focus your efforts, and how you can make the best use of the resources and time you have.
Automation comes to notebooks
When you’re doing spatial data science, you want to spend your time doing the good stuff – performing powerful analysis to unlock new insights – without getting bogged down in gathering, cleaning, updating, and validating your data.
In 2020, this will be easier for ArcGIS Notebooks users to accomplish.
With ArcGIS Enterprise 10.8, released in February, a new operation was added to the ArcGIS Notebook Server Administrator Directory. “Execute Notebook” allows administrators to automate notebook execution – to run an existing notebook without manually accessing the notebook.
While you can currently combine this operation with a scheduling tool such as “cron” to set up a recurring schedule for automated notebook execution, the ArcGIS Notebooks team is working to bring scheduling directly into the notebook editor.
In the Developer Summit 2020 plenary session, two demos showcased this upcoming feature.
Indhu Kamala Kumar and Bill Major worked with publicly available air quality data to create an automated report generator for concerned scientists.
They set up a schedule for their notebook, which included a webhook endpoint and fed into a distributed collaboration pipeline – the ArcGIS Enterprise automation trifecta – so that they could take their hands off and let the automated system run itself.
Get the full scoop on Bill and Indhu’s demo
Scootering around
Shannon Kalisky and Shreyas Shinde demonstrated how scheduling ArcGIS Notebooks can establish an automated data pipeline that regularly updates an ArcGIS Dashboard with the latest information.
Their dashboard is hydrated with real-time scooter and bike data from three companies operating in the Washington, D.C. area. This data populates a feature layer for each company. They also take city data about current events to understand when and where people may be flocking.
To make each feed match up, they deploy an ArcGIS Notebook to gather the data from each source, clean and enrich it, and then add it into their feature layers for the dashboard. They do this using the ArcGIS API for Python, which makes content wrangling and management easy and efficient.
After clearing the dashboard of all data, Shreyas went into his notebook to build the data pipeline back up. When he returned to his dashboard, he found that it was faster than he was – the data was already filling in the dashboard on the fly.
Using these tools – scheduling notebooks, distributed collaboration, webhooks – with the powerful Python functions of ArcPy and the ArcGIS API for Python, the ArcGIS Enterprise team showed how you can integrate automation tools into your workflows.
This automation can be in simple, advanced, and intelligent ways, all of which save your organization time, money, and effort while expanding what you’re able to do with your data.
Article Discussion: