One of the most powerful uses of Imagery, be it satellite, aerial or street view, has been feature extraction and classification. Whether it is finding an object, like a car or plane, to classifying a structure as damaged or undamaged, finding changes in landscapes or to identify different land cover types.
This has also been a very cumbersome and difficult task with human beings having to pour through many images trying to make sense of them and being prone to error.
With the advent of AI and Deep Learning, being able to automate this process using machines to perform feature extraction to solve real world problems. Deep Learning is a type of AI like machine learning that uses neural networks with multiple layers, each being able to extract one or more unique features in an image.
With ArcGIS Pro, you can now perform the entire end to end Deep Learning workflow . Now, you may ask, what is the workflow. Well, the Deep Learning workflow mainly consists of these steps
- Generate training Data : Generating training samples of features or objects of interest so the machine can learn.
- Train Model : Use those training samples to train a deep learning model
- Perform Inference : Using the resulting deep learning model to perform analysis on new data. Inferencing is the process in which information learned during the deep learning training process is put to work detecting similar features in the datasets.
One of the most important aspects of wildlife management is knowing how many animals there are in a defined area. This information is crucial to creating regulation that ensure sustainable harvest as well as long term sustainability of the species. However given the size and remote terrain in many parts of Alaska , it is not an easy feat.
Each animal species offers its own challenges and requires different methods to track them. For the case of caribou, each year in summer time when caribou are harassed by insects, a herd groups together and escapes to the ridgelines, snowfields, and coastlines. This type of grouping behavior allows biologists to monitor caribou by flying over herds with cameras mounted in small aircraft and taking photographs of the groups. They can use these images to take a count or census of the herd.
While this might sound easy , getting these images of caribou congregation takes a while. If you want to learn more about how this is done The Alaska department of Fish and Game have documented this in detail on their site , Alaska Department of Fish and Game
Once the images are collected, they would need to count the caribou. In some herds the number of animals can be as much as 400,000 so this has been a long excruciating task, requiring lot of personnel and prone to human error. Using ArcGIS Deep Learning technology, they can achieve this in short amount of time and a lot more efficiently in 3 easy steps.
Here to showcase the technology, we are going to use one of aerial images that their airplanes captured of a big herd shown below.
Step 1 : Generating Training Data
Prior to training a deep learning model, training samples must be created to represent areas of interest – in this case we will collect caribou and classify them into adult and young.
We will do this using the Label Objects for Deep Learning tool in ArcGIS Pro. We will classify the animal into 2 classes Adult & Calf .
For this case we have collected all together 1273 Features with 870 Adults and 403 Calves
Once the training samples are collected, we then export the features to a training format that can be used to train a Deep Learning Model . Using the Export training data tab of the Label Object Tools, the samples were converted to training data. There are a number of metadata formats that can be used for Object Detection like Pascal VOC , KITTI Rectangle and more. For this case we used RCNN_Masks which is used for Instance segmentation.
Step 2 : Training a Deep Learning Model
Once we had the required training data, a Deep Learning model was trained . The Train Deep Learning Model Tool (in the Image Analyst Tools toolbox) was used to train the model. The model type used for training was Mask_RCNN with a 90/10 training test ratio.
The tool prepares the data, performs data augmentation and set the right hyper parameters to create a good model. Normalization and augmentation such contrast, brightness, rotation are automatically performed as part of the training process.
Because training can be time consuming, it’s always a good idea to first try with smaller number of epochs e.g. 10-30 epochs. We first trained the model for 30 epochs. It gave fairly good result but in order to improve the accuracy, we set the training to 200 epochs. We also wanted to stop training when model accuracy stops improving so we used “Stop when model stops improving” parameter of the tool. The tool stopped training before 100 epochs in under 2 hours.
Once complete, the model metrics were analyzed to verify the accuracy of the model. Based on the training and test loss curves, the model has performed well and there is no overfitting.
Step 3 : Perform Inference
Once the model was created, the Detect Objects for Deep Learning tool was used to perform analysis and Inference. Using one of the image mosaics, the analysis was run on one of the herd gatherings.
For inferencing we leveraged the GPU, which helped us process the data and output the result in couple of minutes. You can see below the detection and classification of adult and calf caribou to be very accurate.
Summary:
This workflow has been able to detect and classify 4,788 Adults and 2,414 Calves in our area of interest which was about 7 acres. In order to get better model accuracy, you might want to try improving your samples by including more examples and more varieties to include geographic and weather conditions. Transfer learning and data preparation greatly decrease time to train efficient models.
Using ArcGIS Pro, you can perform end-to-end Deep Learning workflows for image analysis like feature extraction and classification just to name a few.
Resource links
Deep Learning with ArcGIS Pro Tips and Tricks: Part 1
Introducing ready-to-use geospatial deep learning models
Article Discussion: