What’s new with hosted feature layers: Getting to know Append Part 1 (December 2017)

Managing your hosted feature layers is an important aspect of managing data in ArcGIS Online.  For many, when it comes to adding new data in bulk to existing hosted feature layers, the typical workflow is: download a copy of the data, reconcile the new additions (and any additional edits), and overwrite the layer in ArcGIS Online. In many cases that workflow works; for others, it creates extra steps for an otherwise simple task.

One of the best examples is when working with data that has periodic updates. That’s where the new Append Data to Layer tool (Append) really shines.  With Append, you can simply insert the new data into the layer instead of exporting the layer, merging it with the new data, then overwriting.

Append allows you to upload and append new data to an existing layer from a file.  That file can be a feature class in a geodatabase, a shapefile, or in the case of point layers; CSV(be sure to include x,y fields in lat/long format; don’t worry, we’ll be working toward other coordinate formats in a future release).   Another scenario where it would make sense to append vs overwrite is when your feature layer is large.  Overwriting the entire layer when it’s very large (say millions of features) can take time.  If the number of features you want to append into the layer is proportionally smaller, it is much faster to use Append instead of overwriting.

To get started, go to the item page for your hosted feature layer.  From the Overview tab, you’ll notice a new menu button on the right called Update Data. Click it and you’ll find.  Append is also accessible from the Data tab.

After clicking Append Data to Layer, you’ll be prompted to upload the file that contains the data you want to append.  Supported formats include:

• File Geodatabase (.zip)
• Shapefile (.zip)
• CSV (.csv)

One quick note at this time you can’t append data to a hosted feature layer that has synchronization or change tracking (Keep track of created and updated features) enabled.

To save time and to reduce errors, it is strongly recommended that the schema of the uploaded file matches the schema of the hosted feature service.  Let’s look at an example.  NASA publishes a daily active fires dataset, which can be crucial in mapping fire perimeters and visualizing the day-over-day change.   By using Append instead of publishing new layers, you can store all that information in one layer, with the bonus of being able to use the time slider to visualize daily changes.  You can just download the file and append the new data daily (in the case of the active fires dataset, make sure you take a few extra seconds to define the projection before uploading). By default, ArcGIS Online will attempt to match fields to your feature layer. If you want to change that, just click Show Field Mapping and change the defaults, or unmatch fields you don’t want used.  When you’re happy with the selected options, click Apply to append the data.

You might have noticed a little check box that was skipped that said, “Update existing features”.  While Append is primarily about adding new features into a layer, it also can append updates into existing features.  I’ll cover that option in more detail in a separate blog article (Part 2). So, stay tuned for more details and examples.  In the meantime if you’re looking to read more on managing hosted feature layers you can find additional reading here.