Scene Layers and the I3S specification at work across the ArcGIS Platform

New capabilities for cross-platform 3D data distribution and visualization

One of the most exciting developments at the 2016 Esri User Conference was watching the sudden awareness in our user and partner community that we are succeeding in our efforts to roll out cross-platform 3D capabilities in ArcGIS.  At the UC, users and partners saw that in a single annual release cycle we released the ability to share and visualize a new data type, integrated meshes.  While we will continue to improve the capability of the new integrated mesh layer, enabling ArcGIS users to view and interact with mesh data across our web, Runtime, desktop and enterprise software was an integral step in helping the GIS community to use data from drones, aircraft, and satellites in 3D.

Our ability to deliver the new integrated mesh capability was possible because of the 3D framework underlying Esri’s Web GIS infrastructure that supports common patterns for creating new layer types. Together, Scene Layers and the I3S (Indexed 3D Scene Layer) specification have become the enabler for Esri to release new 3D capabilities such as the integrated mesh layer. After several years of research and development, Esri released I3S under Creative Commons licensing as an open specification in April 2015.  The goal for I3S was to create a mechanism for cross platform distribution of 3D content for visualization and analysis that satisfies field, web, and enterprise use cases.

By releasing I3S as an open specification, partners and users can explore how data and services implementing the specification are used by client applications across the ArcGIS platform.  We’ve already worked with partners who have provided feedback and input that have contributed to the development of scene layers and we expect more collaborative community interaction to drive the development of I3S into the future.  We also hope that, by exposing I3S as an open specification, data providers will be able to create content that will be immediately usable by their customers across the ArcGIS platform in whole new ways.

Scene Layers and I3S at your fingertips

With a few mouse clicks anyone with a web browser or with the free ArcGIS Earth desktop app can access and explore scene layers. For example, by searching for “scene layer” in ArcGIS Online, a user will discover publicly available web scenes and scene layers, shared by the ArcGIS Online community that can be immediately viewed in any modern browser.  These scene layers are based on streaming web services that follow the I3S specification and can be accessed from web browsers and from apps on mobile devices.

In ArcGIS Earth, a user can find scene layers by using Earth’s Add Data interfaceto browse and search content in ArcGIS Online. The web scene viewer in ArcGIS Online or ArcGIS Earth can be used to quickly combine scene layers and other content to tell a story or start an analysis.

Scene Layers and I3S are essential to WebGIS

The ArcGIS platform is an integrated system for creating, curating, and distributing data across enterprises and organizations of any size and complexity.  The platform enables users to work with higher value information products, such as web maps, web scenes, web layers, and web apps that may either be used individually or that may be combined and reused in other apps including in desktop and mobile experiences.

I3S was conceived because 3D content is often much larger than 2D content.  3D models of buildings, for example, may be massively more complex and take up much more storage space than 2D building footprints for the same buildings.

Within the Web GIS architecture, scene layers based on I3S represent a pattern for storing and accessing caches of feature, geometry and attribute information specifically designed to enable large, heterogeneous 3D datasets to be streamed and accessed quickly either over the web or locally on a device.  Scene layers capture the geometries, textures, and attributes or continuously varying fields that of the assets they represent.  Scene layers also capture multiple Levels of Detail (LOD) along with graphics system instructional information such as spatial indices, coordinate system information (global, Cartesian, and vertical), and even explicit LOD display information based on screen display parameters.

Web Scenes and scene layers were introduced in concept in late 2014 with the release of ArcGIS Pro 1.0 and the corresponding update of ArcGIS Online, so they are relatively new in Esri’s ecosystem.

In ArcGIS platform implementations of I3S and scene layers, we are actively working toward enabling scene layers to have the logically equivalent capability of any other layers in the Web GIS.  Layers, including scene layers, should be able to be stylized, analyzed, viewed, combined in apps and maps, and even modified.  While the patterns for maintaining and managing scene layers and their associated caches are still being incrementally implemented across the platform, our direction is clear.  We recognize, for example, that incremental update and linking of scene layers more directly to source data are key enablers for users who are working with large sets of 3D data.

Inside a Scene Layer

Scene layers are based on the I3S streaming specification.  The I3S web service represents all aspects of a  scene layer including geometry, attributes and optional textures in a manner that allows applications to rapidly accessing the portions of the layer that are needed for display and analysis.  Scene layer Items in the portal information model reference the I3S web serviceand add additional stylization information.  The information model provides for a browseable and searchable repository of scene layers and other Items.

I3S was designed specifically for rapid streaming and access to large caches of 3D content.  Within an I3S cache, data are organized in a hierarchical node-based structure that subdivides content according to the pattern that best fits the data type being represented.  The hierarchy of node parent-child relationships captures the progression from lowest to highest LOD representation of the data, which may represent  features (entities) or characterized locations (meshes or point clouds).  For example, in the case of large integrated surface meshes derived from photogrammetry, data providers can use lower LOD parent nodes to store generalized versions of the mesh in which both imagery textures as well as vector mesh geometry are generalized representations of the more detailed data contained in the highest LOD leaf nodes.

Node hierarchies may be heterogeneous.  Based upon spatial clustering within a highly distributed dataset, some sparse areas may require less LOD differentiation than others and this is accommodated in the specification.  I3S was built for extremely large datasets – millions of globally distributed points, dense cities with millions of textured 3D assets, or gigabytes of textured mesh information – and the flexibility of the node-based hierarchy and index definition accommodates wide variation in data distribution.  Content providers can take advantage of optional capabilities to tune the LOD display and visibility behavior that best fits their data.

Each I3S node can contain information including references to child nodes, geometry, attributes, and image textures.  The geometry information or payload, representing 3D models, integrated mesh data, points or point clouds, can be specified either in a simple JSON format or in binary formats depending on the volume and intended usage.  Payload information may include variants to allow the scene layer to perform better on different platforms or devices.  Currently I3S supports optional S3 texture compression, for example, which allows faster display of imagery texture information on Microsoft Windows devices.

Multiple Layer Types

I3S allows for the definition of multiple layer types with associated profiles that can be used to represent different categories of data.  Currently, the I3S specification describes scene layer types and associated profiles for 3D objects, points, and integrated meshes.

3D Object Layers represent large collections of 3D models representing features such as buildings or vehicles.

Point Layers represent vector GIS points with either two dimensional or three dimensional coordinates.   Point scene layers can be symbolized using either billboarded 2D symbols, also called picture markers, or instanced 3D symbols in which each symbol is a 3D model.

Integrated Mesh Layers can be used to represent and textured triangular mesh data and are typically used to describe surfaces. Such layers are often derived from drone, aerial, or satellite imagery via photogrammetric processing. Integrated meshes may be as small as a single building or as large as entire countries or states.

Future planned I3S layer types include types for point cloud data, for 3D lines and for polygons.  We anticipate additional types to be suggested by the 3D GIS community.

Flexible Coordinate systems

One of the areas in which we’ve responded to user need is in the range and type of coordinate system information that is captured in a scene layer. I3S is able to accommodate the major modes of visual representation that users require including global visualizations as well as Cartesian 3D representations in which the surface of the Earth is ‘flat’ but vertical assets are given a height value.  In ArcGIS client applications, global views of I3S are always displayed in the WGS84 global coordinate system.  We have recently also defined a mechanism for declaring a data set’s vertical coordinate information, including units of vertical measure and settings to declare whether or not height values are defined with respect to an ellipsoidal model for the Earth’s surface or with respect to a gravity defined surface (‘mean sea level’).

Usable everywhere through REST interfaces or as Scene Layer Packages

Another important facet of scene layers is that they can be delivered to web, mobile and desktop clients across the ArcGIS platform using a number of different patterns.

Most users will interact with scene layers streamed out of ArcGIS Online or from an ArcGIS Server implementation.   In these cases the cache for the scene layer resides on the server and is returned to clients via the I3S RESTful interface that exposes web addressable resources for the scene layer, its nodes, and their associated resources (geometries, attributes, textures).

Some users will also interact with a scene layer delivered to them as a single large scene layer package that can be locally stored and accessed.  Currently, ArcGIS Earth and Esri’s new Drone2Map are able to view local scene layer packages and ArcGIS Earth can consume and display hundreds of GB of information in the scene layer package format.

We plan to evolve scene layer packages as needed to store new layer types as well as to meet the needs of optimized display and use on different mobile devices. We look forward to receiving feedback on other aspects of I3S distribution and use from our partner and user community.

Publishing Scene Layers as a standard GIS user

Any current licensed user with ArcGIS desktop can create and share scene layers based on the I3S specification.  The minimum configuration that would be needed is the current release of ArcGIS Pro and an active ArcGIS Online subscription.  For example, an ArcGIS Pro user could take building footprints for any area they have content for, including publicly available content, extrude the footprints using ArcGIS Pro to create 3D objects (stored in a multipatch feature class) and then create a scene layer package from the results of the extrusion.  The process of creating the scene layer package generates all of the necessary resources that are part of the scene layer’s cache and packages them into a single file that supports direct access to the individual sub-resources within it.  The scene layer package can then be uploaded to ArcGIS Online and published as a scene layer item making the buildings immediately accessible in the web scene viewer, Web AppBuilder, Story Maps, and ArcGIS Earth.

Beyond this basic pattern of using ArcGIS Pro and ArcGIS Online via upfront creation of scene layer packages in ArcGIS Pro, ArcGIS users can also publish 3D information directly to their ArcGIS Server with cache generation occurring on the server.  Desktop users can also use Python to script and automate I3S publishing processes.  Expect to see additional ways to create scene layers within the ArcGIS platform based on the different patterns for creating and using 3D content.  For example, CityEngine recently released the ability to create scene layer packages and we are also exploring the ability to create scene layers from existing GIS feature layers already hosted in ArcGIS Online.

Consuming I3S across the platform

Scene layers have been designed to be consumed by web, mobile and desktop clients.   Esri has built three distinct technology implementations for consuming scene layers based on I3S into the ArcGIS platform and leveraged these in different ArcGIS 3D applications.  These implementations allow customers to consume I3S content anywhere in their organization depending on their expertise and desired end user experience.  ArcGIS Pro, in addition to being the primary authoring tool for creating 3D content, is a first class consumer of content.  An ArcGIS Pro user may connect to ArcGIS Server or ArcGIS Online, find scene layers that are private in their organization or shared publicly, and use those layers to create scenes, layouts, and even animations.  Scene layers can be used in rich desktop workflows such as high quality video creation in ArcGIS Pro:

The ArcGIS API for JavaScript also includes the capability to access and display 3D content from scene layers.  The JavaScript API ships with both ArcGIS Server and ArcGIS Online.  Scene layers can be used with the web scene viewer, Story Maps, and Web AppBuilder, all of which are based on the same JavaScript API. Currently JavaScript access is read-only, but we have plans to implement read and write access to feature services and other layer types.

We have also implemented the ability to consume scene layers in the ArcGIS Runtime libraries.  ArcGIS Earth and Drone2Map are built on the ArcGIS Runtime SDK for .NET, version 10.2.7, and are great examples of 3D applications built on Runtime that work with I3S, KML, and other 2D and 3D content.  Developers can explore the ArcGIS Runtime to see how to add scene layers into their own custom 3D apps, including iOS and Android apps through beta versions of the ArcGIS Runtime that are now available.

ArcGIS Earth and Drone2Map are fully supported apps that work natively with I3S content.  Any ArcGIS user with access to Drone2Map can process drone-collected imagery and generate integrated mesh SPKs that may be uploaded and published to ArcGIS Online.  ArcGIS Earth can be used to view the resulting scene layers in ArcGIS Online or scene layer packages may be dropped directly into ArcGIS Earth and viewed as local files.

Content creators and I3S

One of the reasons that we released I3S as an open specification was to empower content creators to generate output that would work across the ArcGIS platform.  We recognize that many new 3D data formats and collection methods are gaining rapid adoption in the GIS industry.  Providing the tools to allow content providers to reliably share their content to our large community of GIS users is an essential part of any content business model.  This spring we were able to work with three different data providers – Vricon, Bentley Systems, and Pix4D – who started to create I3S in that can be consumed by end-users.

Vricon

Vricon uses satellite imagery and machine learning algorithms to create detailed textured meshes of large areas of the globe.  Typically working in data chunks that may cover 10,000 km² and that may be tens-of-gigabytes in size, Vricon needs data formats that will help them to deliver data rapidly to a variety of devices and that can be instantly usable by a large segment of the GIS market.  With I3S, Vricon is now able to deliver integrated mesh scene layers as packages and as streamed content from ArcGIS Online.

Pix4D

Pix4D creates and licenses software for creating textured mesh data and point clouds from imagery.  Pix4D technology creates 3D meshes from drone-captured imagery and provides their own products as well as OEM software for partners.

Bentley Systems

We were very excited to hear that Bentley Systems’ ContextCapture team had included I3S in their product release this year.  ContextCapture is a software product that allows users with large volumes of sensor-based imagery to process the imagery and create detailed textured meshes.

We are looking forward to working with many more partners who have started to investigate scene layers and we anticipate that some of the future scene layers that we are planning to share will expand our partners’ opportunities even more.

How to find out more

As a user, the best way to find out more about I3S is to try it by exploring scene layers in ArcGIS Online or by downloading ArcGIS Earth and adding scene layers from ArcGIS Online or the Living Atlas.  We have shared many sample scene layers that are available in the Living Atlas in this blog.

Some links to explore data shared with I3S that we would recommend include:

• One million 3D buildings from New York City
• Phototextured San Francisco 3D buildings from Pictometry and PLW Modelworks
• Vricon integrated mesh example
• Bentley Systems integrated mesh example
• Geodesign story map
• South Boston Zoning code story map
• Manhattan Air Rights viewer – custom JavaScript app

For interested data providers as well as anyone interested in the technical details of scene layers and I3S, we have updated the publicly accessible open specification for I3S.  We encourage data providers and content creators to explore the spec, provide feedback, and let us know how we can improve the specification to meet customers’ needs.  Developers can also find out about I3S on the ArcGIS for Developers site.

Where we’re going next

A look at the capabilities of I3S across the ArcGIS platform today reveals that we have focused initially on visualization and distribution.  As we move into the future, we plan to introduce new 3D data types as well as improve the capability of the existing scene layers to gradually add more analytical, management, and visualization capabilities.  The 3D GIS market is growing rapidly and we intend I3S to evolve to support new technologies, workflows, and even interaction modes such as Virtual Reality, Augmented Reality, and 3D printing.