ArcUser
Summer 2012 Edition
Imagery: Past, Present, Future
By Karen Richardson, Esri Writer
This article as a PDF.
Esri writer Karen Richardson recently sat down with Peter Becker, Esri's manager of imagery products, to understand where the industry is and where it is going. Becker was the technical manager for MAPS Geosystems in the United Arab Emirates for 15 years before joining Esri. At Esri, he worked on the development of image server technology. For the past 4 years, he has served as the product manager for imagery. He graduated with a bachelor's degree in engineering with honors in control engineering from the University of Sheffield in the United Kingdom.
Richardson: Over the past several years you have been working with imagery, you have seen a lot of changes. Is there one significant change that you've seen that stands out more than others?
Becker: It is not one but many. First, I'd have to say the move to digital sensors from the traditional scanned aerial film was a significant change. The volume, speed, and clarity of imagery that digital cameras collect is astounding. Plus, with no scratches or dust on the film to degrade the quality of the images, the processing possibilities and analysis are so much better.
Being able to acquire such a great number of images required a significant change in how imagery should be managed. Instead of viewing and working with a single image at a time, techniques were developed to make it possible to work with very large numbers of images. Naturally, the significant increase in computing power and optimization of algorithms has meant that it's now possible to nearly instantaneously process imagery into multiple products.
Richardson: You talked about imagery data management changing. In what ways has it improved?
Becker: Traditional desktop image processing software was designed primarily to work with a single or a few images at a time and could never scale to a large number of images. The integration of GIS techniques for handling large numbers of vectors, such as image footprints, databases of metadata, and process modeling techniques, enabled image management to be scaled by many magnitudes in terms of number of images as well as efficiency. Databases are used to manage the large number of parameters that define not only the properties of the imagery but the processing to be applied to it. The pixels in the images, in most cases, do not change. It is the parameters that define how to process the images that change.
The processing of imagery has also changed from a push-based model, in which each input image is transformed through a set of processes with intermediate outputs, to a pull-based model, in which the output is defined and the system selects and processes only the required pixels through a concatenated collection of image processing functions. This substantially reduces the data volumes accessed and removes the intermediate products that create bottlenecks when scaling. These new transactional image data management techniques enable image products to be created on the fly directly from the input imagery as soon as the image is available and has the highest value. Processing parameters can then be refined to create a graded product where accuracy and quality of the output improves as additional processing parameters—such as better orientation, terrain models, or color correction parameters—are obtained.
Using a GIS to produce imagery on the fly, together with the ability to manage very large volumes of data, means we do not have to wait for all the data to process to use what we need. We can view many images, zoom in to any location, and visualize the information that is there. This also allows us to quality check our work more quickly, so projects, such as an orthophoto project, that used to take months to create, now can produce images immediately. Going from the traditional input-output model to a transactional mode in which all the data is in a database and quickly accessed has changed the very manner in which large projects are done.
Using a GIS to produce imagery on the fly, together with the ability to manage very large volumes of data, means images can be produced immediately.
Richardson: Image services are one way of serving up data—what is an image service and how is it different from accessing imagery locally?
Becker: I'll use Esri's ArcGIS to give you an idea of what an image service is and split it into two parts. ArcGIS handles very large volumes of imagery in a data model called a mosaic dataset. This is a database model optimized for image data management. A mosaic dataset allows the user to ingest metadata about images into a structured database and then define processing to be applied. This processing can be as simple as cropping imagery and stretching but can also model more complex processes such as pan sharpening, orthorectification, color correction, or seam line creation. The actual processing is only applied on the fly as ArcGIS accesses the imagery and converts the base pixels into different products. The processing parameters can be changed and refined at any point in time. This is very different than the traditional way of managing imagery.
The mosaic dataset can be used directly on the desktop for simple access in small workgroups. The same mosaic datast can also be published to ArcGIS for Server, making that imagery accessible to a larger number of users. The server performs the required processing and returns only the required pixels to client applications. These services are dynamic and enable users to also change various parameters to control the processing and display order of the imagery, exposing the full information content in the imagery.
Image services also act as a catalog. Any area you query, the system not only shows you appropriate imagery but also all the metadata about each of the images, enabling users to refine what imagery is displayed based on metadata such as dates, sensor type, or properties such as viewing angle.
While dynamic image services provide access to rich imagery content, there is also a standard requirement for users to simply and quickly access the best imagery to use as a background in mapping applications. The optimum way to serve such imagery is as a map cache. This consists of large collections of small preprocessed image tiles that can be very efficiently distributed through the web. This is how most imagery that is displayed in Google Maps, Microsoft Bing, and ArcGIS Online is published. Mosaic datasets can be used to define these map caches, and then the server can generate and serve these tiles efficiently. Because they are optimized for cloud environments, there is not much load on the servers.
Richardson: Speaking of cloud environments, we hear a lot about the cloud these days. Is this affecting how people work with imagery?
Becker: From a user perspective, definitely. What most people are familiar with as background imagery in a web application are map caches that are stored and shared from multiple clouds. Pushing cached tiles to cloud storage and distribution is a simple example of using the cloud.
From a dynamic image services perspective, using public clouds is more challenging. Geospatial imagery is typically referred to by IT as "big data," and it creates a challenge for the cloud. A cloud pattern that has emerged for working with big data brings the computational power—the CPU—to the data, not the data to the CPU.
Traditional image processing patterns involve moving the imagery to processing centers. With the vast quantities of imagery, this is not practical. Now, with high processing capacity and software such as ArcGIS for Server, it is much more efficient to bring computing power to the data. The challenge is determining where and how best to store the imagery in the cloud. This will soon be resolved by multiple vendors offering optimized storage and processing options for imagery as well as the use of hybrid clouds. In hybrid clouds, much of the storage and image processing will remain on more dedicated infrastructures internal to organizations but also interface with public clouds. This will provide higher availability of cached maps in public clouds (like ArcGIS Online) created from a server running community clouds specialized for processing but accessible to the public.
Richardson: Last year, Esri provided access to the Landsat GLS [Global Land Survey] dataset as image services. This is Landsat's 40th anniversary. Are you planning anything to celebrate?
Becker: Certainly. The Landsat imagery provides phenomenal value through its ability to provide both temporal and multispectral data. Later this year, Esri will be releasing an updated version of the World Landsat services that provide simple access to the Landsat GLS dataset using image services. We will continue to improve the services by refining the parameters used to process the imagery to return better radiance values as well as color corrections and on-the-fly scan line removal. We will include the GLS 2010 data.
All these changes will further improve the ability for organizations to use Landsat imagery to compare images from different time periods to see how the world has changed. Landsat imagery allows us to view agriculture, encroachment, and the effects of natural disasters, to name a few examples. We are excited to work with USGS [US Geological Survey] to provide access to these important archives now and into the future.
Richardson: There has been talk about access to global elevation data. Can you share some information on that?
Becker: This is a project that brings together publicly available elevation data including SRTM [Shuttle Radar Topography Mission], USGS, NED [National Elevation Dataset], and GMTED [Global Multi-resolution Terrain Elevation Data] as well as samples of commercial datasets, bathymetry, and high-resolution lidar data. This is all combined into a mosaic dataset and provided as an image service from the Amazon Cloud.
Using these services, users can quickly and easily zoom to any location on the earth's surface and immediately use the best publicly available elevation data for their projects. Multiple derived products, such as rendered versions of hillshades, slope, and aspect, are directly accessible. Users can access the actual data values for further processing and, when appropriate, analyses.
The services include terrain and surface models as well as topo-bathy models that combine different sources in multiple ways. This will be of great interest to a number of GIS users—many of whom use elevation in some form—as well as application developers. We will be extending this service in the future with greater analysis capabilities. We will also provide subscription access to elevation from commercial vendors who have valuable elevation datasets.
These services provide a template and best practice for organizations wishing to implement similar systems with their own data. Organizations can get copies of these services as templates that they can then replicate and pour their own data into. This gives them access to the information content of elevation data without taking a lot of time or worrying about how to manage all the data.
Richardson: Is lidar important?
Becker: Lidar is huge both internationally and in the United States, and for a good reason. It has a number of advantages over aerial imagery. Lidar can be collected at any time of the day or night in any season and does not have the limitations of traditional imagery.
Like lidar, 3D points are being used more and more to create surfaces. Both digital cameras and lidar create massive amounts of data, so the challenges are similar. Many of the techniques for dealing with point clouds are the same as with mosaic datasets of imagery—make them accessible by providing dynamic on-the-fly processing capabilities so they can be used to create many different, useful products quickly.
Richardson: Esri has a new release of its software coming out in the next few months. Are there any exciting enhancements included in that release for imagery?
Becker: ArcGIS 10.1 is our latest release, and it has a lot of improvements for working with imagery including simplifying workflows. There are new capabilities available when working with mosaic datasets, including automated image-to-image registration, color correction, and seam line generation, that will make the management of imagery even easier. Additional capabilities, such as mensuration (for the measurement of heights from satellite, aerial, and oblique imagery), will provide more information content. Support for radar and full-motion video will increase the available data sources. ArcGIS 10.1 will also be able to directly bring in lidar LAS files as points, TINs, and rasters for visualization, editing, and analysis. This release will also include the ability to serve lidar as elevation data to desktop and web applications.
The big message is that ArcGIS provides true seamless integration of vector and imagery data, and there is no reason to continue using imagery only as a beautiful background.
Richardson: You've talked a lot about the sharing and accessibility of image services. Why is this so important to remote-sensing professionals?
Becker: It's important not only for imagery professionals but also of interest to both the general public and GIS users. Image services are a fantastic way to provide this rich image, information to large numbers of users. The human brain is amazingly adept at extracting information out of imagery. See an image and, in an instant, your brain will extract information, especially when provided with a spatial context such as another image or basemap. Imagery is important for many applications as it provides the evidence that allows people to gain confidence in their decisions.
As more and more people depend on imagery from multiple sources, at different dates and using various spectral components, understanding this enormous range of data becomes more important. Moving the CPU processing to the data will allow remote-sensing professionals to apply more analysis on the data. Image services will be a big part of the transition in the remote-sensing industry to enable true access and analysis of imagery.
Richardson: Looking ahead three to five years, if we sat down again to discuss remote sensing, what would you hope to be telling me then?
Becker: I hope by that time that geoprocessing of cloud-based imagery will have really taken off. People will be using these services to run different models, ask specific questions, and get answers from multiple cloud-based services. I would hope to be talking about all these great examples and be able to say that we are getting closer to ubiquitous image access.
