ArcSDE 8.2 and Raster Data Management

North Central Texas Council of Governments' iCommunities Internet service combines high-quality aerial images with local government planning.

GIS analysis and map creation primarily use two kinds of data. Vector data represents real-world geographic features in terms of geometric points, lines, and polygons, while raster data, such as aerial photography, represents the same features as points on a grid. The images created by aerial photography, for example, are raster data.

Billions of dollars are spent annually worldwide to collect and store various types of raster data for GIS projects. This data includes digital elevation, aerial photography, digital orthophotos, and satellite imagery. With the current generation of high-resolution, hyperspectral sensors and the use of light detection and ranging (LIDAR) technology for detailed digital elevation data, unprecedented volumes of raster data are becoming available; one national agency reports that it is taking in 194 gigabytes a day in new raster data. Today this data is stored online in a variety of file formats, near-line storage, and tape silos.

The trend is toward more sophisticated uses of raster data, including land and natural resource planning, floodplain analysis, economic and community development, environmental protection, and emergency management, by a user constituency ranging from federal agencies to other government agencies, private industry, and the public. In addition, many organizations are struggling with economic pressures to provide cost-effective solutions for storing, managing, and distributing their growing data collections; at the same time, they are facing internal and external requirements for achieving greater return on their data investments by offering wider organizational and public access.

All of this adds up to a growing recognition that there is a need for a raster data information infrastructure that provides for improved storage and management and offers more efficient access and distribution. Today, organizations are searching for solutions that provide easy integration of raster data with vector-based data and for tools to effectively search these vast and growing collections of photos and images.

The results of recent software development projects have enabled Esri to become the first vendor to be able to offer a complete GIS-based end-to-end raster data management system. The combination of ArcSDE 8.2, ArcIMS 4, and the ArcGIS 8.2 Desktop application provides an out-of-the-box solution to store, manage, update, access, analyze, and distribute collections of aerial photographs, satellite images, and digital elevation models as well as other types of raster data such as scanned maps and CAD drawings. ArcSDE is a core component of Esri's raster data management system, enabling organizations to store their raster data and the related metadata in a relational database management system (DBMS) for fast, online, multiuser access to continuous raster data.

USGS's Seamless Data Map Interface provides access to the national Elevation and Land Cover Datasets.

The ability of an organization to store and manage the actual pixels and related metadata in a DBMS using standard relational tables means that now the same storage, indexing, and spatial search functions can be used to store, query, and retrieve both raster and vector data. Moreover, when an organization moves its raster data into a DBMS, it extends all of the DBMS tools for recovery, backup, security, fail-over, and replication to the raster data collection. This in turn means that just as ArcSDE first enabled organizations to store and manage their vector data in the leading relational databases, including IBM DB2, IBM/Informix, Microsoft SQL Server, and Oracle, along with the rest of their core data, ArcSDE now brings open access and information technology standards to raster data.

ArcSDE is unique in that it was designed from the ground up to store and manage large collections of raster data in a DBMS, providing a fast, scalable platform for multiuser access. ArcSDE provides support for loading all raster data formats supported by the ArcGIS Desktop applications; options for storing raster data with no compression or with data compression (recommended) using lossless, LZ77 compression or lossy, JPEG compression; "pyramids" (a series of reduced resolution representations of each raster data set) for improved display performance; and caching for fast redisplay. The combination of ArcSDE 8.2 with ArcIMS 4 provides an out-of-the-box solution for distributing raster data over Intranets and the Internet, and it also provides the core building blocks for creating e-government and e-commerce spatial distribution portals.

Even when there isn't a requirement for raster data to be stored in DBMS (i.e., with historic aerial photos or satellite image collections that are stored using binary files, near-line storage, or tape silos), the combination of ArcSDE with Esri's ArcCatalog and the ArcIMS Metadata Server provides powerful tools to allow organizations to create, manage, and publish metadata along with subsampled "thumbnail" images. They can also create image footprints that enable users to quickly search large raster data collections based on any combination of geographic extent, content type, data format, or keyword. (See ArcNews Spring 2002, Vol. 24 No. 1, "ArcIMS and ArcGIS Combine to Bring Spatial Metadata to the Internet.")

Many regional, state, and national government agencies have a responsibility for raster data acquisition and warehousing and for offering a seamless solution for searching, viewing, ordering, and delivering raster products. Typical products for these organizations include Digital Orthophoto Quadrangles (DOQs), Digital Raster Graphics (DRGs), and Digital Elevation Models (DEMs). These organizations are finding that even doing something as seemingly simple as using orthophotos as a "wallpaper" backdrop on a Web site can easily mean storing and managing 1,000 or more raster images. State and national agencies can face managing 1,000 or more DEMs per county, and when you multiply that by the number of counties in a mid- to large-size state, the number of DEMs and the resultant database size can be staggering.

Other national government agencies face the same issues, but they also have requirements for managing overlapping global collections of heterogeneous imagery (multisource, multiresolution, high-resolution raster data), high volumes of data, "always on" availability, and unique security needs. Conversely, commercial satellite and imagery Businesses have all of the raster data management needs of the above organizations plus the added requirements of global distribution and having an e-commerce engine to collect payment.

Organizations that have already begun to use ArcSDE to store and manage their raster data collections include:

U.S. Geological Survey

Elevation is one of the seven framework themes of the National Spatial Data Infrastructure (NSDI), and the U.S Geological Survey (USGS) has been designated as the agency that has lead responsibility for national elevation data. The National Elevation Dataset (NED) provides elevation data coverage for the continental United States, Alaska, Hawaii, and the island territories in a seamless format with consistent projection, resolution elevation units, and horizontal and vertical datums.

In keeping with the framework concept of easily accessible data, NED application products and the National Land Cover Dataset (NLCD), a 30-meter resolution raster land cover for the entire United States, are available through the USGS's Web-based Seamless Data Distribution System that uses ArcIMS to provide its Web portal and user interface (the Seamless Data Map Interface) and ArcSDE to store and manage the NED and NLCD raster data sets within USGS's DBMS. The USGS has also developed a Web-mapping portal that provides maps on demand using NED and NLCD with selected vector data like roads, streams, water bodies, and county and state boundaries for their Rapid Prototype Testbed.

North Central Texas Council of Governments

In 2001, the North Central Texas Council of Governments (NCTCOG) entered into a cooperative purchase with local governments for high-quality 1-ft. aerial images and digital elevation contours of North Central Texas. Combined, the two types of imagery will create a three-dimensional picture of more than 9,000 square miles of North Central Texas. Access to the GIS data created by this project, including more than one terabyte of aerial images, is available to the public via a Web site (www.dfwmaps.com). NCTCOG also provides customized and ArcIMS services to its member governments. Its Communities Internet service combines the high-quality aerial image base with local government planning, economic development, and GIS resources customized to a city's or county's needs.

In summary, a new generation of technology from Esri has the potential to change how raster data collections are managed. ArcSDE offers a new approach where raster data can now be stored and managed in a DBMS along with an organization's vector and other core data. The combination of ArcSDE 8.2, ArcGIS 8.2 Desktop applications, and ArcIMS 4 is positioned to provide the core infrastructure needed for raster data management.

For more information, visit www.esri.com/arcsde, call Esri at 1-800-447-9778, or call your Esri regional office. Outside the United States, contact your local Esri distributor.