Rasters: Get Speed, Save Space

In ArcGIS we generally recommend using a TIFF raster format that has been compressed using JPEG compression, to get the overall best performance and conserve disk space when using it in Desktop or serving it with ArcGIS Server. There are two ways we recommend you compress your data—you can use ArcGIS geoprocessing tools, such as Raster To Other Format or Copy Raster, or you can use the FWTools utility created by Frank Warmerdam, which uses the GDAL library.

GDAL is an open source translator library for raster geospatial data formats. Starting at version 10, ArcGIS uses GDAL to read and write its raster data.

Using the Raster To Other Format tool or the Copy Raster tool is similar to using FWTools but they have fewer options. For example, FWTools lets you define a photometric value to control the color space used to store the pixel value and define the interleaving (band or pixel). However; using the tools in ArcGIS you can create the statistics ArcGIS uses, which I don’t think is possible with FWTools.

ArcGIS 10 did introduce some options for writing raster datasets that were not in earlier versions, such as more compression options on the raster dataset and the ability to compress the pyramids.
Recommendations for TIFF file

The recommended settings for converting raster data, such as 8-bit aerial photography, to TIFF files using ArcGIS are:
– BILINEAR pyramid resampling
– JPEG compression for the pyramids and datasets
– 80% compression quality

Also, statistics don’t always need to be calculated from every pixel; therefore, you can increase the speed at which they’re calculated by specifying a skip factor. One way to identify a reasonable skip factor value is to divide the number of columns in the raster dataset by 1000 and use the quotient (integer) as the skip factor. For example 5000 columns = a skip factor of 5. (The examples below use the default skip factor.)

You could also increase the tile size for the TIFF files to 512 for each dimension.

Before you convert all your data using these settings you should resample a few to find the settings that work best for your data. For example, the compression quality could be higher or lower depending on your data and its uses. Or if you’re compressing scanned maps you may get better results with Nearest Neighbor resampling.

Using FWTools

You can apply the same recommended settings using the FWTools utility by running it at the command using the following:
gdal_translate.exe -of Gtiff -co “COMPRESS=JPEG” -co “JPEG_QUALITY=80″ -co “TILED=YES” -co “PHOTOMETRIC=YCBCR” -co BLOCKYSIZE=512 -co BLOCKXSIZE=512 Input.tif Output.tif
A simple interface to the FWTools utility has been posted on the Image Samples Gallery to help you convert your files using these recommended settings, called Compress Imagery. This tool allows you to convert an entire folder of images to JPEG compressed TIFF files.
Using ArcGIS Geoprocessing Tools

You can use the Raster To Other Format tool to convert a collection of raster dataset to JPEG compressed TIFF files. Simply identify the input raster dataset, the output location, the format, and modify the Raster Storage settings mentioned above in the Environments Settings.

You can use the Copy Raster tool to convert a folder of images by batching, creating a model, or writing a script. The Copy Raster tool provides the extra ability to modify the raster datasets by defining a different pixel type or converting pixel values to NoData.

To run this tool using a script, you can use the following:
# Import arcpy module
import arcpy, os

# Set the geoprocessing environments for the input and output folders
arcpy.env.workspace = “e:\inWorkspace”
rasList = arcpy.ListRasters()
outWs = “c:\outWorkspace”

# Define the output settings for compression
arcpy.env.pyramid = “PYRAMIDS -1 BILINEAR JPEG 80″
arcpy.env.compression = “JPEG 80″

# Run the Copy Raster tool
for ras in rasList:
arcpy.CopyRaster_management(ras, ras, “0″, “”, “”, “NONE”, “NONE”, “”, “NONE”, “NONE”)

Keep in mind that this script assumes the inputs and outputs are tiff files and will output raster datasets using the same name.

Submitted by: Melanie Harlow