GDAL: efficiency of various compression algorithms

Let’s say you’ve got a lot of raster files you want to host on your map server, or even just one large backdrop. I needed to do the latter, but either way, you’re faced with the same problem: how can I reduce file size and access time? Whatever else I do – such as building pyramids – my raster file is going to be compressed some way or another. I can’t very well have visitors waiting for my server to run through an elaborate (de)compression method, but neither can I store images of the whole of South Africa and the world at several GB per file without perhaps running out of space.

Depending on the amount of space I have on the server, I might be prepared to sacrifice some speed to save space, or vice versa. So what’s the best compression method for my requirements?

I used GDAL’s translate utility (more info here) and went through several of its options to get a better idea of what’s on offer. For compression, my bash command basically looked like this:

  gdal_translate <compression options> -of GTiff input.tif output.tif

To test response times, I ran these commands to export the compressed file to thumbnails of various sizes:

  time gdal_translate -outsize 80 60 -of GTiff output.tif thumb-output.tif
  time gdal_translate -outsize 800 600 -of GTiff output.tif thumb-output.tif
  time gdal_translate -outsize 1280 1024 -of GTiff output.tif thumb-output.tif

(In retrospect, it may have been better to remove the thumbnail between each step rather than overwriting it, to better replicate conditions. Still, I don’t think it made much of a difference.)

My results are summarized in the graphs below, using the following labeling scheme:

Name: Compression options
N/A : -co COMPRESS=NONE
  A : -co COMPRESS=LZW -co PREDICTOR=1
  B : -co COMPRESS=LZW -co PREDICTOR=2
  C : -co COMPRESS=PACKBITS
  D : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=1
  E : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=1
  F : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=6
  G : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=6
  H : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=9
  I : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9

Fig. 1: File size relative to the uncompressed image. Shorter bars are better.

Fig. 2: Access times in seconds. Shorter bars are better.

Going by these results, you can choose whichever compression best suits your requirements. For the shortest access times, you could use “packbits”; however, this also gave the largest (compressed) file size. For the smallest files size, “deflate” with a predictor of 2 and zlevel of 9. For a compromise, you could consider “deflate” with a predictor of 1 and zlevel of either 6 or 9.

Obviously, there are more permutations of this; you could include pyramids as well (use gdaladdo), for example. Or you may be willing to consider JPEG compression if you don’t mind the quality. In any case, I hope this article gives you ideas on where to look!