Date Tue 31 May 2011

Let's say you've got a lot of raster files you want to host on your map server, or even just one large backdrop. I needed to do the latter, but either way, you're faced with the same problem: how can I reduce file size and access time? Whatever else I do - such as building pyramids - my raster file is going to be compressed some way or another. I can't very well have visitors waiting for my server to run through an elaborate (de)compression method, but neither can I store images of the whole of South Africa and the world at several GB per file without perhaps running out of space.

Depending on the amount of space I have on the server, I might be prepared to sacrifice some speed to save space, or vice versa. So what's the best compression method for my requirements?

I used GDAL's translate utility (more info here) and went through several of its options to get a better idea of what's on offer. For compression, my bash command basically looked like this:

gdal_translate <compression options> -of GTiff input.tif output.tif

To test response times, I ran these commands to export the compressed file to thumbnails of various sizes:

time gdal_translate -outsize 80 60 -of GTiff output.tif thumb-output.tif
time gdal_translate -outsize 800 600 -of GTiff output.tif thumb-output.tif
time gdal_translate -outsize 1280 1024 -of GTiff output.tif thumb-output.tif

(In retrospect, it may have been better to remove the thumbnail between each step rather than overwriting it, to better replicate conditions. Still, I don't think it made much of a difference.)

My results are summarized in the graphs below, using the following labeling scheme:

  Name: Compression options
  N/A : -co COMPRESS=NONE
    A : -co COMPRESS=LZW -co PREDICTOR=1
    B : -co COMPRESS=LZW -co PREDICTOR=2
    C : -co COMPRESS=PACKBITS
    D : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=1
    E : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=1
    F : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=6
    G : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=6
    H : -co COMPRESS=DEFLATE -co PREDICTOR=1 -co ZLEVEL=9
    I : -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9

|Fig. 1 [image]|

|Fig. 2 [image]|

Going by these results, you can choose whichever compression best suits your requirements. For the shortest access times, you could use "packbits"; however, this also gave the largest (compressed) file size. For the smallest files size, "deflate" with a predictor of 2 and zlevel of 9. For a compromise, you could consider "deflate" with a predictor of 1 and zlevel of either 6 or 9.

Obviously, there are more permutations of this; you could include pyramids as well (use gdaladdo), for example. Or you may be willing to consider JPEG compression if you don't mind the quality. In any case, I hope this article gives you ideas on where to look!


Comments

comments powered by Disqus