When Internet connection is a limited resource, a well-designed website doesn’t perform multiple times the same request. This little adjustment can significantly reduce the time required to load and refresh a page. First-world programmers should keep this in mind, or better come to South Africa and experience it in person…

This reminds me how life forms adapt to severe environmental conditions. But it’s a wide topic.

Let’s see how you can easily do this in Python. The snippet is a generic version of a function in views.py of the GeoDjango website we are busy developing. That function caches the result of WMS requests for layer legends in a dedicated directory, assuming that the images are not changing over time.

import urllib
import os

def retrieveDataFromUrl():
 myFileName = "file.txt"
 myLocalPath = os.path.join( os.getcwd(), myFileName )

  if not os.path.exists( myLocalPath ):
    print "Downloading data"
    myUrl = "http://whatever"
    # save it where it should have been found
    urllib.urlretrieve(myUrl, myLocalPath)
  else:
    print "Reading from local file" + myLocalPath
  # then read the file...

This code only checks if the file exists. If the file downloaded in previous run is outdated, then the newer version must be downloaded. This can be a good task for a cronjob – but it’s the topic of another post

Hope this helps!

– anne