Author Archive

Python testing with Nose – even easier!

Recently, a friend pointed me to Python nose, a package providing a very easy way to build tests, based on unittest. I still have to write some code on my own to prove this, but I am too impatient to talk about it! Stay tuned for some code snippets in the next future..
Have a look at nose‘s page:

http://pypi.python.org/pypi/nose

Happy Python coding,

–anne

Final report on my experience @ Linfiniti

Time for a final report has come…

First of all, I thank Tim for all trust he had in my programming and managing capabilities, and for giving me this great job opportunity. I’m deeply thankful to the interns I mentored: Robert, Sam and Petty, the nicest people I could meet, it has been so natural to work together. They made me feel comfortable in our office with flickering internet and power outages, teaching me the African friendliness and patience :)

What I learned:

  • Django - the main project I worked on taught me how to find my way in a huge directory tree and three programming languages. After a tough beginning, I’ve been able to quickly indentify the point(s) of the code to edit and do less side-effect damages possible. Django is really a simple and powerful tool to build websites and manage backend databases.
  • OpenLayers – JavaScript turned to be a quick and flexible enough way to display and manage maps. Still have lot to learn but at least I can read and understand almost all JS code I encounter.
  • LTSP - our office ran Linux Terminal Server Project, on a common PC as server and three FitPC as thin clients. This solution has many advantages: less power consumption, data and programs in only one machine (tribute to SPOT rule), less cost than having N full-featured PCs , very good scalability – it can be easlily used in classrooms and labs.
  • Mentoring – there is a special ability required to transmit knowledge. While explaining concepts, I realised I was using most of them without understanding them fully, and I learned how to build more clear and structured lessons, and to say “I don’t know the answer” instead of inventing something… Thanks to my interns for all precious feedback!

What I planned to achieve:

  • Test-driven development – unfortunately none of us knows enough of it to quickly build tests. Moreover, the code for our webGIS builds an interactive GUI with JavaScript and AJAX. Therefore we have been reduced to test by hand, and quite often the commits broke functionality and I didn’t notice until I had time to extensively test – when the Internet allowed me to refresh the page in less than 10 minutes.
  • Extreme Programming – We succeeded in applying some of its principles: communication (even if English is mother tongue only for Tim), extensive use of the whiteboard for mind dumps and lessons, short release cycles, feedback. We would like to improve our office management integrating other valuable tips – Robert is reading “Extreme Programming Explained” and he’s really enthusiastic!

This experience has been really precious and I will never forget it.

I enjoyed a lot working in a pure open source environment, with strong belief in its values by my employer. It is too common to work for a company and relegate open source to spare time or to side activities.

Living here for three months made me realise how lucky is Europe, with all first-world facilities, and how open source is a valid alternative to the common closed-source solutions. In GIS, especially, there is little place for ESRI to claim the huge (for South African standards) licence fees, while there is a great need of GIS for land management at any level.

I strongly suggest to open source geeks to come and stay in South Africa and work at Linfiniti! It will be a great experience abroad, in a country full of beauty, humanity and hope.

Look forward to come back!!

–anne

OpenLayers Progress Indicator meets CSS

When the OpenLayers map of your site takes some time to load and you wish to give feedback to the user, the first place to look is the Loading Panel Addin. This addin provides an animated bar, that progresses as soon as a layer is loaded on the map. But you are not bound to its standard appearance – CSS offers you an undefined customisation freedom.

I first chose a simple progress bar, with a little globe image added to it for each loaded layer, and I felt happy enough. But I’m still a CSS beginner and moreover a very cold fan of GUIs, unlike Tim, who got a nice nice animated gif to show the progress of the map population. The AjaxInfo website provided a wide range of indicators – have fun and be sure to add it to your bookmarks!

Here are the relevant code snippets of our implementation:

1- Include the LoadingPanel Addin javascript into the head of your HTML page:

<script type="text/javascript" charset="utf-8" src="LoadingPanel.js"> </script>

2- Add the functionality in mainJavaScriptFile.js:

var loadingPanel = new OpenLayers.Control.LoadingPanel();
map.addControl(loadingPanel);    

//show the control
loadingPanel.maximizeControl();
// load your layers here
// remove it as the above function returns
loadingPanel.minimizeControl();

Note: that way, the LoadingPanel doesn’t know how many layers you are loading. The wiki page and the related example do, with much more code. I liked that 5-line implementation – and the animated gif required only a little…

3- CSS tweak

.olControlLoadingPanel {
    background-image: url("/static/images/ajax-loader.gif");
    background-color: none;
    position: relative;
    width: 50px;
    height: 50px;
    background-position:center;
    background-repeat:no-repeat;
    display: none;
}

That rocks heh? Hope this can be helpful to the OpenLayers users community!
–anne

Django’s ForeignKey and inheritance limitations – solved

Last week I had to cope with one of the (few!) limitations of Django ORM model about inheritance. For the GeoDjango website we are developing, we created a model that stores user-layer pairs, so that a logged user can restore the state of map and legend as [s]he left it at previous login, and anonymous users receive a default list of layers. Here is the code for the model:

class UserWmsLayer( models.Model ):
  """ Stores custom user preferences for a layer. """
  wmslayer = models.ForeignKey( WmsLayer )
  user = models.ForeignKey( User )
  is_visible = models.NullBooleanField( null=True, blank=True )
  is_deleted =  models.NullBooleanField( null=True, blank=True, default=False )

The Layers in Legend can be of WmsLayer class or any of its subclasses, at the moment the only subclass is DateQueryLayer, a WMS layer with a filter on the date attribute. We also have a Layer class, that is an ABC (Abstract Base Class), but it’s not possible to create a ForeignKey to an ABC as it has no table in the database. So we decided to create the ForeignKey to WmsLayer as a working, half-hack solution.

That worked nicely until I relied on inheritance and method overriding. I expected Django to store the ForeignKey to the actual class of the object, but the ORM stores it as a reference to the superclass, in our case of WmsLayer. This is very well explained in enlightening answer #4 of this stackoverflow QA.

In few words, I can actually have a User-DateQueryLayer pair, but in the model it will be stored as WmsLayer, and all methods I call will be WmsLayer‘s, not DateQueryLayer‘s as I would expect from OO programming. DateQueryLayer redefines asOpenLayer method, but due to this ORM limitation I could never call it.

The solutions were multiple, more or less Pythonic, Django-ish, brittle and scalable. I took two days to google and get through them, and the only two applicable solutions seemed to be:

  1. Run Time Type identification via a ifelse cascade: that would be the less scalable and most brittle hack.
  2. Django’s Generic Relations: better, but still overcomplicated.

I was about to implement Generic Relations when I realised that asOpenLayers() simply returns a JavaScript string. A simple solution came to me in a flash. Instead of creating the JavaScript code on the fly every time the layer is used somewhere, I decided to store that string into an attribute owed by Layer, the abstract superclass, so that all subclasses will simply inherit it and populate it at instantiation time using their own asOpenLayers method. The User-*Layer pair will therefore rely on an attribute in WmsLayer table without any inheritance issue.

Here is what the relevant code looks like:

class Layer( models.Model ):
  """This is an ABC (Abstract Base Class) for all models that are layers.
  It provides common api so that all layers can be treated in a similar way."""
  name = models.CharField( max_length=256 )
  owner = models.ForeignKey( User, related_name = 'owner' )
  # store the javascript as *attribute*, instead of generating it on-the-fly
  as_open_layer = models.TextField( )

class WmsLayer( Layer ):
  url = models.URLField( max_length=1024, verify_exists=True )
  layers = models.CharField( max_length=256 )
  # link to user-layer model to keep status of legend
  users = models.ManyToManyField(User, through='UserWmsLayer')

  def asOpenLayer( self ):
    """Return a string representation of this model as an open layers
    layer definition. The created layer def will be added
    to the openlayers map of name theMap (which defaults to "map". """
    return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    """ Overrides standard save, generating the asOpenLayers javascript
    and storing it into as_open_layer attribute. """
    self.as_open_layer = self.asOpenLayer()
    super(WmsLayer, self).save( *args, **kwargs)

class DateQueryLayer( WmsLayer ):
  """A layer model for storing user date range queries persistently"""
  date_query_type = models.ForeignKey( DateQueryType )
  sensor = models.ForeignKey( Sensor )
  start_date = models.DateTimeField( null=True, blank=True )
  end_date = models.DateTimeField( null=True, blank=True )

  def asOpenLayer( self ):
  """ Overrides WmsLayer's method. """
  return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    self.as_open_layer = self.asOpenLayer()
    super(DateQueryLayer, self).save( *args, **kwargs)

That’s it.

Once you filter the UserWmsLayer table by user, you won’t call anymore the asOpenLayer method…

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.asOpenLayer()

but fetch the as_open_layer attribute instead!

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.as_open_layer

This allows you to pick up the correct string, without worrying about which type myObject is. This is good. Clean. Scalable. Pythonic. Yes, the asOpenLayer method simply returns a string according to some Layer properties, this solution could not work with more complex methods. But if you face a similar inheritance issue, and your method returns something that can be stored in an attribute of the superclass, that’s one of the cleanest solutions I’m aware of.

Hope this helps, and better solutions are most welcome :)
Happy coding,
–anne

Fancy map toolbar for OpenLayers

Are you looking for giving your OpenLayers map controls a cool appearance, smoothly integrated with the site’s theme, without writing a papyrus and scatter code among lot of files?

Then have a look at jQuery UI CSS framework, a system of classes developed for jQuery UI widgets.

This is the map toolbar of the webGIS site we are busy developing, rendered with UI-Darkness theme:
Fancy OpenLayers control toolbar

The controls (pan, measure and zoom) are OpenLayers’ controls. They are all created in the map’s init() (see first js snippet below). The binding with the buttons is made by name – therefore be sure that the names of the OpenLayers controls match exactly the name properties of the respective buttons. The activation of the selected control is done by the toggleControl() function, further below in the js snippet. That way you can add as many control-button pairs as you need.

Let’s see what the code looks like. It is not so much indeed.. My tribute to the proverbial programmer’s laziness and to the koan of Master Foo’s and the Ten Thousand Lines.

First be sure to include all necessary scripts in the html head:

<link type="text/css" href="/static/css/jquery.fancybox-1.2.6.css" rel="stylesheet" media="screen" />
<script type="text/javascript" src="/static/js/jquery-1.3.2.min.js"></script>
<script type="text/javascript" src="/static/js/jquery-ui-1.7.2.custom.min.js"></script>
<script type="text/javascript" src="/static/js/jquery.fancybox-1.2.6.pack.js"></script>
<script type="text/javascript" src="/static/js/jquery.easing.1.3.js"></script>

then add the buttons – I took inspiration from Filament Group excellent post:

<div id="mapcontrols" class="fg-buttonset fg-buttonset-single ui-helper-clearfix">
<button name='navigate'class="fg-button ui-state-default ui-state-active ui-priority-primary ui-corner-left" >Navigate</button>
<button name='line' class="fg-button ui-state-default ui-priority-primary">Measure line</button>
<button name='polygon' class="fg-button ui-state-default ui-priority-primary">Measure area</button>
<a href="#" name='zoomin' class="fg-button ui-state-default fg-button-icon-solo" title="Zoom in"><span class="ui-icon ui-icon-circle-zoomin"></span> Zoom in</a>
<a href="#" name='zoomout' class="fg-button ui-state-default fg-button-icon-solo ui-corner-right" title="Zoom out"><span class="ui-icon ui-icon-circle-zoomout"></span> Zoom out</a>
</div>

And in the end the JavaScript – add the following snippet in the map’s init() function:

    mapControls = {
	line: new OpenLayers.Control.Measure(
	    OpenLayers.Handler.Path, {
		persist: true
	    }
	),
	polygon: new OpenLayers.Control.Measure(
	    OpenLayers.Handler.Polygon, {
		persist: true
	    }
	),
	zoomin: new OpenLayers.Control.ZoomBox(
	    {title:"Zoom in box", out: false}
	),
	zoomout: new OpenLayers.Control.ZoomBox(
	    {title:"Zoom out box", out: true}
	)
    };

    var control;
    for(var key in mapControls) {
	control = mapControls[key];
	control.events.on({
	    "measure": handleMeasurements,
	    "measurepartial": handleMeasurements
	});
	map.addControl(control);
    }

and these functions at the bottom of your js file:

function handleMeasurements(event) {
    var geometry = event.geometry;
    var units = event.units;
    var order = event.order;
    var measure = event.measure;
    var element = document.getElementById('output'); //TODO redirect to other area?
    var out = "";
    if(order == 1) {
	out += "Measure: " + measure.toFixed(3) + " " + units;
    } else {
	out += "Measure: " + measure.toFixed(3) + " " + units + "2";
    }
    element.innerHTML = out;
}

function toggleControl(element) {
    for(key in mapControls) {
	var control = mapControls[key];
	//alert ($(element).is('.ui-state-active'));
	if(element.name == key && $(element).is('.ui-state-active')) {
	    control.activate();
	} else {
	    control.deactivate();
	}
    }
}

$(function(){
    //all hover and click logic for buttons
    $(".fg-button:not(.ui-state-disabled)")
    .hover(
	function(){
	    $(this).addClass("ui-state-hover");
	},
	function(){
	    $(this).removeClass("ui-state-hover");
	}
    )
    .mousedown(function(){
	$(this).parents('.fg-buttonset-single:first').find\
            (".fg-button.ui-state-active").removeClass("ui-state-active");
	if( $(this).is('.ui-state-active.fg-button-toggleable, \
            .fg-buttonset-multi .ui-state-active') )
	    { $(this).removeClass("ui-state-active"); }
	else { $(this).addClass("ui-state-active"); }
    })
    .mouseup(function(){
	if(! $(this).is('.fg-button-toggleable, .fg-buttonset-single .fg-button,  \
            .fg-buttonset-multi .fg-button') ){
	    $(this).removeClass("ui-state-active");
	}
	//TODO use this else only for measure/pan toggle.
	else {toggleControl(this);}
    });
});

Ok, should be all you have to know to set up the toolbar! Feel free to reuse the code and improve it :)
Oh, and don’t forget to tweak the CSS to get the perfect look and feel ;)

–anne

Saving bandwidth using Python

When Internet connection is a limited resource, a well-designed website doesn’t perform multiple times the same request. This little adjustment can significantly reduce the time required to load and refresh a page. First-world programmers should keep this in mind, or better come to South Africa and experience it in person…

This reminds me how life forms adapt to severe environmental conditions. But it’s a wide topic.

Let’s see how you can easily do this in Python. The snippet is a generic version of a function in views.py of the GeoDjango website we are busy developing. That function caches the result of WMS requests for layer legends in a dedicated directory, assuming that the images are not changing over time.

import urllib
import os

def retrieveDataFromUrl():
 myFileName = "file.txt"
 myLocalPath = os.path.join( os.getcwd(), myFileName )

  if not os.path.exists( myLocalPath ):
    print "Downloading data"
    myUrl = "http://whatever"
    # save it where it should have been found
    urllib.urlretrieve(myUrl, myLocalPath)
  else:
    print "Reading from local file" + myLocalPath
  # then read the file...

This code only checks if the file exists. If the file downloaded in previous run is outdated, then the newer version must be downloaded. This can be a good task for a cronjob – but it’s the topic of another post ;)

Hope this helps!

– anne