class UserWmsLayer( models.Model ):
  """ Stores custom user preferences for a layer. """
  wmslayer = models.ForeignKey( WmsLayer )
  user = models.ForeignKey( User )
  is_visible = models.NullBooleanField( null=True, blank=True )
  is_deleted =  models.NullBooleanField( null=True, blank=True, default=False )

The Layers in Legend can be of WmsLayer class or any of its subclasses, at the moment the only subclass is DateQueryLayer, a WMS layer with a filter on the date attribute. We also have a Layer class, that is an ABC (Abstract Base Class), but it’s not possible to create a ForeignKey to an ABC as it has no table in the database. So we decided to create the ForeignKey to WmsLayer as a working, half-hack solution.

That worked nicely until I relied on inheritance and method overriding. I expected Django to store the ForeignKey to the actual class of the object, but the ORM stores it as a reference to the superclass, in our case of WmsLayer. This is very well explained in enlightening answer #4 of this stackoverflow QA.

In few words, I can actually have a User-DateQueryLayer pair, but in the model it will be stored as WmsLayer, and all methods I call will be WmsLayer’s, not DateQueryLayer’s as I would expect from OO programming. DateQueryLayer redefines asOpenLayer method, but due to this ORM limitation I could never call it.

The solutions were multiple, more or less Pythonic, Django-ish, brittle and scalable. I took two days to google and get through them, and the only two applicable solutions seemed to be:

1. Run Time Type identification via a ifelse cascade: that would be the less scalable and most brittle hack.
2. Django’s Generic Relations: better, but still overcomplicated.

I was about to implement Generic Relations when I realised that asOpenLayers() simply returns a JavaScript string. A simple solution came to me in a flash. Instead of creating the JavaScript code on the fly every time the layer is used somewhere, I decided to store that string into an attribute owed by Layer, the abstract superclass, so that all subclasses will simply inherit it and populate it at instantiation time using their own asOpenLayers method. The User-*Layer pair will therefore rely on an attribute in WmsLayer table without any inheritance issue.

Here is what the relevant code looks like:

class Layer( models.Model ):
  """This is an ABC (Abstract Base Class) for all models that are layers.
  It provides common api so that all layers can be treated in a similar way."""
  name = models.CharField( max_length=256 )
  owner = models.ForeignKey( User, related_name = 'owner' )
  # store the javascript as *attribute*, instead of generating it on-the-fly
  as_open_layer = models.TextField( )

class WmsLayer( Layer ):
  url = models.URLField( max_length=1024, verify_exists=True )
  layers = models.CharField( max_length=256 )
  # link to user-layer model to keep status of legend
  users = models.ManyToManyField(User, through='UserWmsLayer')

  def asOpenLayer( self ):
    """Return a string representation of this model as an open layers
    layer definition. The created layer def will be added
    to the openlayers map of name theMap (which defaults to "map". """
    return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    """ Overrides standard save, generating the asOpenLayers javascript
    and storing it into as_open_layer attribute. """
    self.as_open_layer = self.asOpenLayer()
    super(WmsLayer, self).save( *args, **kwargs)

class DateQueryLayer( WmsLayer ):
  """A layer model for storing user date range queries persistently"""
  date_query_type = models.ForeignKey( DateQueryType )
  sensor = models.ForeignKey( Sensor )
  start_date = models.DateTimeField( null=True, blank=True )
  end_date = models.DateTimeField( null=True, blank=True )

  def asOpenLayer( self ):
  """ Overrides WmsLayer's method. """
  return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    self.as_open_layer = self.asOpenLayer()
    super(DateQueryLayer, self).save( *args, **kwargs)

That’s it.

Once you filter the UserWmsLayer table by user, you won’t call anymore the asOpenLayer method…

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.asOpenLayer()

… but fetch the as_open_layer attribute instead!

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.as_open_layer

This allows you to pick up the correct string, without worrying about which type myObject is. This is good. Clean. Scalable. Pythonic. Yes, the asOpenLayer method simply returns a string according to some Layer properties, this solution could not work with more complex methods. But if you face a similar inheritance issue, and your method returns something that can be stored in an attribute of the superclass, that’s one of the cleanest solutions I’m aware of.

Hope this helps, and better solutions are most welcome
Happy coding,
–anne

22461

To which I can only say: “Wow! I didn’t know that many people use Windows!”

First let me show you my mapserver image format definitions…:

  OUTPUTFORMAT
    NAME 'AGG_Q'
    DRIVER AGG/PNG
    IMAGEMODE RGB
    FORMATOPTION "QUANTIZE_FORCE=ON"
    FORMATOPTION "QUANTIZE_DITHER=OFF"
    FORMATOPTION "QUANTIZE_COLORS=256"
  END 

  OUTPUTFORMAT
    NAME 'AGG_JPEG'
    DRIVER AGG/JPEG
    IMAGEMODE RGB
  END 

  OUTPUTFORMAT
    NAME          "AGG_PNG"
    DRIVER        "AGG/PNG"
    IMAGEMODE     RGB
    TRANSPARENT   ON
    EXTENSION     "png"
    FORMATOPTION  "INTERLACE=OFF"
  END 

  OUTPUTFORMAT
    NAME          "AGGA_PNG"
    DRIVER        "AGG/PNG"
    IMAGEMODE     RGBA
    TRANSPARENT   ON
    EXTENSION     "png"
    FORMATOPTION  "INTERLACE=OFF"
  END 

  OUTPUTFORMAT
    NAME "GD_JPEG"
    DRIVER "GD/JPEG"
    MIMETYPE "image/jpeg"
    IMAGEMODE RGB
    EXTENSION "jpg"
  END

  OUTPUTFORMAT
    NAME "GD_PNG"
    DRIVER "GD/PNG"
    MIMETYPE "image/png"
    IMAGEMODE RGB
    EXTENSION "jpg"
  END

Now lets look at how each performed in terms of rendering quality versus file size:

File Size : 10.5kb Format: RGB JPEG using GD renderer

File Size : 19.7kb Format: RGB Png using GD renderer

File Size : 11.9kb Format: RGB Quantised Png using AGG renderer

File Size : 37.6kb Format: RGB Standard PNG using AGG renderer

File Size : 37.6kb Format: RGBA Png using AGG renderer

To be fair I haven’t added the needed ANTIALIAS clauses for the GD renderer to produce anti-aliased images for true comparison with AGG. That said even GD at it’s best isn’t a patch on AGG in my opinion. Needless to say In this project I’ve gone with the Quantised AGG outputs which are almost as small as GD JPEG images but with much nicer quality.

Note: Updated 1 March to inline images rather having to click to view the full size.

This internship has given me that opportunity and exposed me to a whole range of Open Source tools, especially those related to GIS. The programme was run so that touched on a variety of sections, explaining basic concepts and applying this information to practical projects. The sections could later be built on, beyond the internship. The training also included work from live projects running at Linfinti Consulting, which was a rewarding experience because it involved interaction with clients and handing over a product at the end of the day.

The days usually began with a session from Tim or Anne explaining what we would be covering that day. They explained all the necessary theory in a down-to-earth, and understandable way, ensuring that we understood. And after that we would get busy with our clicking, typing, learning and building. They would patiently assist us when we got stuck, and help us get to the root of the issue when there were bigger problems.

Tim and Anne were both excellent, knowledgable and practical teachers with tremendous enthusiasm for Open Source GIS, which has left me inspired.

Some of the new tools and concepts I was exposed to:

• Concepts of FOSS and the FOSS GIS stack
  
• Linux operating system – some bash, basic tools and software on Linux
• PostGIS – (first time I’ve used a geodatabase through open source and seen its practicality!)
  
• Qgis – user interface, new symbology and cartography, plugins (very useful!)
• Programming - coding in Python to create our own Qgis plugins, coding standards, SVN
• Open source community support – through IRC, mailing lists and forums
• GRASS - introduction to the interface and commands, and practical application in flood simulation
• Mapserver, Openlayers – exporting QGIS projects to mapfiles, editing symbology of the mapfiles, displaying through OpenLayers
  
• HTML, CSS, Javascript (JQuery) – linking these up, and building a webpage
• Joomla CMS

Tim also expained how all these sections were connected and added to one another, and we got to see this through our own practical work.

I really enjoyed and learnt a lot, and would like to thank Tim, Anne, Marcelle and Robert for the valuable time I spent here. It’s a sad goodbye to the team and office, but I am hoping that the new skills that I have learnt here will contribute positively to the future.

Looking foward to FOSS GIS growing in Africa! \o/

- sam

Its been a busy month. A few months ago I set things in motion by advertising for interns to come and work and learn about FOSS GIS at linfiniti.com. In tandem with that I invited Anne Ghisla to come out to South Africa from Italy to act as mentor for the interns.

When Anne arrived we interviewed various candidates and selected two – Robert Makondo and Samantha Lee Pan. You will no doubt read postings from them elsewhere on this blog as I have encouraged them to start writing articles and sharing their learning post publicly.

The aim of the internship programme is simple: transfer real-world FOSS GIS skills to interns so that we can build up the skills base in South Africa and eventually greater Africa. One day I would love to be running a kind of academy where we have a continual succession of interns from all over Africa participating in our internship programme and taking FOSSGIS skills and enthusiasm back to their place of work or study. What we are doing now is a small step in that direction.

Unfortunately Samantha (Sam) could only stay with us for the month of February as she is going into permanent employment next month. I decided to bid Sam farewell by taking the group out for a morning at the Lion and Rhino park about 30 minutes from our Linfiniti Offices in Lanseria, near Johannesburg, SA. Of course being a bunch complete geogeeks we did a GPS drive as we went (i.e. we captured our route using the new GPS plugin for QGIS that Marco and I wrote). Here is a little map of our trip (click for larger image)!

Capturing data in this way is a fun and entertaining way to expose interns to the process of primary data capture. The location was also a great choice as some in our group had never before seen lion, buffalo, wild dogs and other charismatic wild animals.

Sam and Robert sat in the back and learned to operate the GPS and the GPS Tracker plugin for QGIS.

We finished our little outing with a short walk to look at the hippos wallowing in a pond. Heres a shot of us all on the hippo viewing platform (sorry we couldnt get the hippos into the shot too ).

Next week we will say ‘goodbye’ to Samantha and ‘hello’ to Petronella, a Zimbabwean lady who will be joining us as an intern for the months of March and April. We will also be joined for the week by two other interns from the start-up company of my friend Andiswa Silinga. Andiswa is going to be getting her interns to do some digitising work using FOSSGIS so they will come to Linfiniti for occasional visits to get up to speed with the FOSSGIS way of doing things.

Training Environment

One of the cornerstones of my setup for providing a training environment has been LTSP (the Linux Terminal Server Project). I have blogged previously about my testing experiences with LTSP. For the LTSP server, I purchased one fairly good spec quad core pc which then acts as a server for up to four thin clients. I bought 3 Fujitsu Futro 100 units for thin clients which connect to the LTSP server. The system works admirably well and we have had ‘nary a hiccup over the month of solid production use. There have only been a few small issues. The GIMP for example causes X to hang when opening a file.

Using thin clients has many advantages – it is extremely simple for users to share files since they are all logged on to the same system. Also, there is only one server / machine to manage. We can get connected to the internet using a single 3G modem which we plug in to the server and then everyone on the thin clients gets internet access.

Since buying the Futro clients I have also being experimenting with using other clients. I dug out a very old and mostly dead thinkpad laptop and set its bios to do etherboot. I plugged it in to the power (its battery is long since deceased) and into an external CRT monitor (its lcd display has also given up the ghost) and voila we have another client for our network. I will probably avoid using old desktop PC’s for this purpose since they consume a lot of power and make a lot of noise, but if you are in the position where you want to maximise value and minimise your landfill contribution, using old desktop pc’s would also work fine.

One more thing I wanted to mention is the the iTalc application that you can run on top of LTSP. iTalc lets you view a gallery of connected thin clients, broadcast a message to any client, share your screen or a window with all clients (e.g. while doing a demo), lock, reboot etc. the clients. I think if I scale up to a classroom environment, it really is going to prove to be a great option.

Morning Sessions

We have established a pattern of having morning ‘geekout’ sessions of around an hour long where we discuss a topic for the day. I don’t rigidly plan what the topics will be – they either relate to some work we are trying to produce, something I have been thinking about, or another piece of the FOSSGIS jigsaw puzzle I want to let the interns know about. Here is a pic of our whiteboard doodlings from such a session:

Mentors

After the morning ‘geekout’ session we task the interns with some work for the day and then I usually run off to a client or try to knuckle down to some work. Anne continues providing mentorship to the interns through the day. The system works really well – although if I had one complaint it would be that my productivity has been reduced somewhat by the extra activities surrounding the interns. I really can’t praise Anne enough here – if it wasn’t for her, having interns would probably consume far larger amounts of my time. She is patient and enthusiastic and she instills her sense of FOSSGIS enthusiasism into the interns. I will be really sad to see Anne go at the end of March. However, I’ve had a couple of other folks in europe interested in providing mentoring services so stay tuned for the musings of other FOSSGIS celebrity guest mentors on this blog!

Biting off more than I can chew

Maybe we are biting off more than we can chew trying to launch a internship / mentoring programme from such a small company, but the wheels of government and NGO support and funding turn slowly and I wield little to no influence in the circles where decent funding gets allocated for these activities. So in the spirit of Open Source we are starting something small (like a little bit of code to scratch and itch) and hoping that others will pick up on it and let us expand the concept out to the larger African audience out that is starving to get a leg up in a GIS world dominated by expensive proprietary software that does them no favours.

Note: Edited Feb 27 to fix some 2am typos and bad grammar.

1.Install a maperver export plugin on qgis (if not installed)
Click Plugins => Manage plugins
This will take you to the Qgis plugin manager window
Click on the checkbox next to Mapserver Export(Version 0.4.3)
Then click OK
2.Add some layers to your qgis
I added two layers (roads layer and mbizana_munic layer) using the data of Mbizana municipality
3.Exporting mapfile
I first saved my qgis project in /home/robert/mapserver
Then you click Plugins => Mapserver export => Mapserver export (will open the mapserver export window)
Click the “save as” and give the name to the mapfile.
I saved mine as mbizana.map in the mapserver folder (/home/robert/mapserver/).
Then click OK.
5.Edit the mapfile
In the mapfile just comment the lines that point to the symbology file and font file, to look like this:
#SYMBOLSET “./symbols/symbols.sym”
#FONTSET “./fonts/fonts.list”
6.Test your mapserver with your mapfile
I put this /localhost/cgi-bin/mapserv?map=/home/robert/mapserver/mbizana.map&mode=map

Now i can access the mapserver using Openlayers. Happy mapping !!!!!!!!!!!!

Then have a look at jQuery UI CSS framework, a system of classes developed for jQuery UI widgets.

This is the map toolbar of the webGIS site we are busy developing, rendered with UI-Darkness theme:

The controls (pan, measure and zoom) are OpenLayers’ controls. They are all created in the map’s init() (see first js snippet below). The binding with the buttons is made by name – therefore be sure that the names of the OpenLayers controls match exactly the name properties of the respective buttons. The activation of the selected control is done by the toggleControl() function, further below in the js snippet. That way you can add as many control-button pairs as you need.

Let’s see what the code looks like. It is not so much indeed.. My tribute to the proverbial programmer’s laziness and to the koan of Master Foo’s and the Ten Thousand Lines.

First be sure to include all necessary scripts in the html head:

<link type="text/css" href="/static/css/jquery.fancybox-1.2.6.css" rel="stylesheet" media="screen" />
<script type="text/javascript" src="/static/js/jquery-1.3.2.min.js"></script>
<script type="text/javascript" src="/static/js/jquery-ui-1.7.2.custom.min.js"></script>
<script type="text/javascript" src="/static/js/jquery.fancybox-1.2.6.pack.js"></script>
<script type="text/javascript" src="/static/js/jquery.easing.1.3.js"></script>


then add the buttons – I took inspiration from Filament Group excellent post:

<div id="mapcontrols" class="fg-buttonset fg-buttonset-single ui-helper-clearfix">
<button name='navigate'class="fg-button ui-state-default ui-state-active ui-priority-primary ui-corner-left" >Navigate</button>
<button name='line' class="fg-button ui-state-default ui-priority-primary">Measure line</button>
<button name='polygon' class="fg-button ui-state-default ui-priority-primary">Measure area</button>
<a href="#" name='zoomin' class="fg-button ui-state-default fg-button-icon-solo" title="Zoom in"><span class="ui-icon ui-icon-circle-zoomin"></span> Zoom in</a>
<a href="#" name='zoomout' class="fg-button ui-state-default fg-button-icon-solo ui-corner-right" title="Zoom out"><span class="ui-icon ui-icon-circle-zoomout"></span> Zoom out</a>
</div>


And in the end the JavaScript – add the following snippet in the map’s init() function:

    mapControls = {
	line: new OpenLayers.Control.Measure(
	    OpenLayers.Handler.Path, {
		persist: true
	    }
	),
	polygon: new OpenLayers.Control.Measure(
	    OpenLayers.Handler.Polygon, {
		persist: true
	    }
	),
	zoomin: new OpenLayers.Control.ZoomBox(
	    {title:"Zoom in box", out: false}
	),
	zoomout: new OpenLayers.Control.ZoomBox(
	    {title:"Zoom out box", out: true}
	)
    };

    var control;
    for(var key in mapControls) {
	control = mapControls[key];
	control.events.on({
	    "measure": handleMeasurements,
	    "measurepartial": handleMeasurements
	});
	map.addControl(control);
    }

and these functions at the bottom of your js file:

function handleMeasurements(event) {
    var geometry = event.geometry;
    var units = event.units;
    var order = event.order;
    var measure = event.measure;
    var element = document.getElementById('output'); //TODO redirect to other area?
    var out = "";
    if(order == 1) {
	out += "Measure: " + measure.toFixed(3) + " " + units;
    } else {
	out += "Measure: " + measure.toFixed(3) + " " + units + "2";
    }
    element.innerHTML = out;
}

function toggleControl(element) {
    for(key in mapControls) {
	var control = mapControls[key];
	//alert ($(element).is('.ui-state-active'));
	if(element.name == key && $(element).is('.ui-state-active')) {
	    control.activate();
	} else {
	    control.deactivate();
	}
    }
}

$(function(){
    //all hover and click logic for buttons
    $(".fg-button:not(.ui-state-disabled)")
    .hover(
	function(){
	    $(this).addClass("ui-state-hover");
	},
	function(){
	    $(this).removeClass("ui-state-hover");
	}
    )
    .mousedown(function(){
	$(this).parents('.fg-buttonset-single:first').find\
            (".fg-button.ui-state-active").removeClass("ui-state-active");
	if( $(this).is('.ui-state-active.fg-button-toggleable, \
            .fg-buttonset-multi .ui-state-active') )
	    { $(this).removeClass("ui-state-active"); }
	else { $(this).addClass("ui-state-active"); }
    })
    .mouseup(function(){
	if(! $(this).is('.fg-button-toggleable, .fg-buttonset-single .fg-button,  \
            .fg-buttonset-multi .fg-button') ){
	    $(this).removeClass("ui-state-active");
	}
	//TODO use this else only for measure/pan toggle.
	else {toggleControl(this);}
    });
});

Ok, should be all you have to know to set up the toolbar! Feel free to reuse the code and improve it
Oh, and don’t forget to tweak the CSS to get the perfect look and feel

–anne

This reminds me how life forms adapt to severe environmental conditions. But it’s a wide topic.

Let’s see how you can easily do this in Python. The snippet is a generic version of a function in views.py of the GeoDjango website we are busy developing. That function caches the result of WMS requests for layer legends in a dedicated directory, assuming that the images are not changing over time.

import urllib
import os

def retrieveDataFromUrl():
 myFileName = "file.txt"
 myLocalPath = os.path.join( os.getcwd(), myFileName )

  if not os.path.exists( myLocalPath ):
    print "Downloading data"
    myUrl = "http://whatever"
    # save it where it should have been found
    urllib.urlretrieve(myUrl, myLocalPath)
  else:
    print "Reading from local file" + myLocalPath
  # then read the file...

This code only checks if the file exists. If the file downloaded in previous run is outdated, then the newer version must be downloaded. This can be a good task for a cronjob – but it’s the topic of another post

Hope this helps!

– anne

I was working on a shapefile of the suburbs in Cape Town. A client required the suburbs to be grouped by region. After the tedious part of manually grouping the suburbs (using a created field, REGION), the unioning (dissolving) of the suburbs proved to be quick and painless through PostgreSQL, using this SQL command that implements the geomunion function:

create table ct_regions as select geomunion(the_geom), "REGION" \
 from "ct_suburbs" group by "REGION";

“ct_suburbs” is the original shapefile that was loaded into the PostGIS database using the Quantum GIS ‘SPIT’ plugin. “REGION” is the class (attribute) that I wish to union by.  And ct_regions will be the output shapefile. See the result here:

Before dissolving (ct_suburbs shows suburbs)

After dissolving by suburbs (ct_regions shows collections of suburbs that have been merged)

Hopefully this will be of some use when it comes to your own mapping!

We have hired Anne Ghisla (Italian QGIS and GRASS geek) to come over to South Africa and spend two months mentoring our interns.

The interns were offered a two month period during which they would receive a monthly stipend to cover their living expenses, and receive on-the-job experience in the practical use of FOSS GIS to solve real world problems.

Robert Makondo studied computer science in Pretoria and worked for a year as an intern at the State IT Agency (SITA). Robert is a relatively newcomer to FOSS and FOSSGIS but is already showing great enthusiasm in learning the new tools of the trade!

Samantha Pan is from Cape Town where she studied GIS at UCT. Samantha has used GIS during her studies but is new to the world of FOSSGIS (which she is adapting to extremely well!). Unfortunately Samantha is only staying with us for one month but we hope she will use the skills she learns here in her future place of work.

Having the mixture of CS and GIS disciplines in our interns is really nice since they complement each other well and can problem solve together. I think Anne is really enjoying her stay here in South Africa and mentoring our interns. We will be sad to see her go at the end of March .

If any readers out there are interested in coming to South Africa for a few months to help with mentoring interns please let me know! We will cover your costs and provide accommodation. In return you get to help bridge the digitial divide between Africa and the western world, while at the same time having a great experience in a different country!