Linfiniti Geo Blog » QGIS

A week of FOSSGISsing in Johannesburg, SA

Tim Sutton — Sat, 24 Jul 2010 20:24:42 +0000

Horst Duester and I spent the week in Johannesburg (South Africa) doing FOSSGIS training. As I mentioned in a previous article, Horst is visiting South Africa on a sabbatical. He has kindly donated his time to provide training services in order to promote and facilitate the use of FOSSGIS. We badly need more people evangelising FOSSGIS here – there are many people looking for low cost and good GIS solutions and few people with the time and know-how to show them how to proceed.

We drove up to Jhb from Swellendam in one day – 1400km – a first for Horst who calculated that with an equally long trip from his home town he would have landed up in Russia!

Our first port of call was at Wits University (the University of Witwatersrand, Jhb). Devlyn Hardwick convened the course at Wits and kindly provided us with accommodation for the week. We spent the monday afternoon installing the Ubuntu VMWare image we had prepared onto all of the computers in the lab. At the end of the day it was a real thing of beauty to see so many Ubuntu workstations.

From Tuesday to Thursday we (ok mainly Horst) presented an enlightening overview of QGIS and GRASS to the participants. He covered the process you need to follow to produce a professional quality map – from importing or digitising your date, to symbolising that data to conducting an analysis and finally culminating in the use of the QGIS map composer to create a great map. Much of the development of the map composer was funded by the SO!GIS group at Solothurn and it was really great to revisit just how far QGIS has come along in its ability to produce a high quality map.

Training at Wits University

Devlyn and Horst enjoying sumptuous fare at the Ethiopian restaurant

Horst putting QGIS through its paces

On friday our activities moved to the CSIR Meraka institute in Pretoria (which is very close to Johannesburg). At Meraka we held a one day PyWPS workshop which was convened by Graeme McFerren. During the workshop we walked the participants through the process that you need to follow to get PyWPS set up on your server, and then followed that with some examples of how to write a PyWPS process. The workshop was quite technical in nature. Those particpants with less technical knowledge were given an understanding of what the OGC WPS standard sets out to facilitate, while those with deeper technical knowledge left with the building blocks they needed to go off and deploy their own PyWPS processes.

PyWPS training on the go

One of Horst’s motivations for spending his sabbatical here in South Africa was to improve his English. So it was quite a feat on his behalf to conduct his training sessions entirely in English for four days straight.

All in all it was a great and successful week and we are now en-route back to the Cape – though we are breaking up the trip with an extremely pleasant stay over at the Karoo National Park. In true geek fashion we are sitting in our chalet using e2fsck to repair Horst’s dodgy hard disk which seems to have thrown a wobbly after being exposed to our South African electrical system.

Horst taking a break from riding through the desert on a horse with no name

As an aside for those interested, the way to check your disk under Ubuntu / Linux and mark bad sectors so they don’t get used is this:

e2fsck -c /dev/sda5

In the weeks to come Horst will be presenting more free and commercial FOSSGIS training workshops here in South Africa. Hopefully in the future others who feel like a break from the daily grind will come and pay a visit to South Africa and follow suite – the effort is greatly appreciated here!

Cleaning geometries inside PostGIS

Tim Sutton — Wed, 07 Jul 2010 20:36:32 +0000

Its been an enlightening few days. Horst Duester (of SO!GIS fame) is paying me a visit for a two month sabbatical. Horst is here to provide valuable training and knowledge exchange to South Africans interested in FOSS GIS (more on that to come in future blog posts no doubt!). Its been really great having an opportunity to sit down with a fellow FOSSGIS geek and exchange notes – and today Horst showed me something really cool. Horst has written a PostGIS function to clean geometries within the database. Normally I use GRASS if I want to clean the topology of my vector dataset. With Horst’s approach this is no longer needed which is a great timesaver. Here’s what to do in a nutshell:

Assuming you already have a spatial database created (with lwpostgis, spatial_ref_sys etc. sql scripts run on it), download and install Horst’s clean geometry function. Here is on way to do that:

wget http://www.sogis1.so.ch/sogis/dl/postgis/cleanGeometry.sql
psql mydatabase < cleanGeometry.sql

Now lets create a test table and show you how to use it. I used Martin Dobias' excellent PostGIS Manager plugin (available in the python plugin repositories) to create a table as shown in the screenshot below (click for a larger version). As an aside, Martin's tool is a great way for new users to get started with building their own spatially enabled database.

Next I created a self-intersecting polygon (in this case a figure of eight-ish shape) as shown below:

The little green cross in the middle of the polygon shows the self-intersection. To clean the geometry, I then ran a little sql command that uses Horst's cleangeometry to fix all the geometries in the table like this:

update testpolys set the_geom=cleangeometry(the_geom);

The result is a multipart polygon with no self intersections but preserving the original shape and attributes. In the image below I've pulled the connecting vertices in the center apart so you can see that two sub-polygons were created:

Eye-Spaghetti: Visualising QGIS dependencies

Tim Sutton — Mon, 05 Jul 2010 07:21:34 +0000

While looking at the options for apt-cache today I noticed ‘dotty’ as an option. I presumed this could be used to make grahpviz diagrams of package dependencies. After a bit on googling on how to use that, I came across debtree:

sudo apt-get install debtree graphviz

Of course the first command I ran with it was this

debtree -I --rdeps-depth=3 qgis | dot -Tpng -o qgisdeps.png ; display qgisdeps.png

Take a look at the beautiful (ok and just a teeny bit complex) diagram it produced. Its quite amazing really to see how many different components there are out there in the FOSS world that can be combined to produce cool software like QGIS. It really lets you see that QGIS is just as much an integration effort as it is a bespoke software development effort.

Translating the Gentle Introduction to QGIS videos (take #2)

Tim Sutton — Sat, 12 Jun 2010 21:10:55 +0000

A little while ago I posted here that the videos we created for the Gentle Introduction to GIS series were online at dotsub.com. I received feedback from some readers that the videos were still locked and so subtitles could not be added. I believe they have been unlocked now and if folks would like to subtitle in their own language (which would be great!) you can go ahead and do this. Currently all the videos should be available with both English and Spanish subtitles and it would be great to see more languages added to the list. Here are the links to all the vids:

Python testing with Nose – even easier!

Anne Ghisla — Thu, 06 May 2010 08:02:48 +0000

Recently, a friend pointed me to Python nose, a package providing a very easy way to build tests, based on unittest. I still have to write some code on my own to prove this, but I am too impatient to talk about it! Stay tuned for some code snippets in the next future..
Have a look at nose‘s page:

http://pypi.python.org/pypi/nose

Happy Python coding,

–anne

Your chance to translate the Gentle Introduction to GIS videos

Tim Sutton — Thu, 22 Apr 2010 08:03:28 +0000

From the ‘this is cool dept’: dotsub is now hosting the GIS gentle intro videos – you can go there and add subtitles. I still need to add markers to them to indicate where each sentance should start – at the moment I dont have time or bandwidth…maybe someone else will pick that up…

Biodiversity at FOSS4G2010

Tim Sutton — Tue, 20 Apr 2010 20:33:09 +0000

If you are interested in biodiversity and attending FOSS4G2010 in Spain this year, take a look at my friend Javier de la Torre’s initiative to form a FOSSGIS special interest group in Biodiversity at the conference. Unfortunately I can’t attend the conference but Javi’s work really resonates with me and FOSSGIS has a lot to offer those who are passionate about keeping out planet alive for another year or two!

Final report on my experience @ Linfiniti

Anne Ghisla — Fri, 09 Apr 2010 11:41:06 +0000

Time for a final report has come…

First of all, I thank Tim for all trust he had in my programming and managing capabilities, and for giving me this great job opportunity. I’m deeply thankful to the interns I mentored: Robert, Sam and Petty, the nicest people I could meet, it has been so natural to work together. They made me feel comfortable in our office with flickering internet and power outages, teaching me the African friendliness and patience

What I learned:

Django - the main project I worked on taught me how to find my way in a huge directory tree and three programming languages. After a tough beginning, I’ve been able to quickly indentify the point(s) of the code to edit and do less side-effect damages possible. Django is really a simple and powerful tool to build websites and manage backend databases.
OpenLayers – JavaScript turned to be a quick and flexible enough way to display and manage maps. Still have lot to learn but at least I can read and understand almost all JS code I encounter.
LTSP - our office ran Linux Terminal Server Project, on a common PC as server and three FitPC as thin clients. This solution has many advantages: less power consumption, data and programs in only one machine (tribute to SPOT rule), less cost than having N full-featured PCs , very good scalability – it can be easlily used in classrooms and labs.
Mentoring – there is a special ability required to transmit knowledge. While explaining concepts, I realised I was using most of them without understanding them fully, and I learned how to build more clear and structured lessons, and to say “I don’t know the answer” instead of inventing something… Thanks to my interns for all precious feedback!

What I planned to achieve:

Test-driven development – unfortunately none of us knows enough of it to quickly build tests. Moreover, the code for our webGIS builds an interactive GUI with JavaScript and AJAX. Therefore we have been reduced to test by hand, and quite often the commits broke functionality and I didn’t notice until I had time to extensively test – when the Internet allowed me to refresh the page in less than 10 minutes.
Extreme Programming – We succeeded in applying some of its principles: communication (even if English is mother tongue only for Tim), extensive use of the whiteboard for mind dumps and lessons, short release cycles, feedback. We would like to improve our office management integrating other valuable tips – Robert is reading “Extreme Programming Explained” and he’s really enthusiastic!

This experience has been really precious and I will never forget it.

I enjoyed a lot working in a pure open source environment, with strong belief in its values by my employer. It is too common to work for a company and relegate open source to spare time or to side activities.

Living here for three months made me realise how lucky is Europe, with all first-world facilities, and how open source is a valid alternative to the common closed-source solutions. In GIS, especially, there is little place for ESRI to claim the huge (for South African standards) licence fees, while there is a great need of GIS for land management at any level.

I strongly suggest to open source geeks to come and stay in South Africa and work at Linfiniti! It will be a great experience abroad, in a country full of beauty, humanity and hope.

Look forward to come back!!

–anne

Learning GIT

Tim Sutton — Wed, 07 Apr 2010 20:01:36 +0000

For source code management in the QGIS project we have been happily using subversion for some time now, having made a not-too-painful transition over from CVS several years ago. Lately we have bandied around the idea of moving over to GIT. This is mostly out of fear of being called bad things by Linus Torvalds for using SVN (ok I’m kidding). Actually there are a lot of technical reasons why a move over to GIT would work well for us – not least of which is the idea of easier merging of branches and the redunancy offered by multiple repos being ‘out there’.

Adding a little impetus to my interest in learning GIT is the fact that more people are using it for managing their code so I need to update my skills to be able to collaborate with them effectively.

All this brings me to my real point: today I found a great resource which provides a guide for those familiar with SVN and wanting to migrate to GIT.

2 Great tools for web developers

Tim Sutton — Tue, 30 Mar 2010 13:15:34 +0000

I just thought I would make a note of two little gems:

favicon.cc is an online tool for creating that perfect favicon for your web site. Yes the irony hasnt escaped me that my own site needs one!

ajaxload.info is a cool little site that helps you to create an ‘waiting’ gif that you would typically use in an ajax application while the user waits for something to happen.