Wrapping up the QGIS Meeting in Lisbon, April 2011

The meeting was hosted by IGOT (Instituto de Geografia e Ordenamento do Território) which is a research institute within the ‘Faculdade de Letras de Universidade de Lisboa’. We would like to extend our heartfelt thanks to the Instute and University for hosting us, and in particular the following academic staff members who supported our meeting:

• Prof. Doutor Diogo Abreu (Director of Centro de Estudos Geográficos)
• Prof. Nuno Marques Da Costa
• Prof. Paulo Morgado
• Prof. Nelson Mileu
• Dr. Fernando Benedito

We would also to thank Manuel Ordas (Director) and the staff of the Faculty Informatics Department (Unidade de Informática e Telecomunicações) for the excellent job they did in supporting the meeting with our Internet and IT needs!

The meeting had several sponsrs:

• Municípia
• OSGEO-PT
• Faunalia.PT

We are extremely grateful for their sponsorship.

The meeting was organised by Giovanni Manghi  and Vânia Neves of Faunalia.PT. They did an an incredible job of organising the meeting, printing t-shirts, keeping everyone fed and dealing with all the behind-the-scenes logistics so well that everything in the venue felt seamless and we could really focus on what we were here for. A big personal thank you from me too to Giovanni and Vania for hosting me at their house in the lovely town of Evora when I arrived in Portugal.

Vânia Neves and Giovanni Manghi - meeting organisers and all round superheros!

Town Hall Meeting

On the sunday we had a townhall meeting to discuss various project wide topics. First on the agenda was the QGIS Certification Programme. Plans for the programme are still under development and there has been some progress and discussion on the QGIS-edu mailing list. We still have a lot more to do to make this a reality and will try to get some more movement in place on this topic by the next hackfest.

GIT and Redmine. The topic of GIT and Redmine took up most of the discussion. I spent some time explaining the benifits of moving to GIT as I see them and some of the implications it will have on the project and our working practices. There was some discussion as to whether it was a good idea to migrate to Redmine (a new ticket management system) at the same time. The motivation for moving to GIT is summarised somewhat here. It was agreed that we will move to GIT after we have branched for 1.7 release, but before we make the release annoucenments.  It is proposed that the project master repository will be hosted on github under our organisation area (where the current test QGIS git repository exists). A lot of discussion centered around how the social dynamics would work – and what working practices we would convey to users and developers. In particular there was some question about whether it was a good idea to use the github pull request system and the git ticket queue – the central concern being that we would like the provenance of who is submitting changes and from what repostories they were pulled to be preserved within our ticket system. As such the approach we will take will be to ask contributors to submit their pull requests via our own ticketing system rather than github pull request mechanism, and to ask users not to use the github ticketing system.

A parallel discussion took place about migrating from trac to redmine. The central argument being that in order to provide a platform for users to submit tickets relating to third party plugins, and to provide a platform for plugin writers to easily create a issue tracking system we would like to use the new redmine instance that has been set up on QGIS.org servers. However, this would result in us having two disparate ticketing systems and we would prefer to unify and simplify our web offerings. In addition, a migration to GIT will also entail some substantive updates to our trac instance, so it was  suggested to use the period of change introduced by GIT to also update our issue tracking at the same time.

More details will be thrashed out via the mailing list, but the take-home message is this: we will move to GIT and Redmine after we have branched for release for QGIS 1.7, but before we make the release annoucenments. In the release announcements, we will simultaneously inform our user and developer community that they should use GIT and Redmine henceforth.

The next item on the agenda was the finance report from Paolo Cavallini. It was extremely encouraging to see that the flow of donations has been increasing and has been able to sustain our QGIS meetings thus far. As our developer community grows, more are taking advantage of the funding opportunities provided by the QGIS treasury. We are looking to bring in around 2500 euros in order to support the next meeting (more on that below) so continued support via the donation button on QGIS.org will always be appreciated. There was a collective ‘thank you’ from those present to the kind focus who have donated to the project thus far! We also had some discussion on how to deal with ‘targetted’ donations – following the kind donation of 740 euros by Ramon Andinach. In making the donation, the request was made that it should be used towards bug fixing and improving QGIS. It was generally agreed that we are ok with such a donation targeted towards a general area of activity in the project. A further part of the discussion centered around how we could use the money to create some form of bounty system whereby developers or contributors would receive a small amount for each bug they close. The exact system that will be used was discussed at length but remained unresolved. Paolo is going to draw up a proposal which we will discuss further via the mailing list. In general we hope to structure the bug hunting exercise in a way that it will minimise the supervision required from those with commit access and at the same time attract new developer talent into our community.

Following this, we discussed the venue for the next QGIS Meeting. Marco Hugentobler offered to convene the meeting in Zurich in November, which we will all look forward to attending. More details on the venue, dates, accommodation etc. will be made available via our mailing lists in the coming months.

We took the opportunity after the town hall meeting to take a group photo (below).

QGIS Meeting attendees, Listbon, April 2011 (not all attendees present in photo)

In the next sections I will try to summarise other interesting discussions that took place over the course of the meeting.

Web Infrastructure

A number of us held a meeting to discuss the web infrastructure on QGIS.org. We are in the process of rolling out some new web applications, with Alessandro Pasotti leading the effort. We discussed at length the requirements for the new plugin repository . The new repository will make it easier to administer plugins, prevent plugins becoming orphaned, and will form a platform in the future for building more social based capabilities around the plugin repository. These will include a tagging system, ratings, download stats to reflect popularity, and so on. There are further  plans for the web site, with the general idea of minimising the diversity of different online applications we need to maintain (the new batch all forming part of a single django project), making maintanance easier, and providing richer community based resources to our users in the future.

Borys espousing his ideas at the web infrastructure meeting

QGIS Mapserver

A meeting was held to discuss the QGIS Mapserver project (spearheaded by Marco Hugentobler). Marco is planning various improvements to rendering performance, some of which he worked on during the meeting, including support now for on-the-fly reprojection of rasters via the QGIS Mapserver.

User Interface

A break out meeting was held to discuss the user interface for QGIS and ways in which it can be improved. Unlike the discussion on the same topic we held in Wroclaw, where we got a little lost in the enormity of reviewing every dialog in QGIS, this time we took a different approach. The talk centered around identifying cross cutting user interface issues that can be addressed by the creation of generic Qt widgets. Several different areas were discussed.

First up, we discussed the need for a generic managed tree widget (which will probably be implemented as QgsManagedTreeWidget). The idea here is that there are many places in the user interface where functionality is required to manage lists. In each case this functionality is represented as separate, standalone logic and different user interface elements. An example of this can be seen in the vector layer properties dialog on the actions tab. In this case a list of actions is provided with a row of buttons underneath. In another instance, the new generation symbology symbol layer editor also implements a managed list for the symbol layers, but with completely different button, icons etc. There are numerous other places in the QGIS application that have different, diverging implementations of managed lists, which we will seen to unify via the implementation of a custom widget.

Similar discussions were held about the legend areas, where we plan to introduce additional functionality to support:

• a table of contents view (as currently available)
• a providers view — a tree with top level nodes for each provider and leaves representing each resource from that provider that can be added to the map (via drag and drop). This will allow you to for example browse your file system tree and add one or more vector or raster layers, then from within the WMS provider node, drag in a WMS resource, and so on.
• a plugins view – a tree which can be filtered and which lists each plugin action available. Plugin actions will be able to be dragged onto the user interface – for example to allow user to create their own toolbars with a mix and match of plugins contained therein.

We discussed other potential views that we could implement in the future.

The last area of the interface that received detailed discussion was the vector NG symbol selector and designer. After a bit of brain storming we mocked up a more ergonomic approach to the selection, customisation and creation of symbols. Once again this will be implemented as a generic widget when we have the needed resources available.

Orfeo Toolbox Integration

I took the opportunity to have a long one on one chat with Julien Malik from the Orfeo Toolbox project. He kindly walked me step by step through the setup and configuration process on my own laptop. The following day Julien spent an hour or so presenting the OTB project to the entire group. The OTB project covers a huge range of functionality of particular interest to those involved in remote sensing activities. It can carry out tasks such as atmospheric correction, image classification, resampling, feathering, mosaicking, pan sharpening and many more to boot. At the end of the more formal part of his presentation, Julien demonstrated live some of the tools and showed the state of integration of OTB within QGIS. To summarise, the OTB project is principally a library (making heavy use of template programming) for C++ programmers to build remote sensing tool chains. On top of this library they have created a number of applications. The approach they have taken to build these applications is to use application generators. Thus in order to create an application for say, reseampling an image, they create a short application that calls the needed library functions and that implements a specific set of API calls that they have defined. This could be considered a metaprogram because it is compiled into a library and then various user interfaces are automatically generated for it. For example a command line app, a Qt widget, a Qt standalone application, a QGIS plugin, and so on. This approach means they can rapidly and easily produce a huge range of tools based on their library, and have them generated into applications useful in different environments. One of the disadvantages is that currently this approach means that when creating an image processing chain, the result of each step in the toolchain needs to be written to disk – which can consume huge amounts of space. However internally, the OTB library has the capability to deal with chaining tools in a very efficient  whereby only the final output gets written to disk and all intermediate products or pipelined from one tool to the next and never written to disk. Although the current implementation uses the formerly described approach of writing each output to disk, there was some discussion about how we can implement the more efficient model in QGIS. Marin Dobias also made some suggestions to create python bindings from the OTB-Applications and then generate QGIS python plugins using the same mechanism as described above. This has three potential benifits:

• it will remove a bug in OTB when it is linked to QGIS caused by the Ubuntu GDAL & GTIFF library packages
• it will possibly allow end users to contribute more readily to the python plugins for OTB
• it will allow the user interface component of the OTB integration to be updated and pushed out to users via the standard QGIS repository mechanism.

I have probably done the OTB project and its project a disservice by giving it such scant coverage above – I believe it can potentially be a huge boost to the QGIS project, bringing it into the hands of remote sensing specialists to instead of just GIS professionals and users.

The Orfeo Toolbox built from the OTB-Applications project

New Plugin In a Weekend

Raymond Nijssen gave a cool little presentation at the end of Sunday showing the plugin he drummed up over the weekend. The plugin implements a vector attribute calculator where the formula used to calculate the new field values can be any evaluated python statement as opposed to the restricted list of operators allowed by the implementation in QGIS core. In addition you can access the geometry column so the computed value can include geometry operations via the python geos bindings. Its really impressive to see just how quickly significant new features and tools can be created for QGIS now that we have a rich library as foundation. Ramon’s plugin should make it’s way into the python plugin repositories soon. I bumped into Ramon again in the airport on my way home and he kindly stood me a cup of tea and we had an enjoyable chat for half an hour while waiting for our respective flights – thanks Ramon!

Documentation

Jean Roc gave an interesting talk where he showed off the work they had done getting the QGIS documentation (french version) into a print ready state and publishing it on Lulu.com (an on-demand online print service). He had an exemplar copy with him and the product looks great – really professional. He briefly mentioned some of the steps they needed to take to get the documentation ready for printing. At some point we will aim to get an english version onto lulu.com and see if it is able to help plump out the income stream to the QGIS project. If you are a French speaker, you can buy the bound user manual here.

Mobile

At our last developer meeting I didn’t notice it to be the case, but this time around virtually every developer was toting a nice Android phone of some sort. I saw some really crazy things (yes I’m talking about you Saber – running a debian install in a chroot in your phone and then vnc’ing into it with a mobile vnc client is pretty nuts), and we had some quite hilarious incidents (imagine 30 GIS geeks ambling around the streets of Lisbon staring at our phones trying to find a restaurant and taking an hour route for what should be a ten minute walk. There were also a few folks running other smartphone platforms like Maemo and so on.

One of the offshoots of all this smartphone mania is that there was quite a lot of interest in seeing QGIS as a mobile platform. Now when I say QGIS, I’m not talking about the whole desktop environment, but rather the core QGIS library, some raster and vector providers and some custom mobile orientated  user interface on top of them. Porting QGIS is not entirely trivial since all the dependencies (GDAL, GEOS, PROJ4 etc.) need to be recompiled for the platform. Qt4 is already available on android (search for Ministro on the app store) so thats at least one thing we don’t have to worry about too much.

At the meeting there were a handfull of folks who seem to be well versed in building apps for mobile platforms and they set to work towards getting some basic QGIS port underway. I spent a very informative and interesting hour or two with Gabriele Franch. He demonstrated QtQuick to me and we built a simple hello world app and mixed in native QtWidgets with QtQuick declarative derived user interface components. We also spent some time discussing what kind of user interface and functionality we would like to see make it onto the mobile platform. For me I can see an environment something like:

• a full screen map canvas
• tools that appear after swiping your finger from offscreen to onscreen
• offline editing capability
• local viewing of QGS projects (perhaps migrated to sqlite3 with sqlite raster support)
• simple forms and data capture capabilities
• reading and capturing data from GPS

Many of these were share (or originated) from those at the meeting. Although there is nothing concrete available yet, I get the feeling a mobile version of QGIS won’t be long in coming won way or the other – something I will look forward to!

QGIS – PgAdmin3 Integration

Vincent Picavet gave an interesting talk on some work he has done to integrate QGIS and PGAdmin3.  Basically he has implemented a macro in PGAdmin3 that places a query you have created into a notification area in the database. On the QGIS side, he implemented a plugin that observes the database for these notifications, and when one is received, runs the query and displays the results on the QGIS map. This approach means that you can use the comfortable and powerful tools in PGAdmin3 to create queries, and then quickly visualise the results directly in QGIS. There are some existing python plugins that let you create and run arbitrary queries with QGIS and view the results directly, but none will give the wealth of capabilities that PGAdmin3 offers.

Some General Thoughts

This QGIS Meeting was a little different in that it was the first time we have held the meeting during feature freeze so there wasn’t the same ‘feature fever’ that has accompanied our past meetings (something I have always enjoyed). Also we seemed to spend a lot more time engaged in talks and discussions and less time coding – perhaps ironic with the change in moniker from ‘hackfest’ to ‘meeting’ for our 6 monthly get-togethers. We still conducted very much an ‘unmeeting’ with little in the way of formal agenda or session planning before hand, and I think the format works well, although my feeling is that next time we should try to constrain the non coding potions of the event to a single day (or even better, morning session).

There was still quite a bit of work going on in the bug fixing front and some long standing and critical map rendering problems were resolved.

Bug #7 - one of the oldest bugs in our bug queue (5 years old) killed by Juergen Fischer

There were a fair share of new faces at the meeting which is always great, and I had the pleasure of meeting Maxim Dubinin, Alessandro Pasotti, Giovanni Manghi and Vânia Neves in person, all of whom I have known virtually up until now.

In summary, it was a great meeting, personally I had a great time and left enthused more than ever about my favourite FOSSGIS Desktop software!