Posted by & filed under QGIS.

FOSS4G2013 is starting in just over a month and we (the QGIS project) are really hoping to make QGIS 2.0 available for the event. Its going to be a huge splash as 2.0 is a giant leap forward over version 1.8. However there is a hitch: We have 15 blocker issues holding up the release. We are committed to putting out the best release we possibly can but to do that we need your help! The QGIS project has grown so much in the last few years and with the number of people that depend on it, we just can’t afford to put out anything but the best possible release.

If you have the means to help please do! There are several concrete things you can do to help:

  • fund the project directly – we are looking to pay some of our core developers to put in the time needed to close off the remaining blocker issues. Please see the QGIS Sponsorship page for more information on how you can do this. QGIS also incurs many other overheads and we receive no revenue other than your donations, so we thank you for your support.
  • sponsor developer time – if you have developers working for you who are able to help closing off our blocker tickets we will be super grateful if you can allow them to spend some of their work time closing off blockers!
  • test and help to characterise issues – the more people who test QGIS, the better our prospects for a good quality QGIS 2.0 release! Visit the downloads page to find out how you can get a test build of the upcoming release.

 

As a last bit of news on the QGIS front, nominations are open for 3 new positions on the QGIS Project Steering Committee – if you know of someone you think will be well suited to the post please let us know using the instructions provided.

 

Posted by & filed under QGIS.

No words for this one, just pictures!

What we are trying to achieve

Above: What we are trying to achieve…

Top layer

Above: Our symbol layers

Highlight layer with data bound fill colour

Above: The roof colour is data defined

Shadow layer

A shadow Layer

Bottom Layer

And a highlight layer

One other hint – don’t forget to enable symbol levels!

Posted by & filed under General FOSSGIS, QGIS.

A few months back I published an article about the large scale deployment of QGIS and FOSSGIS at the state administration of Vorarlberg, Austria. Shortly after I published the article, they asked for the opportunity to update the article with more details. The article that follows below is that amended version.

In 2011, the State of Vorarlberg, Austria became a new sponsor for the QGIS project. I was quite interested in the work they were doing as they are yet another great example of QGIS and FOSSGIS being used in an enterprise level setting. The “Landesamt fuer Vermessung und Geoinformation” – LVG, is the department in the state government of Vorarlberg responsible for all general tasks concerning surveying and geoinformation. I carried out the following interview with Nikolaus Batlogg, Marion Heinzle, Johannes Kanonier and Martin Studer from the LVG as representatives for the administration wide GIS group that was involved in a project in 2011/12 to substitute the still widely-used ArcView 3.x for QGIS. Besides QGIS other FOSSGIS components especially server applications had been integrated to into the GIS environment of Vorarlberg’s administration previously.

TS: Could you briefly introduce yourselves and the positions you hold at Vorarlberg?

Nikolaus Batlogg (NB): MSc in Meteorology and Geophysics. Due to my former job, experience in Network and Server Administration and Configuration (MS-Windows, mostly Linux and IBM/OS2, Mac OSX) and a little bit programming (C, Visual Basic, Fortran and Perl). Since 2003 I’m working as a GIS Technician for LVG. Due to my former experience I slid again into the IT. Together with Andreas Siegel (AS) I am responsible for the technical infrastructure: GIS Server Configuration and Administration (Linux and MS), web-services, our GDI, programming with ArcObjects and now with QGIS.
Marion Heinzle (MH): MSc in Geography and Mathematics. Before I joined the LVG team in 2009 I have been working part time as a teacher for mathematics and part time as a GIS technician for the state administration of Tyrol. My main duties in our team are planning, organising and holding GIS education, providing relevant information to our users and organise the transfer of knowledge in general.
Johannes Kanonier (KA): MSc in Surveying, Photogrammetry and Remote Sensing, postgraduate degree in Environmental Studies: I started in 1993 in the environmental protection department and became GIS coordinator for the state administration in 1998. After a reorganisation in 2004 the general GIS group (me and a colleague at that time) joined the LVG, since 2011 I am the head of the GIS division in the LVG. I was in charge of the project management for the substitution of ArcView 3.x.
Andreas Siegel (AS): MSc in Meteorology and Geophysics. In my former job I worked as developer for banking software for almost 10 years. In 2009 I changed to the LVG. According to my experience in programming (mostly C) and Linux administration the main focus of my job is, together with Nikolaus, the technical infrastructure of our GIS and also database administration.
Martin Studer (ST): Federal Secondary College of Engineering graduation. One of my main duties is the data dissemination and the support of our customers in this context. In our department I am also the general and first contact person to the central IT department of the state administration.

Johannes Kanonier

Nikolaus Batlogg

 Martin Studer

Marion Heinzle

Andreas Siegel

TS: Could you tell us a little about the administration of Vorarlberg? What kind of resources do you manage and what is the size of the population you provide services for?

LVG: Vorarlberg is the westernmost and with 2,600 km² the second smallest federal state (Land) of Austria. The region shares its frontiers with the Swiss cantons of St. Gallen and Graubünden, the German state of Bavaria, the Principality of Liechtenstein and the Austrian state of Tyrol. At present 372,000 people are living in Vorarlberg. Since 1900 Vorarlberg’s population has climbed by 185 %, a growth rate higher than any other state in Austria. The Rhine Valley is the second most densely populated region in Austria, right behind Vienna. Vorarlberg is politically structured by 96 municipalities in 4 districts.

The general administration of the State of Vorarlberg employs around 1,600 persons. In accordance with the principles of modern administration management it is managed as a service provider. The State administration does more than just execute State law. It also implements federal law and fulfils a wide range of tasks of the State and federal governments in the form of private enterprise administration. The work of the State administration thus ranges, for example, from the granting of commercial licences to the granting of social welfare benefits, vehicle registration, planning permission, landscape and environmental protection matters and promotion of agriculture right up to the restoration of old buildings. As an internal service provider the department “Landesamt fuer Vermessung und Geoinformation” (LVG) is the central contact point in all questions concerning surveying and geo-information. In our department we are 20 employees working in the two divisions surveying and GIS.

The general tasks in the GIS division are

  • to capture own basic geodata (e.g. digital orthoimages, DEMs) or provided by third parties (e.g. cadastral information)
  • to collect and disseminate all administrational geoinformation
  • provide appropriate tools and the knowledge for the use of geo-information

At the moment we handle 1.5 TB of geodata: orthoimages 900GB , laserscanning DEM(s) 400GB, vector data 200GB

TS: Could you explain how your GIS group is setup? What staffing levels do you have, what are the key technologies you use?
LVG: We are 8 employees in the GIS group of the LVG, 7 hold an academic degree either in geodesy, geography, meteorology and geophysics or landscape planning, one person has a federal college of engineering degree.
Our GIS group works closely with 10 more GIS technicians in several other departments that have a high demand for spatial information (e.g. regional planning, road construction, water management, environmental protection). In this way the GIS in the state administration of Vorarlberg is organised in a network with the LVG as the leading hub in its centre.
In our group everybody has to be an all-rounder to a certain extent as we try to support the reliability of our services with an extended substitution plan. Nevertheless everybody has some key tasks, e.g. three persons are mainly responsible for the infrastructure, one person mainly for the acquisition of the basic geodata, one person mainly for training and transfer of knowledge, three persons for data delivery and support and so on.

Regarding key technologies we distinguish between the specialized information systems of the departments on the one hand where specific geodata is captured, updated and analysed and the collection and general dissemination of geodata on the other hand:

  • The departments decide in general on their own which information system the use and GIS is sometimes only a secondary aspect. So the spectrum of key technologies in the different information systems varies, not only but predominantly ESRI ArcGIS. In the emergency communication department for example an Intergraph application is in use.
  • The key for us to be successful as a central data provider is the commitment of all departments to deliver their geodata in open standard formats, which can be interpreted by the most commonly used GIS and the main CAD programs, e.g. shape simple features for vector data. With the collection, the central hosting and the dissemination of the administrational geodata we aim to deliver open, ready and easy to use geo-information internally, to our contractors and also to the public. This is our main task and for this purpose we use a wide range of applications, more and more FOSSGIS components.

TS: What kind of services do the GIS group provide both internally and to the citizens of Vorarlberg?
LVG:

Data Services:

  • Capture of basic geodata (e.g. digital orthoimages, DEMs, historical maps)
  • Central Hosting of administrational geodata
  • Data Download via SSH/SFTP
  • “R&D-harddisk” for educational institutions with all our geoadata

Web Services for most of the available data:

  • WMS, some WFS

Applications:

  • GIS Web Application “Vorarlberg Atlas” for the public and an internal “Pro”-Version
  • Desktop-GIS, adaption and extension

We deliver our data also manually in response to direct request mainly from contractors of the public administration. In addition to the technologies we provide training, information, communication, transfer of knowledge to our users.
Having a leading role in the cooperation between the state administration and all the municipalities in Vorarlberg we provide the basic GIS-infrastructure including our central data storage in a corporate network and share it with the communal administrations.

TS: What kind of effort has gone into building your FOSS based GIS? Who were the key people involved in the different aspects of the system?
LVG: The story started long before the first implementation of FOSSGIS components in 2007 with a few simple decisions concerning our central GIS data storage, mainly the decision to maintain the file-system and to migrate our data from proprietary formats to open standards (e.g. shape), which should support our open government efforts.  As the data basis is the core of every GIS the measure to get independent of proprietary formats and specific software producer in this part of our GIS was a very important step forward and put us in the position to consider FOSSGIS technologies in all decisions concerning the development and adaption of our GIS infrastructure. In this way we have been able to substitute for example the main proprietary server applications with Open Source software during the last years (e.g. ArcGIS Server with MapServer).

In this way it opened also the door for OpenSource when we had to decide about the successor product of ArcView 3.x The initial phase of the project included an observation of the software market where we checked a large amount of different, mainly open source software packages. GIS is identified and recognized as a cross-section topic and fortunately GIS is regarded as an integral part of the entire IT nowadays. In this context we have to emphasize our strong cooperation with the general IT department which it is crucial for our success. It is important that we act in accordance with the general IT conditions. It is also obvious to rely on the assistance of the general IT as it would be impossible for us to carry out projects in such large scales on our own.
The GIS technicians of the special department have a key role in all the projects. On one hand they provide the content of our central data storage to a large extent and on the other hand they support a large amount of users in their own surrounding So to say they are also the spokesmen for all our changes and innovations.

TS: Could you describe your deployment of QGIS? How long ago did you start using it, what were you using before? How many people use it, and what kind of activities do they carry out?
LVG: The title of the project that led to the deployment of QGIS was “Substitution of ArcView 3.x”. During the 15 years of use the standard program of ArcView 3.x had been permanently enhanced with an extensive menu for easy loading and visualization of data, a standard layout, some search routines and other functions. When we started the project in late spring of 2011 we were nevertheless quite surprised to find out that we still had almost 300 ArcView users spread over our entire administration although our WebGIS was very well established for years and very popular and widely used. ArcView was intended for users that need more functionality, stability and performance than our Web Application (like every Web Application) provides. So the goal was clear and so were the needs for the successor product. The new program should start at least were ArcView had got to in the end.

After the specification and evaluation phase we decided in October 2011 to continue with QGIS as the candidate for our new desktop GIS. We started our migration process immediately and the programming, evaluating data access possibilities, special user needs was my main task, including the programming of python plugins to enhance the functionality and provide features to our users, which are not part of QGIS at the moment. In addition, using the API to develop standalone applications for data checks, data updates etc. The rollout was strictly bound to the organisation wide WIN7 rollout which lasted from April to December 2012. So the schedule for development, testing and so on was really tight. We didn’t have any experience with QGIS, neither Qt, neither Python, neither the API before.

In addition all GIS-technicians from all different departments supported the deployment of QGIS by developing QGIS layers and projects for all their different data themes.
Together with the central IT department we developed also the rollout package for the automatic installation during the general Windows7 rollout. All the future updates and upgrades will be carried out with the same procedures.
Another task was the development and organisation of a training program, including the compilation of an extended training manual. We gave 27 one-day trainings for almost 300 QGIS users.

At the moment QGIS is installed on 400 desktops of the state administration. In the meantime in Vorarlberg other administrative organisation of the federal and municipal level and also some enterprises have joined us in our QGIS project and attended for also our trainings. This shows one of the most positive side effects we did not encounter at first: QGIS provides us the opportunity to deliver a complete and cost free GIS package (data and application) to our partners, to our contractors and to the public. This is open government in its best sense.

On the whole the state of Vorarlberg invested at least 3 person years and € 50,000 for development purposes.

 

ArcView menu

Above – ArcGIS menu

 

 

QGIS menu

 Above: QGIS Menu

TS: Did you customise QGIS using plugins or by adding core functionality in order to achieve your goals? If so what kind of customisations did you carry out?
LVG: Both. For the first part – core: Support for Measured Shapes, Oracle Spatial Provider (just finished) . For the second part – Plugins: A tool that makes the access to our central data pool much easier. It also includes functionality for searching inside the data, easy loading and displaying special information. Improvements for joining tabular data in QGIS (like in ArcView), Linear Referencing (Sourcepole). Some of them are already available on the official QGIS repository, others will follow as soon as they are multilingual and usable.

TS: What kind of issues do you encounter with QGIS, or has it all been plain sailing?
NB: A very difficult question – because it was almost really plain sailing. Of course that was steep and high mountain we had to climb (we didn’t have any experience with QGIS, neither QT, neither Python, neither the API before). But beside our problem of too less knowledge we had success. One important part of this access is that QGIS is – and provides – a sophisticated framework.
Without that, we would have had no chance. QGIS has some smaller bugs that are unavoidable for a complex application. But we can report them. One personal issue we have is: Loading shapefiles over the network is rather slow compared with ArcView. We suppose, the reason is latency in our network in combination with a lot of communication QGIS does in addition to the loading process? But, the conclusion is: QGIS was the right choice.

TS: If you had to rewind the clock and start your QGIS deployment again, is there anything you would have done differently? What advice based on your experiences would you give to others?
LVG: No. Advice: Don’t be conservative or ignorant. Only because proprietary software is more expensive than Open Source, it doesn’t have to be better. Conversely, Open Source can be a much better choice, as we found in QGIS. Spend the money for service (the work people do) not for software licence.

TS: Vorarlberg sponsored QGIS in 2011 – what motivated you to do this and was it a worthwhile thing to do? Was it difficult to get your bosses to agree to this?
LVG: As ArcView 3.x has been used over a period of more than 15 years our decision for QGIS is based on long term considerations. In this context we see it as our duty to make regular contributions to ensure a prosperous development of the whole QGIS project. Like in 2011 it is our aim for 2013 and further on to sponsor QGIS directly with a certain amount of money and indirectly by placing orders for certain developments and enhancements that are useful for our, and hopefully for a lot of other user’s purposes.
The decision to introduce QGIS in our GIS environment has been very well prepared and is approved by all key persons in our administration regarding both the decision level and the user level.

TS: What has your approach been in order to get support for your QGIS deployments? Do you find all the help you need from the community, or have you made use of commercial support providers too?
LVG: Till now, we didn’t need commercial support for QGIS itself. We had and have commercial support for certain development Tasks – e.g. the Linear referencing Plugin. For the most of my programming we use the QT class reference and the QGIS class reference. In most cases we can manage it that way. If not, we try to get help from the internet or at http://gis.stackexchange.com

A lot of information about our department and about the state administration of Vorarlberg (unfortunately all in German) you find at www.vorarlberg.at/lvg

Posted by & filed under OpenStreetMap.

Today is Open Data Day 2013. I did’t have much time to hack, but I made a few tweaks to my ‘just for fun’ osm-reporter project to provide average, min and max counts per active day for each user. I also did a bunch of code cleanups under the hood which probably nobody except me will care about :-) Here is what the new goodies look like.

 

New stats added to report

 

The new stats (shown in the green, blue and orance dots in the last row of the report) give you an idea of what kind of capture rate a person is able to sustain and what they peak at when really ‘going for it’. Only active days (where 1 or more ways were captured) are considered. I still have plans to add a bunch of different badges / achievements based on different metrics – something for the next time I have a day of fun hacking available to me. Head over to http://osm.linfiniti.com and zoom to your area to see what your your stats look like.

Posted by & filed under gdal, GeoDjango.

Yes I know its a bit wierd to see the word ‘OSX’ in an article by me…I recently bought a Mac for testing InaSAFE and fooling around with FOSSGIS software on OSX. Besides the 2 or three days it took to get it set up as a development machine for FOSSGIS stuff (as compared to a few hours – limited mainly by internet speed – on Linux) it hasnt been too bad an experience.

I used William Kyngesburye’s excellent packages for installing all the FOSSGIS dependencies (barring QGIS which of course I wanted to build myself). I’ve been trying to see if I can use OSX as a usable platform for doing GeoDjango work and ran into a few problems:

  • GDAL python package needs compilation when used in a python virtualenv (which of course I use)
  • William’s packages provide PostGIS 2.0 whereas GeoDjango really only works out of the box with 1.5 still

In this post I am outlining how I got GDAL to build in my venv. Note that I assume you have Williams frameworks and xcode / compilation toolchain in place.

source venv/bin/activate
pip install --no-install GDAL
cd venv/build/GDAL
python setup.py build_ext \
  --gdal-config=/usr/local/gdal/1.9/bin/gdal-config \
  --library-dirs=/Library/Frameworks/GDAL.framework/Versions/1.9/unix/lib/ \
  --include-dirs=/Library/Frameworks/GDAL.framework/Versions/1.9/Headers/
pip install --no-download GDAL

 

Now verify it is installed:

pip freeze

You should see GDAL listed. Another check:

 

>>> from osgeo import gdal
>>> gdal.__version__
'1.9.2'

 

Confirm it is really using your module in your venv:

>>> gdal.__file__
'/Users/timlinux/dev/python/catalogue/venv/lib/python2.7/site-packages/osgeo/gdal.pyc'

 

 

Posted by & filed under General FOSSGIS, OpenStreetMap, Web Development.

This weekend I implemented a new feature for my ‘just for fun’ project osm-reporter. The feature implements timeline reporting for Open Street Map contributors. Its probably easiest to explain with a screenshot:

 

OSM-Reporter with timelines

 

Here is another one showing a few charts together:

 

OSM-Reporter with timelines

I added the feature because I wanted to see how many person days were involved in data gathering for a particular area and feature type. It does have some limitations – it ignores deletion and ownership transfer of features. It does however provide a nice quick overview of effort. Try it on your own neighbourhood and see how much work went into the OSM coverage for your area!

I’ve also had some really awesome contributions from Yohan Bonifiace (added leaflet support, feature type switching, extent urls) and Sun Ning (added heatmap support). Its really great making a small, simple and limited scope project and seeing it grow with random hacks of kindness from strangers! Here is a little screenshot of the heatmap feature:

Heatmaps

 

I hope you all enjoy the new version, and look forward to more improvements and suggestions from the community. Its all freely available from my github repository. You can test out the current version of the software by visiting http://osm.linfiniti.com.

Posted by & filed under QGIS.

 A couple of weeks ago I had the pleasure of doing an interview on behalf of the InaSAFE project as part of the selection process for the Black Duck Software ‘open source rookies of the year’ competition. A week later we heard that we had made it into the top 10! The award is based on a selection or projects from the popular ohlog.net web site:

“Using data on open source projects from Ohloh.net and the Black Duck® KnowledgeBase™, Black Duck reviewed thousands of open source projects that were initiated in 2012 to select the fifth annual Open Source Rookies of the Year. Using a weighted scoring system, points were awarded based on project activity, commits pace, project team attributes and other factors. Black Duck determined the top 10 Rookie projects following an audit of its findings and normalization of scores.”

You can see all the winners listed at the official competition page here. We were also fortunate to be mentioned in Wired, DirectionsMag and various other places.

We have worked incredibly hard over the last year to make the InaSAFE plugin for QGIS, with a team of dedicated developers from AUSAID, WorldBank, Linfiniti, opengis.ch and a number of other developers and contributors, so it is really great to receive this acknowledgement! Here is looking to another great year of working on InaSAFE for 2013!

“We are pleased to recognize InaSAFE as one of the 2012 Open Source Rookies of the Year,” said Tim Yeaton, CEO and president, Black Duck Software. “The Rookies demonstrate how community innovation, particularly within the JavaScript and mobile projects, mirrors the need for innovation in web experiences, mobile devices and enterprise application. Being able to identify and showcase these up-and-coming open source projects is an important part of our mission at Black Duck to bring together the open source community and businesses around the world.”

Past winners of the prestigious award include Twitter Bootstrap, Cloud Foundry, Mozilla Persona (formerly known as BrowserID), Red Hat OpenShift, Eclipse Orion, Apache Rave, Salt Stack, OpenStack, Diaspora and many other notable projects.