I would like to announce QMap: A simple data collection application built using QGIS and Python.

QMap is a QGIS based field data collection application that was built by myself and a work college at Southern Downs Regional Council.  QMap is now opened source under the same licence as QGIS, GPLv2.  The project homepage can be found at http://nathanw2.github.com/qmap/ and source at https://github.com/NathanW2/qmap.

Before I go into to many more details I will preface with: The application is currently under active development and as such there might be bugs or little rough bits that I haven’t cleaned up yet. However having said that, the program is functional and we are using it at work for the purpose it was built.

Features

• Simple to use
• Simple to manage
• Simple to install
• Forms built using Qt Designer
• Loads normal QGIS projects
• Anything QGIS supports QMap does too (snapping, PostGIS, etc)
• It’s just QGIS with a tablet friendlier interface.
• Syncing support (MS SQL 2008 only at the moment)

The Story

The program was developed after we looked around and decided that nothing really fit our needs quite right and to our satisfaction. (Within budget of course)

We had a list of, I think simple, requirements:

• Must be simple to use by field staff
• Can deal with complex or simple forms
• Fully offline but with a syncing option
• GPS support
• Easy maintenance

The first point for me is a big one.  I have a seen a lot of data collection applications and unfortunately this is where I feel a lot of them fall down.  Most seem to be designed with people like me in mind, people who understand computers, understand menu systems, etc. If you work in local government or with an older age group of outside workforce you will know that this assumption doesn’t hold true.   Most of our field staff are not computer people, a few don’t even have home computer, expecting them to navigate a menu just to enable the GPS, or click on a small 16×16 pixel icon, on a tablet PC is not a option.

I decided to take the same approach as QGIS and use Qt Designer to build the forms. Why invent another tool? Using Qt Designer can also give us the flexibly of creating simple or complex forms in our own layout.

We do have pretty good 3G connection coverage over our region however we only have limited bandwidth to play with and having a solution that is full connected doesn’t really give the best user experience.  I would rather just store everything locally on the device and then sync when needed.  As all our layers are stored in MS SQL Server 2008 for the syncing we decided to use .NET Sync framework.  I would love to have sync support for PostGIS, Spatialite, or any other normal files, but it was out of scope for the project (at the moment).

For me easy maintenance means two things: not having to deal with crap loads of configurations; and having the power to change what I don’t like. For the first point I’m a big fan of convention over configuration.  I like to just drop stuff in a folder with a certain naming convention and it should just pick it up and work.  This also goes for the form bindings, just name the control the same as the field in the layer and QMap will bind it for you.  If I can follow a convention for things I have. Conventions make setup easier and consistent.

Once you have tasted the open source kool aid it can be quite hard to go back. Knowing that if there is a bug in QGIS I can fix it to make my project better is a comforting feeling.  There is also the lack of licence fees which makes open source very attractive for jobs with small budgets.

How does it work?

At the moment the core of the application is built as a QGIS plugin, however there is one little trick here that is worth mentioning. QMap is really a script that loads QGIS and sets the –configpath in order to load all the QGIS settings from a supplied path, inside the supplied path is the plugin. Think of it as a sandboxed QGIS which only loads the QMap plugin.  I’m also using the new customization function to remove all unneeded interface items.

Notes

Here are some notes worth bring up:

The application is still under development so things might change.
There is only point support at the moment, although adding line and region support wouldn’t be hard.
Syncing only works using MS SQL 2008 and the code is a bit rough. Will be cleaned up over time.
The build script only works on Windows and there is some win32 stuff for power management in the code. This is not because I don’t want to support the other platforms just that it was out of scope at work.
You need to be using the latest development build of QGIS (qgis-dev for those using OSGeo4W) this is because there have been a few bug fixes that make the application work as expected that aren’t in 1.8.

Recently there have been some great new additions to the QGIS project. Being part of such a fast moving project is a great feeling, and it’s only going to get better.

This post is going to be a quick over view of some of the newer features that I really like.

HTML Annotations

Well of course I like this one, I just added it. The reason I added this feature was because I really wanted to way to have popup images on the map canvas for flood damage reports on roads. I also wanted it dynamic so I could use template like syntax to replace values at run time.

The HTML annotations use QtWebKit and as a result support full HTML, CSS, and Javascript. The HTML can contain a QgsExpression – the same expression used in the expression labels and layer actions – inside [% %] which is replaced at run time with the data from the underlying feature e.g. [% "roadname" %]

I’ll let you think of some nice use cases for this new addition.

Project macros and non blocking notifications

This new feature comes from Giuseppe Sucameli of faunalia with the work done for ARPA Piemonte.

The task was to add Python macros that run when a project is open, saved, closed. As a side effect of the task the issue of security was raise and how to notify the user that macros are going to run.  For me this was less about security and more about how to present that information to the user without annoying the crap out of them. Most of the time popup dialogs in software are a anti-pattern and are often abused for tasks like this. So knowing I would throw my computer out the window if I had to dismiss yet another dialog I suggested a less intrusive method being used a lot these days. The handy slide out notification bar. Giuseppe  was very welcoming to the idea and implemented it nicely.

 Of course this addition can also expanded into other areas of the program. My first plan is to use it for notifying the user of plugins to failed to load.  There is nothing in QGIS that annoys more then starting and seeing this:

Dear dialog, why are you in my face!

To make matters worse if more then one plugin fails to load then I have to dismiss each dialog. So we can now use the notification bar to present it to the user in a nice non-blocking way. Something like “BTW four plugins failed to load at startup. What would you like me to do?”

Remember each time you use a blocking popup dialog it’s pretty much yelling at the user “OMG GIVE ME ATTENTION!! NO YOU CAN’T KEEP WORKING! GIVE ME ATTENTION!”

I’m working on a patch  to move this stuff into the notification bar just no ETA at the moment as I’m a bit busy.

Labeling improvements

Larry Shaffer has been working on some great improvements to the new labeling engine in order to make our maps look a lot more professional. Larry has been doing a lot of work in this area and is still going so I’m not going to go into all the details. However one new labeling feature that I really like is the ability to to set the  spacing between letters and words.

Before spacing

With a little bit of spacing

There is also the new ability to set the transparency of the label and the buffer.  The buffer transparency is something that I really like as sometimes you need a buffer but a solid buffer can then block out your map features; by adding a 45% transparent buffer I still have the labels pop off the map but not in your face or blocking features.  It’s hard to make a picture to explain it well so you’ll just have to experiment.

Project Templates

This one could be quite handy for people that make a lot of maps with the same base data. Thanks to Etienne Tourigny QGIS can now load projects as a template. This means you can create a project with all your base layers, styles, labels, etc, configured and then load it by default, or from the file menu, and you will have everything setup. All you have to do is save the a normal .qgs project file in

~/.qgis/project_templates folder and the project will be shown in the file menu.

Template list

You can also set the current project as the default template:

Handy!

Symbol Manager

And last but not least. This years GSoC student Arunmozhi got the improvements he had (has) been working on included into the master build. Arun was very welcoming to any feedback that Martin and I gave him about how we would like symbol stuff to work.  Anita Graser has already covered a lot of the new features over on her blog so I’m not going to go over everything again, although one thing she didn’t really touch on was the smart groups and tagging.

The tagging and smart groups are one of my favorite additions to the new symbol manager.

Symbol tagging

I love this new feature as not all the symbols I create belong to a single group so the tagging and smart groups fit this bill well.  I can now tag all the council symbols with ‘SDRC’ and include them a SDRC smart group but at the same time tag the sewer ones with ‘sewer’ and they can live in the sewer style smart group; or how about all sewer symbols that are also SDRC ones:

Smart group with sewer and sdrc symbols

You can then filter by this group in the symbol selector:

Filter based on smart group

Conclusion

I really love how fast QGIS is moving forward.  There almost isn’t a week that goes by that something isn’t getting done or someone is adding something new. Of course the great people on the project make this process a hell of a lot of fun and enjoyable.

Have fun experimenting! (remember that these features are in the master development build and may or may not have bugs)

I’m very please to announce the Australian QGIS User Group. Chris Scott (DMS) and I created this group because the interest in QGIS here is growing fast and it would be nice to have a place where us Aussies can hang out and chat about QGIS stuff, discuss local issues, organize local events, put shrimps on the barbie.

My ultimate goal would be to turn this into a full user group much like the Swiss QGIS User Group but for now we will keep it light.

Everyone is free to view the content of the home page and the Google group, however you have to request to join the Google Group to post.  This is only so that we can keep it local and not reduce the official QGIS mailing lists.

So in saying all that. If you live in Australia. Use QGIS, or are interested in QGIS. Feel free to join the group.

From QGIS 1.8 and onwards the Plugin Installer plugin will no longer include the option to add the 3rd party repositories.  This was by design and intended to move people over to using the official plugin repository at http://plugins.qgis.org/ so we can provide a richer experience and keep everything in one place.

If you have plugins that are still not on the official repository then I would strongly recommend that you migrate them as a lot of new 1.8 users will be missing out on your great work.

New fresh QGIS feature! So fresh in fact you can still smell the wet paint :)

QGIS (development build) can now display map tips using HTML (a subset anyway).

To enable the new map tips: Open the Layer Properties dialog for a layer and select the Display tab

Display tab to set HTML map tips

In action

Layer properties for HTML map tip

Notice how we can also use a QGIS expression. Anything inside [% %] will be evaluated and replaced with the value in real-time. We can even use a CASE statement. Pretty cool!

And the result when hovering over a feature

HTML in QGIS map tip? Yes! WOOT!

Hold on. Pause the track! We can even use some CSS to make it more fancy.

<style>
h1 {color:red;}
p.question {color:blue;}
</style>
<h1> [% "NAME" %] </h1>
<br>
<img src="[% "image" %]" />
<br>
<p class="question">Is this place a country?</p>
<br>
[% CASE WHEN "TYPE" = 'Country' THEN 'Yes' ELSE 'No. It is a ' || "TYPE" END %]

CSS in a html map tip

Happy Mapping :)

Something that I need to do now and then is generate points along a line at supplied distance.  I had never really looked into doing it in QGIS until this question poped up on gis.stackexchange.com.  This is a quick blog post because I thought it was a pretty handy little thing to do.

In the development version there is a new method on QgsGeometry called interpolate. This method takes a distance and returns a point along a line at that distance. Perfect! We can then just wrap this in a loop and generate a point increasing the distance as we move along

from qgis.core import (QgsFeature, QgsGeometry,
                       QgsVectorLayer, QgsMapLayerRegistry,
                       QgsField)
from PyQt4.QtCore import QVariant
from qgis.utils import iface

def createPointsAt(distance, geom):
    length = geom.length()
    currentdistance = distance
    feats = []

    while currentdistance < length:
        # Get a point along the line at the current distance
        point = geom.interpolate(currentdistance)
        # Create a new QgsFeature and assign it the new geometry
        fet = QgsFeature()
        fet.setAttributeMap( { 0 : currentdistance } )
        fet.setGeometry(point)
        feats.append(fet)
        # Increase the distance
        currentdistance = currentdistance + distance

    return feats

def pointsAlongLine(distance):
    # Create a new memory layer and add a distance attribute
    vl = QgsVectorLayer("Point", "distance nodes", "memory")
    pr = vl.dataProvider()
    pr.addAttributes( [ QgsField("distance", QVariant.Int) ] )
    layer = iface.mapCanvas().currentLayer()
    # Loop though all the selected features
    for feature in layer.selectedFeatures():
        geom = feature.geometry()
        features = createPointsAt(distance, geom)
        pr.addFeatures(features)
        vl.updateExtents()

    QgsMapLayerRegistry.instance().addMapLayer(vl)

The above code might look a bit scary at first if you have never done any Python/pyqgis but hopefully the comments will ease the pain a little. The main bit is the createPointsAt function.

Cool! If we want to use this we can just stick it in a file in the .qgis/python folder (lets call it pointtools.py) and then run import pointtools in the python console.

So lets have a go. First select some objects then run the following in the Python Console

import pointtools
pointtools.pointsAlongLine(40)

That will generate a point every 40 meters along then selected lines

Distance nodes along line in qgis-dev

To generate nodes along different lines, select the new features, then call pointtools.pointsAlongLine(40) in the Python console.

Simple as that!

(Who knows, someone (maybe me) might even add this as a core object function in QGIS in the future)

We are currently working a kerb line digitization and defect capture project at work.  This process involves looking at the aerial photo along with video of the roads and drawing lines on the kerb layer using QGIS, not overly hard just tedious.  As I mentioned in my Using QGIS in local government post, the defect points are snapped to the lines in order generate the distances, or chainage, along the kerb line for reporting reasons e.g Defect 1 at 10m, Defect 2 at 11.5m.  In order for this to happen the kerb lines must be running the correct direction, the correct direction here is defined by the road direction.    The kerb line also has an attribute to define what side of the road it is on, left or right, in reference to the direction of the road center line.

So we have two conditions:

1. Must run the same way as the road
2. Must have the correct side of the road assigned

The problem is how to clean up any lines that are already wrong (we were 90% of the way though when the above conditions were added).

In QGIS we can add line directions by using a two layer symbol for the kerb line:

Line with direction

Showing line direction

Not too bad but I still have to focus a lot to see which direction the lines are going.  Viewing them at this scale is fine but once you start to move the arrows all become a blur after a while.  Plus this also doesn’t let me check the side of road attribute quickly. Yes I can look at the color and the label but still I would like a quick way to look at line and see if it is facing the right way and with the correct side of the road attached.

What we can do is offset the arrows on the line so that they will be on the inside of the kerb line, between the kerb line and the road direction markers when they are facing the correct direction and have the correct side of road attribute.

For the left side we will of set the marker Line offset to 3, and for the right side we offset by -3

Offset arrow

Using the 3 and -3 offsets will mean the arrows are rendered on the inside of the kerb line if the line is facing the correct direction. Lets have a look

Opps no that isn’t right

Ohh no that isn’t right!  See how the line directions are facing the wrong way and it is showing the arrows on the outside of the line, further away from the road line.  This isn’t right.  Lets swap those line directions using a plugin that I wrote called Swap Line Direction (Search for ‘Swap’ in the plugin installer).

Lines facing correct direction

So now the arrows are on the inside of the line and are facing the correct way.

But wait there is more

This styling also helps me check that it is assigned the correct side of the road.  If we assign the top line the value ‘left‘, which is wrong in this case, we will see that the arrows are now on the wrong side of the line

Wrong side of road assigned

Of course here the obvious thing here is that there is two green lines which you can’t have, but also having the arrows on the wrong side of the line lets you quickly see which one is wrong.

It is impossible to get the arrows to be on the inside and facing the correct way.  If we swap the direction of the line the arrows are now on the inside but are facing the wrong way

Wrong side running the wrong way

Using this style trick allows me to quickly see at a glance which sections might be wrong when I have more then a single road in view

Summary

This post is a quick example of how you can use QGIS styles to help you visually validate you data.  The way I have done things in the post my not work for you and you might find it less helpful and more distracting then I did; however I found it worked well with my eyes and reduced strain.

Atlas is a super handy little QGIS plugin that lets you generate a map series from a composer and a coverage (grid) layer.

The authors of the plugin are now looking for some funding to make it a core part of QGIS.  Making it part of QGIS would mean integration into the composer and maybe APIs for developers to add extensions.

The authors are looking at raising 7000€ in order top get the work done, but the more money raised the more features that are added.

More information can be found at http://www.oslandia.com/?p=1243

Version note: This will only work in the latest dev build of QGIS – not in 1.8

The lack of uni for the next couple of weeks has left me some time at night to work on some features that I really wish QGIS had.  One of these features was better date and time support in the expression engine.  Date and time is an important concept when working on inspection data and not being able to style my features in QGIS using date operations was bugging me.  So in good open source fashion I added some.

Here are the current functions (more to come in the future):

• $now – returns the current date and time
• age({datetime},{datetime}) - returns the difference between the two dates
• todate({string}) - converts a string to date type
• totime({string}) – converts a string to time type
• tointerval({string}) – converts a string to a interval type (details below)
• day({datetime} or {interval}) – returns the day from a datetime type or the number of days in a interval.
• hour(…) – Same as above but for hours
• minute(…)  - Same as above but for minutes
• second(…)  - Same as above but for seconds
• day(..)  - Same as above but for days
• week(..)  - Same as above but for weeks
• month(…)  - Same as above but for months
• year(…) - Same as above but for years
• {datetime} – {interval} = {new datetime} – returns a new datetime subtracting the interval 
• {datetime} + {interval} = {new datetime} – returns a new datetime adding the interval

The interval type

Functions like age(..), tointerval(), {datetime} -/+ {interval}, day(..), hour(..), etc, use, or return, Intervals.  An Interval is a measure of time that we can use for different things.  An example of an Interval is ’1 Year 2 Months’ this is then converted to a number of seconds and used for any calculations.

For example one can take away 10 days from the current date by doing the following ( -> marks the output ):

todate($now - '10 Days')
-> 2012-06-20

as

todate($now)
-> 2012-06-30

We can also do something like:

todate($now + '2 Years 1 Month 10 Days')
-> 2014-08-10

The age() function will return an interval which we can use extract what information we need.

The number of days between two dates:

day(age('2012-06-30', '2012-06-10'))
-> 20
-- Think of it as '2012-06-30' - '2012-06-10'
-- Note: day(), month(), etc, functions return doubles so you can get
-- 21.135234 days if you are using date & time type rather than just date type
-- wrap the result in toint() to get a more sane output.

Day() will also work on a plain date:

day('2012-06-30')
-> 30

We can even get the number of seconds between two dates:

second(age('2012-06-30', '2012-06-10'))
-> 1728000

Currently the only date format supported is {year}-{month}-{day} as seen in the examples above. Shouldn’t be too hard to add support to the todate(), todatetime(), totime() functions for giving it a pattern to use when converting the string e.g. dd-mm-YYYY, or something like that.

More on this fancy new stuff

When I wrote the new expression builder dialog a while ago I made it dynamic so that any new functions added to the expression engine will show up automatically.  So here they are:

List of new date and time functions.

We can also use these functions in the rule based rending, which is where the power really comes in.  Lets see something like that in action:

Styled using days and years

Should be pretty straight forward to understand. We are using the age() and day() functions to style the events that are older than 30 days, within 30 days, for today, or in the future.  We also check the year of the event using year() and year($now) to make sure we only see this years events, or style them differently depending on if they are last years events or in the future.

This is the result of the above rules:

Result of using date functions in rule based renderer

I’m also using the date functions in the expression based labelling to label the features using the following expression:

CASE
WHEN year( "dateadded") < year($now) THEN
	'Last Year'
WHEN day(age("dateadded", $now)) < 0 THEN
	day(age("dateadded", todate($now))) || ' Days old'
ELSE
	day(age("dateadded", todate($now))) || ' Days to go'
END

Well that’s it. Hope you find it handy in your day-to-day mapping. I know I will be using it a lot.
Thanks to Martin and Jürgen for the code reviews during the process; venturing in an unknown part of the code base always makes me nervous but that is all part of learning, and sometimes you can make some pretty cool stuff.
Some other random notes: The general idea has been modelled of how Postgres handles dates and times, it’s not an exact copy but follows the same kind of ideas. The interval class also uses the same number of seconds for one year that postgres does so that we can be consistent with the output.

So this years first uni semester is done and dusted, now I have some free time.  Blog all the things!

This is a follow up post for discussion that was started on LinkedIn about showing features older, or newer, then a certain date different colours   The main post was about using free, or low cost, solutions in order to aid in mapping water networks. I recommend that everyone watch it. A very good presentation.

I recommended that you could use the rule based rendering engine but the expression engine in QGIS doesn’t have any date functions yet.  All good we can add them if we need and once I get my head around the expression engine I plan on doing exactly that. But for now we can do it a different way.

We are going to use Spatialite, but any database will do (syntax and process will vary).

Lets have a look an inspection layer we have in our Spatialite database viewed in QGIS:

Pretty boring and hard to see what has been done in the last 30 days.  Now with the lack of support for dates in the expression engine we have to use another methods.  For this example we will use the really handy DBManger plugin that now ships with QGIS from 1.8.  Load it, connect to your database, and run the following query:

SELECT id,
              CASE WHEN DATE("date_checked") > DATE('now', '-30 days') THEN
                         'Within 30 Days'
              ELSE
                         'older'
              END as age, date_checked, geom
FROM  "inspections"

As you can see anything that is within the 30 days now has the “Within 30 Days” string in the age column, or else it has “older”.  CASE statements can be very powerful things in SQL sometimes.

Now load it into QGIS, style, and label it using the new age column

and Bob’s your uncle

You now have a layer that is style based on age but is also dynamic.  Adding a new inspection point will now will be styled according to those rules. (Although you will have to edit the normal inspection layer with it turned off as views/queries are not editable – without some setup anyway)

It might be a simple thing to some but sometimes it’s hard to find the right words to describe what you want when you are looking for this kind of thing. So hopefully this has helped a few people get started with visualizing their time/date based data in QGIS.

Happy mapping!

After almost a year and a lot of hard work QGIS 1.8 is finally out.  This is the best QGIS version so far, packed full of fancy new features.

The official release notice can be found here: http://qgis.org/index.php?option=com_content&view=article&id=149 and downloads can be found at http://download.qgis.org

Here is the change log of all the new stuff in 1.8:

- QGIS Browser - a stand alone app and a new panel in QGIS. The
browser lets you easily navigate your file system and connection based
(PostGIS, WFS etc.) datasets, preview them and drag and drop items
into the canvas.
- DB Manager - the DB manager is now officially part of QGIS core. You
can drag layers from the QGIS Browser into DB Manager and it will
import your layer into your spatial database. Drag and drop tables
between spatial databases and they will get imported. You can use the
DB Manager to execute SQL queries against your spatial database and
then view the spatial output for queries by adding the results to QGIS
as a query layer.
- Action Tool - now there is a tool on the map tools toolbar that will
allow you to click on a vector feature and execute an action.
- MSSQL Spatial Support - you can now connect to your Microsoft SQL
Server spatial databases using QGIS.
- Customization - allows setting up simplified QGIS interface by
hiding various components of main window and widgets in dialogs.
- New symbol layer types - Line Pattern Fill, Point Pattern fill
- Composers - have multiple lines on legend items using a specified character
- Expression based labelling
- Heatmap tool - a new core plugin has been added for generating
raster heatmaps from point data. You may need to activate this plugin
using the plugin manager.
- GPS Tracking - The GPS live tracking user interface was overhauled
and many fixes and improvements were added to it.
- Menu Re-organisation - The menus were re-organised a little – we now
have separate menus for Vector and Raster and many plugins were
updated to place their menus in the new Vector and Raster top level
menus.
- Offset Curves - a new digitising tool for creating offset curves was added.
- Terrain Analysis Plugin - a new core plugin was added for doing
terrain analysis – and it can make really good looking coloured relief
maps.
- Ellipse renderer - symbollayer to render ellipse shapes (and also
rectangles, triangles, crosses by specifying width and height).
Moreover, the symbol layer allows to set all parameters (width,
height, colors, rotation, outline with) from data fields, in mm or map
units
- New scale selector with predefined scales
- Option to add layers to selected or active group
- Pan To Selected tool
- New tools in Vector menu - densify geoemtries, Build spatial index
- Export/add geometry column tool can export info using layer CRS,
project CRS or ellipsoidal measurements
- Model/view based tree for rules in rule-based renderer
- Updated CRS selector dialog
- Improvements in Spatial Bookmarks
- Plugin metadata in metadata.txt
- New plugin repository
- Refactored postgres data provider: support for arbitrary key
(including non-numeric and multi column), support for requesting a
certain geometry type and/or srid in QgsDataSourceURI
added gdal_fillnodata to GDALTools plugin
- Support for PostGIS TopoGeometry datatype
- Python bindings for vector field symbollayer and general updates to
the python bindings.
- New message log window
- Benchmark program
- Row cache for attribute table
- Legend independent drawing order
- UUID generation widget for attribute table
- Added support of editable views in SpatiaLite databases
- Expression based widget in field calculator
- Creation of event layers in analysis lib using linear referencing
- Group selected layers option added to the TOC context menu
- load/save layer style (new symbology) from/to SLD document
- WFS support in QGIS Server
- Option to skip WKT geometry when copying from attribute table
- upport for zipped and gzipped layers
- Test suite now passes all tests on major platforms and nightly tests
- Copy and paste styles between layers
- Set tile size for WMS layers
- Support for nesting projects within other projects

Thanks to all the sponsors and everyone who put a lot of work into this release!

If you are in Australia and are looking for some QGIS training Digital Mapping Solutions have just announced that they are now providing a  2 day Introduction to QGIS training course.

The course covers such things as:

• User interface and basic tools
• QGIS projects and preferences
• Coordinate Reference Systems and On-The-Fly (OTF) projection
• Map navigation and QGIS tools
• Vector map objects: Adding, deleting & modifying
• Attribute data: Field editing & modification
• Layer manipulation
• Digitizing
• Styles and labelling
• Object manipulation: Combine, split & buffer
• Registering raster imagery
• Creating layouts for printing
• Utilising WMS and WFS Web Services
• Plugins
• Database connectivity

More details and a booking form for the course can be found at http://www.mapsolutions.com.au/training/quantum-gis/introduction-to-qgis.aspx

Training like this is great for QGIS popularity in Australia.  Working in local government I know how hard it can be to get IT to say yes to software unless there is formal training of some sort, and now thanks to DMS there is.

Enjoy :)

Something that I always find interesting is how people are using different open source tools to get their work done.  This post attempts to outline how I/we are using QGIS at work for different projects.

Kerb mapping, condition, and defect pickup

This project is currently being done by a 67 year old foreman who has worked for the council for a very long time and has great knowledge of the town.   QGIS, with the main working layers stored in PostGIS, was setup so that he can:

• Digitize kerb lines from aerial photos.
• Split the existing kerb lines into segments depending on different asset rules.
• Give each segment an overall condition rating.
• Add defect points along the each kerb segment e.g. broken, lifted, etc,

Each defect point is snapped to the underlying  kerb line and chainages (distance along line) is generated using a update statement at the end of the project (could be done using a insert trigger if needed) using ST_line_locate_point(line, point).

Defect points coloured by risk captured against the kerb line

Overall QGIS has been great for this project.  The built in data entry forms have been a great help to allow fast and correct data entry. Each form has four drop downs all with present values and descriptions to aid in data entry.

Flood damaged claim maps

We recently suffered, like the rest of Queensland, some really major flooding which caused large amounts of damage to our road infrastructure. We got off pretty light compared to some places, nevertheless we still had a lot of damaged assets.  And so began the process of collecting data that could be used for state government funding claims.

Anyway, onto the QGIS bit.  QGIS was installed on one of the main engineers computers in order for him to make maps for each claim.  Having the ability for him to have one map window but multiple frames in the composer helped him to create multiple  views of the same data with ease.

In total there are 42 QGIS project files with a main project file which served the base layers to the other projects, using the cool Embed Layers and Groups feature.  This means any change in main base project was reflected up(down?) to the other projects next time they are opened.  The main project file has things like, property layer; normal road layers, with labels; road layer with roads for claims.   The other 42 projects have a filtered, and styled, road layer to only show roads in that batch, and its composers (print layouts).

Normally we would use MapInfo for this kind of thing but consider this: There are at least 3 print layouts per claim, each layout could have more then one map frame.  Now with MapInfo only being able to have a 1:1 ratio between the map window and the map frame in the layout you would need at least 3 map windows per claim.  Quick calc:

42 * 3 =  126+n map windows + 126 print layouts (n = extra map frames in layouts)

Each map window has its own copy of every layer, making change once apply every where changes hard.  This of course doesn’t apply to styles as they are stored in the .map (tab) file, but does for labels, style overrides, etc.   I’ll pass.

QGIS is no means perfect for printing or print layouts but the 1:N map window to map frame ratio worked really well for this project.  The styling options in QGIS also helped to change the display of the map depending on what was needed to be shown quickly, one even used the rule-based rendering.

You get the point.
Moving on.

Processing GPS photos with road chainages

This one I am quite proud of.  It’s nothing fancy but still saves a lot of time.  While not really QGIS only but a combination of QGIS+Spatialite it process GPS photos and assigns them a road name and chainage.

The issue: A large influx of GPS photos for the different flood damage projects and the need to process them quickly so that they got assigned to the correct road and chainage.  Now you can map GPS photos easy enough but then you still have to go to each one and assign a road name, chainage, and move it into the correct folder.  To hell with doing that by hand, this is why we invented GIS.

The result is a little (140 line) python script that:

1. extracts the coordinates from each photos,
2. finds the closest (within tolerance) road distance node (distance nodes are generated at 5m intervals along the road, around 800,000 in total for the whole shire),
3. gets the road name, locality, and chainage for that node,
4. creates a folder with that road name,
5. renames the photo with {name} @ {chainage},
6. moves it into the road name folder it is assign to.
7. inserts a record for that photo into the spatialite database that can be viewed in QGIS.

Porting our planning scheme maps

I have been involved in creating, and maintaining, our planning scheme maps for the council.  It’s been a pretty fun project, apart from the constant moving target that is the state planning specifications, but I digress.

Planning scheme in QGIS

This project was done, and still is, in MapInfo. While there is nothing technically wrong with that, it has become a bit more of a pain to maintain then one would hope.  The planning scheme is not just one map but rather a series of different maps all with different scales and requirements.  I’m sure by now you can start to see the issues that can arise:

1. No dynamic scale bar for layouts (not even a scale bar object rather just text and boxes made to look like a scale bar. With no group items feature moving these around is a pain).
2. 1:1 map window to map frame means excessive map windows when the data is all the same with just different views.
3. Legends don’t support ordering, adding items, or groups.
4. With no embedding base maps feature like in QGIS it’s hard to change one thing and apply it to all the map windows/workspaces.

The specifications also ask for lines with symbols along them to show things like bikeways, footpaths etc, something that can’t be done in MapInfo, well it can by using the line style editor but I would rather stab myself in the eye.

The one thing I haven’t fully worked out how to do in QGIS yet is fully automate the printing process. Currently I open MapInfo using a batch file and pass it a workspace and MBX which prints the layouts and exits. I do this for each map type.    In QGIS I have a few options:

1. Create a plugin that runs though each project and prints off its composers.
2. Create a python script that runs from a batch file using qgis.core and qgis.gui QGIS python bindings.
3. add a –code option to the command line of QGIS so that you could run: qgis.exe –noplugins –code “print.py”, which would open QGIS and run the python code and exit.

I’m yet to explore what option is the best for this project but I’ll get back to you on that.  Once I have the above issue sorted I plan on creating the maps in QGIS to see how it would turn out (time permitting)

One thing I didn’t explain very well  in my other post was how to correctly set up value binding between your custom form and QGIS.  I didn’t explain it because at the time I didn’t know how.

The other day I was building a custom form QGIS for a project I am working on. I had named all the fields right, set the ui as the edit form for the layer, but only the line edits were getting bound to the correct values.

 So having a dig around in the code I noticed that QGIS uses the same methods to bind the built-in edit forms as it does for the custom forms, meaning that you must set what kind of control you want to use in Layer Properties -> Fields 

Correctly binding values

First create the form with the controls you need, remember to name them the same as your fields.

Custom form with controls using the same name as the fields

Note that here I have a QComboBox with the FeatureCla name, this will bind the combo box to the FeatureCla field in my dataset in QGIS.

Now set the custom form as the Edit UI for the layer

Set the Edit UI to your form

Tip: You can use relative paths if you store the form along side your project file

 Flick to the Fields tab and set up the Edit Widget type for each field that you have used on the custom feature form.

Set the Edit Widget that matches your control

I have set the FeatueCla field to use Unique values widget, this tells QGIS to collect all the unique values from that column and add them to the QComboBox.  There are a range of different edit widgets you can set

Each will map to a different set of control types (Widgets) e.g. If you want to have a checkbox on your form you must select Checkbox in the Edit Widget list to get it to bind correctly.

Save the properties and head back to you map.  Use the Identify Tool to select a feature.

Values bound to form

And that is it. Pretty cool hey!

Final thoughts

This is one feature I really like in QGIS.  The ability to create custom forms for people to do data entry without the need to build a plugin is very cool.  Couple this the built-in GPS module for QGIS and you have yourself a nice simple field data collection program.

I have some ideas to make this feature even more powerful, but more on that later once I get some time to add it in.

Ok the title is a bit of a lie. QGIS did support MS SQL Server 2008 before by using OGR but this is a native provider so it’s a lot more integrated..

Good news everyone!

QGIS now has a native MS SQL 2008 provider. The provider can found using the new toolbar button (purple icon) or in the MS SQL node in the QBrowser tree. The provider also supports drag and drop import.

The work was sponsored by Digital Mapping Solutions (Australia) and completed by Tamas Szekeres

Any bugs can be assigned to “tamas” on hub.qgis.org.

A big thanks to both Digital Mapping Solutions and Tamas.

This addition will open QGIS up to a whole new set of users who have to use MS SQL but love QGIS.

Currently this is only in master but I will be in the 1.8 release when it comes out.

Note: At the moment you have to have a geometry_columns table in the database in order to connect, this table is the same format used by PostGIS and can be made by importing a layer using the ogr2ogr method. There will be a fix coming for this at some stage.

Did you know QGIS is also on all the major social networking sites?

No?

Well now you do :)

Google+
Facebook
Twitter

Search for QGIS on Twitter
Search for QGIS on Google+

Come join the discussions on your preferred platform. If you prefer not to say much, social networks are still a great way to keep up to date with all the cool stuff happening in QGIS land.

Anita Graser, of underdark.wordpress.com, and myself manage the Google+ and Facebook pages.

If you have something cool that you have done with QGIS and would like to pimp it to the world, free feel to contact me, or even just mention QGIS in the post, Mentioning QGIS will make it show up in the search on Twitter or Google+.  I keep a keen eye on all sites for anything cool that I can reshare on the main QGIS pages.

Just a quick non-technical blog post.

My wife and I would like to introduce Harrison Mark Woodrow. The newest edition to our family.

Nothing has ever come close to how nervous I was yesterday morning, not knowing if Stace would finally be induced and we would have our little man. It was a long 13 hour labour but at 11:53pm (DST) 14/02/12 in Tweed Hospital, Australia little Harrison was born with all his cuteness.

All I can say is thank science for all the handy painkillers that have been created/made/found. That Epidural is one handy analgesia. Seeing my wife in so much pain before the Epidural was one of the hardest things I have ever had to the deal with.  Ripping my heart out, hard.

Both Harrison and Mum are doing well.

Recently I have started reading two books, well one book and one collection of stories, on open source software and interaction design. This is all in an attempt to suck a little less at working on software, open source in particular i.e become a better person, programmer, and open source contributor.

The Books

About Face 3

The first book I started reading was About Face 3: The Essentials of Interaction Design. In my opinion this is a must read for anyone building software, or at least those who work with the UI and users.

One design principle the book covers is not leaking the Implementation Model into the UI. The Implementation Model is when the UI is designed around how the code works in the background, rather then the users mental model of the task. Most users don’t understand complex structures, or nested hierarchy, but yet we see it a lot within UI design due to it fitting the code design and the programmers view of the world perfectly well. Try explaining the branching, merging, and rebasing model of a Git tree to a non programmer and you will see what I mean.

That one design principle alone echoes strong with me, as lets be honest, most programmers are not UI designers and tend to do a pretty bad job at it, even me. When working on a feature the UI tends to be the last thing that is thought of and is just a quick interface for the code underneath.

Loss of orientation is another big thing. And what is the quickest way to get lost in a program? Dialog boxes! They popup, get in your face, most of the time have to be dismissed before you can see the results. Generally just a bad idea and people tend to get lost quick once you have more then one on the screen.

Those two principles alone are not going to make you a good UI designer but at least they give you something to ask yourself when working on a UI:

• Do I really need that button. Can’t I just do it for the user.
• Is there a reason this needs to be shown as a nested tree. Why not just a flat list. Can I do the same action with a different control.
• Do I really need a another dialog here. (This applies to annoying the user when something happens e.g non-fatal warnings or errors)

Open Advice

The second (free) book was brought to my attention by Brian on my Google+ feed, entitled Open Advice. Open Adivice is a collection of stories from people with differencing experience working on open source projects telling their stories on what they learnt and what they wish they had known when they started. The book aims to cover the answers to “”What would you have liked to know when you started contributing?”, which it does quite well.

It’s not heavy reading, but the story telling works well to bring home some of the things that everyone working on open source. Some of the stories cover things like; getting your first patch rejected; having a bad first IRC experience; writing good documentation; how to be a better community.

The book covers a range of topics so it’s a good read for everyone, regardless of your experience or knowledge area.

Those who don’t know history are destined to repeat it

Summary

So they are my two books for the start of 2012, hopefully they are a good read for you. I think in order to be good at anything you should strive, every year, to suck a little less at everything you do, even just a little bit. My role model in the software world is Scott Hanselman, generally a pretty cool dude, and has echoed a lot on his podcast this notion of learning sometime new every year to just get a little better at what you do, to become a better person and programmer.

So even if your not into learning or reading a lot, just reading these two books I can almost guarantee you will come out a better person for it.

Gary Sherman has just published a new Python plugin for QGIS that I think people will find very handy, I know I will.  The plugin allows you to run Python scripts inside QGIS for tasks that don’t really require, or warrant, a whole plugin.

Go check out Gray’s post about the new plugin at http://spatialgalaxy.net/2012/01/29/script-runner-a-plugin-to-run-python-scripts-in-qgis/

The new plugin can be installed via the Plugin Installer using the “runner” or “script”.  The Plugin Installer is another one of my favorite plugins for QGIS, being able to push out a new plugin and know that everyone can get it is a good feeling :)

The rule based rendering in QGIS has just got a make over to improve in some of the old usability issues it used to have.  Most of the improvements are UI related. If you would like to try them out you will need to grab a copy of the latest dev build (qgis-dev in OSGeo4W)

Main improvements include:

• Nested rules.  If the parent rule evaluates to false none of the child rules are applied. This replaces the priority system in the old dialog.
• Disable symbol for rules. Rules with no symbol only act as a check for the child rules e.g nothing is rendered for the rule but child rules still are (unless also disabled).
• Drag and Drop rules (multi-selection is supported).  Rules can be dragged onto other rules in order to nest them and set up a rendering hierarchy.
• Inline editing of rule labels, expressions, scales
• Overall tweaks to the dialog

The new rule dialog

As you can see in the screenshot, the rules are now organized in a tree which clearly expresses which rules should be applied and when.

In the example above, all the rules under the Sealed rule will only be applied if that rule is true. The old system would have you managing all rules in one big list and dealing with priorities in order to get the rules to apply right, the new dialog is a major improvement.

And the results! As you can see below, QGIS will only render the colored squares if the Sealed rule is true otherwise it just shows a green line.

The rules applied

The work was sponsored by Ville de Morges, Switzerland and developed by Martin Dobias.  Thanks to both of them for these improvements.

More info:

• See Martin’s notes for a more detailed breakdown, and maybe some future stuff to come, of the improvements  http://lists.osgeo.org/pipermail/qgis-developer/2012-January/017909.html

Note: As this is a brand new feature there might be some bugs, or things that don’t quite work as expected. If you do find something don’t hesitate to file a bug report at hub.qgis.org so it can be fixed, or at least known about.