PostGIS 2.0 is out and the awesomness continues! You can install PostGIS 2.0 on Ubuntu using packages which is exactly what I am going to show you here. Read on for details on how to get up and running and do your first simple raster analysis! Note: You should make good backups first!   Before we... Read more »

PostGIS 2.0 is out and the awesomness continues! You can install PostGIS 2.0 on Ubuntu using packages which is exactly what I am going to show you here. Read on for details on how to get up and running and do your first simple raster analysis!

Note: You should make good backups first!

Before we begin, you should uninstall your existing postgis packages:

sudo dpkg --purge postgis postgresql-9.1-postgis

Then add a new repository and install PostGIS from there (based on this post):

sudo apt-add-repository ppa:sharpie/for-science  # To get GEOS 3.3.2
sudo apt-add-repository ppa:sharpie/postgis-nightly
sudo apt-get update
sudo apt-get install postgresql-9.1-postgis

Next we should create a new template database (optional but recommended).

createdb -E UTF8 template_postgis2
createlang -d template_postgis2 plpgsql
psql -d postgres -c "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis2'"
psql -d template_postgis2 -f /usr/share/postgresql/9.1/contrib/postgis-2.0/postgis.sql
psql -d template_postgis2 -f /usr/share/postgresql/9.1/contrib/postgis-2.0/spatial_ref_sys.sql
psql -d template_postgis2 -f /usr/share/postgresql/9.1/contrib/postgis-2.0/rtpostgis.sql
psql -d template_postgis2 -c "GRANT ALL ON geometry_columns TO PUBLIC;"
psql -d template_postgis2 -c "GRANT ALL ON geography_columns TO PUBLIC;"
psql -d template_postgis2 -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
createdb training -T template_postgis2

Ok now we can load a raster (see sample data download below):

raster2pgsql -s 4326 srtm_4326.tif | psql training
shp2pgsql -s 4326 -d -g geom -I places.shp places| psql training

Good - now our spatial database is ready to use - and has raster support! Here is a nice example of what you can do. The query looks up the altitude from the SRTM raster for each place listed using the ST_Value function:

select ST_Value(rast, geom, true) from places, srtm_4326;

It should produce something like this:

Further reading: A really good place to start is the Boston GIS cheatsheets - I am really looking forward to exploring all the new features that are available in PostGIS 2.0, thanks to all those involved in building it!

Sample data for the example listed

I’m running a training course next week and will be basing it on the nightly build of QGIS, so I wanted to quickly build an installer for use on the course. I simply ran these commands (starting in a checkout of the Quantum GIS source code). sudo apt-get install nsis cd ms-windows/osgeo4w ./creatensis.pl qgis-dev cd... Read more »

I'm running a training course next week and will be basing it on the nightly build of QGIS, so I wanted to quickly build an installer for use on the course. I simply ran these commands (starting in a checkout of the Quantum GIS source code).

sudo apt-get install nsis
cd ms-windows/osgeo4w
./creatensis.pl qgis-dev
cd ..

Now you should see the new installer listed:

1.1K 2012-05-03 13:56 python_plugins.nsh
25K 2012-02-01 09:01 QGIS-Installer.nsi
111M 2012-05-03 14:13 QGIS-OSGeo4W-1.8.0-14615-Setup.exe <-- new installer
9.4K 2011-06-05 00:54 QGIS-Packager.bat
744 2012-02-01 09:01 quickpackage.sh
11K 2011-06-05 00:54 README.html
7.1K 2011-06-05 00:54 ui.nsh

Be aware that the process takes some time and doesn't really show much on the screen other than an dump of the nsis file created while it is happening. Just sit tight and wait for the command line prompt to return to you.

Its been two weeks now since I returned from the QGIS Hackfest in Lyond, France, but I haven’t had the time to write up my experiences yet….until now. Read on for more! It was the first time that the hackfest (which is a twice yearly event) has been held in France. Lyon is a lovely... Read more »

Its been two weeks now since I returned from the QGIS Hackfest in Lyond, France, but I haven't had the time to write up my experiences yet....until now. Read on for more!

It was the first time that the hackfest (which is a twice yearly event) has been held in France. Lyon is a lovely city with good transport links from the airport and a modern inner city transport network. The venue for the hackfest was a university hospital (Université de Médecine et de Pharmacie Grange Blanche et Lyon Nord) and it was a short walk from our accommodation which was great. Yves Jacolin (OSGeo-fr president, camptocamp), Jean-Roc Morreale (QGis translation team, CG62), Hugo Mercier (Oslandia) and Vincent Picavet (Oslandia, OSGeo-fr treasurer) of the francophone OSGEO local chapter kindly organised the event. It is a HUGE amount of running around to organise such an event, so we really extend our heartfelt thanks to them for all the effort they put into the event.

We had four event sponsors:

• 3Liz (http://www.3liz.com)
• Camptocamp (http://www.camptocamp.com)
• Oslandia (http://www.oslandia.com)
• Services geographiques (http://servicesgeographiques.com)

As well as the sponsors, many people and organisations around the world regularly donate to the QGIS project. These funds are used to defray the costs of holding the event, for the most part in covering transportation and accommodation costs of the attendees. These contributions of our sponsors and our donors really make a HUGE difference to the project, so we are deeply indepted to you all!

The four day event followed our usual 'unmeeting' format - a format which works well since it keeps things informal and lets us intersperse discussions and talks into moments when they will have the best traction with the attendees. There was so much going on that I probably will miss some important things, but hopefully other attendees will add appropriate comments to this article as needed.

During the Zurich QGIS hackfest we had some extended discussions about migrating our documentation away from LaTeX to sphinx because the latter offers a more approachable syntax for casual documentation writers and has good support for internationalisation via gettext. This week I am going to our first 2012 QGIS hackfest (to be held in Lyon,... Read more »

During the Zurich QGIS hackfest we had some extended discussions about migrating our documentation away from LaTeX to sphinx because the latter offers a more approachable syntax for casual documentation writers and has good support for internationalisation via gettext. This week I am going to our first 2012 QGIS hackfest (to be held in Lyon, France) and we will be spending some time and effort to move forward with the migration to sphinx. In order to get familiar with sphinx, I have slowly been moving my own documentation work from txt2tags (which I love for its simplicity, but it is not great for complex documents that need interdocument references) to sphinx (which is a little more complex but has many more bells and whistles). Sphinx provides a number of themes 'out of the box'  for generated html content, but I don't really like any of them much and I want my stuff to look a little more unique. So I have dipped my toes a little into the waters of sphinx theme creation. This weekend I decided to make my efforts public. There are still a number of layout issues I want to address (mainly in the TOC sidebar), and I want to add some more explicit markup for various sphinx constructs, but what I have is already usable and hopefully interesting for others to try out. Here is a little screenshot of what the theme looks like:

You can download the theme from its github project page (https://github.com/timlinux/linfiniti-sphinx-theme)  - the accompanying README should give you enough info to get started with it. I am hoping that we can use a variant of this theme for the QGIS project documentation too. I look forward to any patches to make the theme better any of you html/css gurus out there may have to offer!

You probably looked at the GDAL Tools (under the raster menu in QGIS) and  blissfully ignored that ‘Creation Options’ panel that appears near the bottom of some dialogs. This is definately a power user feature, but a very handy one. In this quick article I will show you how to compress the rasters you create... Read more »

You probably looked at the GDAL Tools (under the raster menu in QGIS) and  blissfully ignored that 'Creation Options' panel that appears near the bottom of some dialogs. This is definately a power user feature, but a very handy one. In this quick article I will show you how to compress the rasters you create (bits are a valuable resource right?) by adding in some options there.

So this is really a short post because most of it is explained by first reading Rudi's article on GDAL raster compression. Now you can simply transpose those options into the GDAL tool you are using in QGIS like this:

In the above example, I converted an ASCII grid file to a .tif and compressed the tiff in the process. Here is a little comparison of the before and after file sizes, and the same operation done with no tif compression

1.1M Flood_Design_Depth_Jakarta_geographic.asc
116K Flood_Design_Depth_Jakarta_geographic.tif
1.6M Flood_Design_Depth_Jakarta_geographic-uncompressed.tif

Summary: Its definately worth doing this if ever you are creating raster data in QGIS (and probably one day we should make these options easily available as default profiles in the GUI of QGIS I guess) - the compressed tif is around 1/10th of the original ASCII dataset and the uncompressed tif is actually bigger than the ASCCI grid file! You can set any other valid options you like here - consult the GDAL documentation for the various supported file formats to see further things you can do.

One of the really cool features in QGIS that doesn’t get much press is the ability to run actions based on a feature selection. Under 1.7.4 this works by using the identify tool and then choosing an action from the action list in the identified feature(s) attributes. A new improvement in QGIS master (and should... Read more »

One of the really cool features in QGIS that doesn't get much press is the ability to run actions based on a feature selection. Under 1.7.4 this works by using the identify tool and then choosing an action from the action list in the identified feature(s) attributes. A new improvement in QGIS master (and should be in the upcoming 1.8) added by Giuseppe Sucameli adds an action map tool which allows you to click on any feature and run a related action. Actions can include tokens referencing feature attributes so that you can, for example, display a picture related to a feature using an external program, passing the program a file name taken from an attribute of the selected feature. Most people immediately grasp that you can just add the name of an external app e.g. firefox and pass attributes as parameters. There are some shortcomings with this approach in that it adds an unknowable dependency - will the end-user have the software required to execute the action on their system? Its easy to forget that QGIS sits on top of the incredibly powerful and versatile Qt4 library, and all that goodness is just a heartbeat away with a python line or two. Also, in QGIS actions can be arbitrary, tokenised (i.e. using variable information based on the contents of a field attribute) python commands! In this short article I will demonstrate how to use an a python action to show a web page. The url will be generated based on an attribute value and best of all, it requires no browser on the user's system since it uses the Qt4 QWebView class (which is a webkit based html widget) to display the content in a pop up window.

For this article I used the 'gadm' global borders dataset which I described a little in a previous article. My intention is to use the 'name_0' attribute to construct a wikipedia url like this:

http://wikipedia.org/wiki/South_Africa

Note: that this is a completely contrived example, I haven't checked to see if it will work in all possible values obtained from the gadm name_0 field.

Ok so to create our layer action, open the gadm layer properties dialog and head over to the Actions tab.

Now set up a new action using the following properties for the action:

Type: Python

Name: Wikipedia

Action (as listed below but all on one single line):

from PyQt4.QtCore import QUrl; from PyQt4.QtWebKit import QWebView;  myWV = QWebView(None);
myWV.load(QUrl('http://wikipedia.org/wiki/[% "name_0" %]')); myWV.show()

Ok so there are a couple of things going on here:

• All the python code is in a single line with semi-colons separating commands instead of newlines
• [% "name_0" %] will be replaced by the actual attribute value when the action is invoked
• The code simply creates a new QWebView instance, sets its url and then calls show to make it visible as a window on the user's desktop

As I mentioned, this is a contrived example - in the real world more likely you might import your logic from a python module and then call a function with a field attribute as parameter. You could equally use the approach to display an image without requiring that the user has a particular image viewer on their system.

Having a good world dataset is kind of essential for most GIS users to have as a standard reference dataset, give context to maps and so on. Anita Graser’s recent blog post about the Natural Earth Data project added another nice dataset to my world dataset collection. The Natural Earth Data set (I got the... Read more »

Having a good world dataset is kind of essential for most GIS users to have as a standard reference dataset, give context to maps and so on. Anita Graser's recent blog post about the Natural Earth Data project added another nice dataset to my world dataset collection. The Natural Earth Data set (I got the 'full' one) is provided as shapefiles. Of course as a PostGIS fan I'd rather have these in my geospatial database, so I wrote a one liner in bash to load all the datasets.

First download the full dataset from here, and then extract the contents into a working directory somewhere. Then simply run this line, adjusting your destination database and schema as needed:

for FILE in `find . -name *.shp`; do \
  BASE=`basename $FILE .shp`; \
  /usr/lib/postgresql/9.1/bin/shp2pgsql -s 4326 -I $FILE world.$BASE \
  | psql gis; done

Ok that looks like four lines above, but only because I added some line breaks so things would be readable on this blog.

Note that you should replace 'world' with the name of the schema you are importing the data into, and 'gis' with the name of the database you are importing your data into. Thanks for the awesome dataset Natural Earth Team!

I often do this – I work away on my master branch and suddenly realise ‘damn I should have branched before I started this’. Usually this happens because what I think will be a trivial change snowballs into something much more sweeping. I thought I would record here how to deal with situations like this... Read more »

I often do this - I work away on my master branch and suddenly realise 'damn I should have branched before I started this'. Usually this happens because what I think will be a trivial change snowballs into something much more sweeping. I thought I would record here how to deal with situations like this by retroactively creating a branch and rolling master back to a previous state. Note: backup your data first and this is only ideal if you haven't pushed changes to a remote repo yet.

So here was the situation. My last 'known good' state was 8399227db and I wanted to move all subsequent work out of master and into a branch. So first lets create that branch starting from the last known good state:

git branch refactor_branch 8399227db

Next I check out the branch and merge all the changes from master into it:

git checkout refactor_branch
git merge master

Now I switch back to master and chuck away all changes made since the branch point (did I mention you should backup your work first?):

git checkout master
git reset --hard 8399227db

So I rebased the master branch to an older commit. Now I push my branch up to the remote repository and breath a sigh of relief that my work is backed up in the cloud and master remains usable for others working on it:

git push origin refactor_branch

As a last couple of steps, I remove my local branch and checkout the remote branch as a tracking branch:

git branch -D riab_to_inasafe
git branch --track riab_to_inasafe origin/riab_to_inasafe
git checkout riab_to_inasafe

Git being git, there are probably a thousand other ways to accomplish the same thing, but the above process works for me!

Here is a quick handy tip if you are using GIT and want to be able to quickly diff or checkout the code base from a week, month etc. ago. You can use the git reflog to do this: @{1.hour.ago} @{1.minute.ago} @{1.hour.ago} @{1.day.ago} @{yesterday} @{1.week.ago} @{1.month.ago} @{1.year.ago} So for example to enter a detached state... Read more »

Here is a quick handy tip if you are using GIT and want to be able to quickly diff or checkout the code base from a week, month etc. ago.

You can use the git reflog to do this:

@{1.hour.ago}
@{1.minute.ago}
@{1.hour.ago}
@{1.day.ago}
@{yesterday}
@{1.week.ago}
@{1.month.ago}
@{1.year.ago}

So for example to enter a detached state showing the code as it was a week ago do:

git checkout @{1.week.ago}

When you are done, get back to master by doing:

git checkout master

Or if you just want to see a diff between now and a week ago:

git diff @{1.week.ago}

Have fun!

Some time ago I spent at least a day trying to find a good quality, free world political borders and coastlines dataset. Each product I found had some limitation – incomplete coverage or poor resolution. Today by happenstance a client pointed me to quite a good one. The site hosting the dataset is www.gadm.org. These folks... Read more »

Some time ago I spent at least a day trying to find a good quality, free world political borders and coastlines dataset. Each product I found had some limitation - incomplete coverage or poor resolution. Today by happenstance a client pointed me to quite a good one.

The site hosting the dataset is www.gadm.org. These folks have obviously taken a lot of time and effort to create a top-notch dataset. The compressed world dataset weighs in at around 333mb in its zip for and around 1.5gb uncompressed. Needless to say, the first thing you will probably want to do is load it into postgis. Secondly, you will probably get best results using a simplified version of it at small scales and only enable it to render in QGIS at larger scales. The dataset is probably best used at scales up to 1:150 000 - at greater scales there is an obvious stair stepping effect showing what seems to be the raster origins of the dataset. It would be interesting to see if someone can come up with a smoothing algorithm to remove the step effect. That aside its a great dataset if used within its limitations. I've included a few screenshots below so you can get a better feel for it.