Last week a QGIS hackfest was helf in Piza, Italy. Unfortunately I couldn't attend but luckily there is a nice write up on the QGIS home page! Hopefully I will make it along to the next one!

After the completion of the 'Gentle Introduction to GIS' resource (available here) work that we did in 2009, Nwabisa Kakaza and I wrote an article on the training resource which now appears in PositionIT (a local GIS industry publication). For your reading pleasure, the article is downloadable here.

When the OpenLayers map of your site takes some time to load and you wish to give feedback to the user, the first place to look is the Loading Panel Addin. This addin provides an animated bar, that progresses as soon as a layer is loaded on the map. But you are not bound to its standard appearance - CSS offers you an undefined customisation freedom.

I first chose a simple progress bar, with a little globe image added to it for each loaded layer, and I felt happy enough. But I'm still a CSS beginner and moreover a very cold fan of GUIs, unlike Tim, who got a nice nice animated gif to show the progress of the map population. The AjaxInfo website provided a wide range of indicators - have fun and be sure to add it to your bookmarks!

Here are the relevant code snippets of our implementation:

1- Include the LoadingPanel Addin javascript into the head of your HTML page:

<script type="text/javascript" charset="utf-8" src="LoadingPanel.js"> </script>

2- Add the functionality in mainJavaScriptFile.js:

var loadingPanel = new OpenLayers.Control.LoadingPanel();
map.addControl(loadingPanel);

//show the control
loadingPanel.maximizeControl();
// load your layers here
// remove it as the above function returns
loadingPanel.minimizeControl();

Note: that way, the LoadingPanel doesn't know how many layers you are loading. The wiki page and the related example do, with much more code. I liked that 5-line implementation - and the animated gif required only a little...

3- CSS tweak

.olControlLoadingPanel {
    background-image: url("/static//images/ajax-loader.gif");
    background-color: none;
    position: relative;
    width: 50px;
    height: 50px;
    background-position:center;
    background-repeat:no-repeat;
    display: none;
}

Last week I had to cope with one of the (few!) limitations of Django ORM model about inheritance. For the GeoDjango website we are developing, we created a model that stores user-layer pairs, so that a logged user can restore the state of map and legend as [s]he left it at previous login, and anonymous users receive a default list of layers. Here is the code for the model:

class UserWmsLayer( models.Model ):
  """ Stores custom user preferences for a layer. """
  wmslayer = models.ForeignKey( WmsLayer )
  user = models.ForeignKey( User )
  is_visible = models.NullBooleanField( null=True, blank=True )
  is_deleted =  models.NullBooleanField( null=True, blank=True, default=False )

The Layers in Legend can be of WmsLayer class or any of its subclasses, at the moment the only subclass is DateQueryLayer, a WMS layer with a filter on the date attribute. We also have a Layer class, that is an ABC (Abstract Base Class), but it's not possible to create a ForeignKey to an ABC as it has no table in the database. So we decided to create the ForeignKey to WmsLayer as a working, half-hack solution.

That worked nicely until I relied on inheritance and method overriding. I expected Django to store the ForeignKey to the actual class of the object, but the ORM stores it as a reference to the superclass, in our case of WmsLayer. This is very well explained in enlightening answer #4 of this stackoverflow QA.

In few words, I can actually have a User-DateQueryLayer pair, but in the model it will be stored as WmsLayer, and all methods I call will be WmsLayer's, not DateQueryLayer's as I would expect from OO programming. DateQueryLayer redefines asOpenLayer method, but due to this ORM limitation I could never call it.

The solutions were multiple, more or less Pythonic, Django-ish, brittle and scalable. I took two days to google and get through them, and the only two applicable solutions seemed to be:

1. Run Time Type identification via a ifelse cascade: that would be the less scalable and most brittle hack.
2. Django's Generic Relations: better, but still overcomplicated.

I was about to implement Generic Relations when I realised that asOpenLayers() simply returns a JavaScript string. A simple solution came to me in a flash. Instead of creating the JavaScript code on the fly every time the layer is used somewhere, I decided to store that string into an attribute owed by Layer, the abstract superclass, so that all subclasses will simply inherit it and populate it at instantiation time using their own asOpenLayers method. The User-*``Layer`` pair will therefore rely on an attribute in WmsLayer table without any inheritance issue.

Here is what the relevant code looks like:

class Layer( models.Model ):
  """This is an ABC (Abstract Base Class) for all models that are layers.
  It provides common api so that all layers can be treated in a similar way."""
  name = models.CharField( max_length=256 )
  owner = models.ForeignKey( User, related_name = 'owner' )
  # store the javascript as *attribute*, instead of generating it on-the-fly
  as_open_layer = models.TextField( )

class WmsLayer( Layer ):
  url = models.URLField( max_length=1024, verify_exists=True )
  layers = models.CharField( max_length=256 )
  # link to user-layer model to keep status of legend
  users = models.ManyToManyField(User, through='UserWmsLayer')

  def asOpenLayer( self ):
    """Return a string representation of this model as an open layers
    layer definition. The created layer def will be added
    to the openlayers map of name theMap (which defaults to "map". """
    return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    """ Overrides standard save, generating the asOpenLayers javascript
    and storing it into as_open_layer attribute. """
    self.as_open_layer = self.asOpenLayer()
    super(WmsLayer, self).save( *args, **kwargs)

class DateQueryLayer( WmsLayer ):
  """A layer model for storing user date range queries persistently"""
  date_query_type = models.ForeignKey( DateQueryType )
  sensor = models.ForeignKey( Sensor )
  start_date = models.DateTimeField( null=True, blank=True )
  end_date = models.DateTimeField( null=True, blank=True )

  def asOpenLayer( self ):
  """ Overrides WmsLayer's method. """
  return "the JavaScript string" #omitted for brevity

  def save( self, *args, **kwargs ):
    self.as_open_layer = self.asOpenLayer()
    super(DateQueryLayer, self).save( *args, **kwargs)

That's it.

Once you filter the UserWmsLayer table by user, you won't call anymore the asOpenLayer method...

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.asOpenLayer()

... but fetch the ``as_open_layer`` attribute instead!

myObjects = UserWmsLayer.objects.filter(user__username = "anonymous")
for myObject in myObjects:
  myObjects.as_open_layer

This allows you to pick up the correct string, without worrying about which type myObject is. This is good. Clean. Scalable. Pythonic. Yes, the asOpenLayer method simply returns a string according to some Layer properties, this solution could not work with more complex methods. But if you face a similar inheritance issue, and your method returns something that can be stored in an attribute of the superclass, that's one of the cleanest solutions I'm aware of.

When we released QGIS 1.4 in January, I decided to host the QGIS-1.4.0-1-No-GrassSetup.exe download file here on my server at Linfiniti so I could try to get a sense of how many people are using our software. Estimating users in a FOSS project is a hard thing to do - our software is free and freely redistributable so one copy of the software may make its way onto many desktops. Also some operating system's packaging technology make it hard to track download numbers - we cant easily see how many downloads have come out of the ubuntu-gis PPA for example. But for the user profile that runs windows and doesn't want GRASS functionality, I can reveal that the total downloads from my server for the months of Jan and Feb is:

22461

To which I can only say: "Wow! I didn't know that many people use Windows!"

This has been done before by others, but its always interesting to look at image quality versus size when starting a new project. Each project has different types of data and thuse different optimal configuration. Here is a simple breakdown of size vs quality I did for a project.

First let me show you my mapserver image format definitions...:

OUTPUTFORMAT
  NAME 'AGG_Q'
  DRIVER AGG/PNG
  IMAGEMODE RGB
  FORMATOPTION "QUANTIZE_FORCE=ON"
  FORMATOPTION "QUANTIZE_DITHER=OFF"
  FORMATOPTION "QUANTIZE_COLORS=256"
END

OUTPUTFORMAT
  NAME 'AGG_JPEG'
  DRIVER AGG/JPEG
  IMAGEMODE RGB
END

OUTPUTFORMAT
  NAME          "AGG_PNG"
  DRIVER        "AGG/PNG"
  IMAGEMODE     RGB
  TRANSPARENT   ON
  EXTENSION     "png"
  FORMATOPTION  "INTERLACE=OFF"
END

OUTPUTFORMAT
  NAME          "AGGA_PNG"
  DRIVER        "AGG/PNG"
  IMAGEMODE     RGBA
  TRANSPARENT   ON
  EXTENSION     "png"
  FORMATOPTION  "INTERLACE=OFF"
END

OUTPUTFORMAT
  NAME "GD_JPEG"
  DRIVER "GD/JPEG"
  MIMETYPE "image/jpeg"
  IMAGEMODE RGB
  EXTENSION "jpg"
END


OUTPUTFORMAT
  NAME "GD_PNG"
  DRIVER "GD/PNG"
  MIMETYPE "image/png"
  IMAGEMODE RGB
  EXTENSION "jpg"
END

Now lets look at how each performed in terms of rendering quality versus file size:

To be fair I haven't added the needed ANTIALIAS clauses for the GD renderer to produce anti-aliased images for true comparison with AGG. That said even GD at it's best isn't a patch on AGG in my opinion. Needless to say In this project I've gone with the Quantised AGG outputs which are almost as small as GD JPEG images but with much nicer quality.

Note: Updated 1 March to inline images rather having to click to view the full size.

When I started off this internship, I had little practical experience of Open Source software. I had heard about it, and knew that it was an interesting, expanding field for a collective good, but hadn't an opportunity to delve in deeper through my education career.

This internship has given me that opportunity and exposed me to a whole range of Open Source tools, especially those related to GIS. The programme was run so that touched on a variety of sections, explaining basic concepts and applying this information to practical projects. The sections could later be built on, beyond the internship. The training also included work from live projects running at Linfinti Consulting, which was a rewarding experience because it involved interaction with clients and handing over a product at the end of the day.

The days usually began with a session from Tim or Anne explaining what we would be covering that day. They explained all the necessary theory in a down-to-earth, and understandable way, ensuring that we understood. And after that we would get busy with our clicking, typing, learning and building. They would patiently assist us when we got stuck, and help us get to the root of the issue when there were bigger problems.

Some of the new tools and concepts I was exposed to:

• Concepts of FOSS and the FOSS GIS stack
• Linux operating system - some bash, basic tools and software on Linux
• PostGIS – (first time I've used a geodatabase through open source :) and seen its practicality!)
• Qgis – user interface, new symbology and cartography, plugins (very useful!)
• Programming - coding in Python to create our own Qgis plugins, coding standards, SVN
• Open source community support – through IRC, mailing lists and forums
• GRASS - introduction to the interface and commands, and practical application in flood simulation
• Mapserver, Openlayers - exporting QGIS projects to mapfiles, editing symbology of the mapfiles, displaying through OpenLayers
• HTML, CSS, Javascript (JQuery) - linking these up, and building a webpage
• Joomla CMS

Tim also expained how all these sections were connected and added to one another, and we got to see this through our own practical work.

I really enjoyed and learnt a lot, and would like to thank Tim, Anne, Marcelle and Robert for the valuable time I spent here. It's a sad goodbye to the team and office, but I am hoping that the new skills that I have learnt here will contribute positively to the future.

Hi my name is Robert. I am an intern at Linfiniti Consulting. I am having a great time at the company, exploring different Open Source GIS technologies. I would like to take a moment to discuss one of the projects i have been working on. I created a simple mapserver project using a mapfile that I exported from qgis. I used data that covers Mbizana municipality. MapServer is an open source development environment for building spatially-enabled internet applications. The following is a step-by-step explanation of how I did it:

localhost/cgi-bin/mapserv?map=/home/robert/mapserver/mbizana.map&mode=map

Now i can access the mapserver using Openlayers. Happy mapping!!!!!!!!!!!!

Are you looking for giving your OpenLayers map controls a cool appearance, smoothly integrated with the site's theme, without writing a papyrus and scatter code among lot of files?

Then have a look at jQuery UI CSS framework, a system of classes developed for jQuery UI widgets.

This is the map toolbar of the webGIS site we are busy developing, rendered with UI-Darkness theme:

The controls (pan, measure and zoom) are OpenLayers' controls. They are all created in the map's init() (see first js snippet below). The binding with the buttons is made by name - therefore be sure that the names of the OpenLayers controls match exactly the name properties of the respective buttons. The activation of the selected control is done by the toggleControl() function, further below in the js snippet. That way you can add as many control-button pairs as you need.

Let's see what the code looks like. It is not so much indeed.. My tribute to the proverbial programmer's laziness and to the koan of Master Foo's and the Ten Thousand Lines.

`` <link type="text/css" href="/static/css/jquery.fancybox-1.2.6.css" rel="stylesheet" media="screen" /> <script type="text/javascript" src="/static/js/jquery-1.3.2.min.js"></script> <script type="text/javascript" src="/static/js/jquery-ui-1.7.2.custom.min.js"></script> <script type="text/javascript" src="/static/js/jquery.fancybox-1.2.6.pack.js"></script> <script type="text/javascript" src="/static/js/jquery.easing.1.3.js"></script>``

`` <div id="mapcontrols" class="fg-buttonset fg-buttonset-single ui-helper-clearfix"> <button name='navigate'class="fg-button ui-state-default ui-state-active ui-priority-primary ui-corner-left" >Navigate</button> <button name='line' class="fg-button ui-state-default ui-priority-primary">Measure line</button> <button name='polygon' class="fg-button ui-state-default ui-priority-primary">Measure area</button> <a href="#" name='zoomin' class="fg-button ui-state-default fg-button-icon-solo" title="Zoom in"><span class="ui-icon ui-icon-circle-zoomin"></span> Zoom in</a> <a href="#" name='zoomout' class="fg-button ui-state-default fg-button-icon-solo ui-corner-right" title="Zoom out"><span class="ui-icon ui-icon-circle-zoomout"></span> Zoom out</a> </div>``

mapControls = {
line: new OpenLayers.Control.Measure(
    OpenLayers.Handler.Path, {
    persist: true
    }
),
polygon: new OpenLayers.Control.Measure(
    OpenLayers.Handler.Polygon, {
    persist: true
    }
),
zoomin: new OpenLayers.Control.ZoomBox(
    {title:"Zoom in box", out: false}
),
zoomout: new OpenLayers.Control.ZoomBox(
    {title:"Zoom out box", out: true}
)
};

var control;
for(var key in mapControls) {
control = mapControls[key];
control.events.on({
    "measure": handleMeasurements,
    "measurepartial": handleMeasurements
});
map.addControl(control);
}

and these functions at the bottom of your js file:

function handleMeasurements(event) {
    var geometry = event.geometry;
    var units = event.units;
    var order = event.order;
    var measure = event.measure;
    var element = document.getElementById('output'); //TODO redirect to other area?
    var out = "";
    if(order == 1) {
    out += "Measure: " + measure.toFixed(3) + " " + units;
    } else {
    out += "Measure: " + measure.toFixed(3) + " " + units + "2";
    }
    element.innerHTML = out;
}

function toggleControl(element) {
    for(key in mapControls) {
    var control = mapControls[key];
    //alert ($(element).is('.ui-state-active'));
    if(element.name == key && $(element).is('.ui-state-active')) {
        control.activate();
    } else {
        control.deactivate();
    }
    }
}

$(function(){
    //all hover and click logic for buttons
    $(".fg-button:not(.ui-state-disabled)")
    .hover(
    function(){
        $(this).addClass("ui-state-hover");
    },
    function(){
        $(this).removeClass("ui-state-hover");
    }
    )
    .mousedown(function(){
    $(this).parents('.fg-buttonset-single:first').find\
            (".fg-button.ui-state-active").removeClass("ui-state-active");
    if( $(this).is('.ui-state-active.fg-button-toggleable, \
            .fg-buttonset-multi .ui-state-active') )
        { $(this).removeClass("ui-state-active"); }
    else { $(this).addClass("ui-state-active"); }
    })
    .mouseup(function(){
    if(! $(this).is('.fg-button-toggleable, .fg-buttonset-single .fg-button,  \
            .fg-buttonset-multi .fg-button') ){
        $(this).removeClass("ui-state-active");
    }
    //TODO use this else only for measure/pan toggle.
    else {toggleControl(this);}
    });
});

Ok, should be all you have to know to set up the toolbar! Feel free to reuse the code and improve it :)

Oh, and don't forget to tweak the CSS to get the perfect look and feel ;)

--anne

When Internet connection is a limited resource, a well-designed website doesn't perform multiple times the same request. This little adjustment can significantly reduce the time required to load and refresh a page. First-world programmers should keep this in mind, or better come to South Africa and experience it in person...

This reminds me how life forms adapt to severe environmental conditions. But it's a wide topic.

Let's see how you can easily do this in Python. The snippet is a generic version of a function in views.py of the GeoDjango website we are busy developing. That function caches the result of WMS requests for layer legends in a dedicated directory, assuming that the images are not changing over time:

import urllib
import os

def retrieveDataFromUrl():
 myFileName = "file.txt"
 myLocalPath = os.path.join( os.getcwd(), myFileName )

  if not os.path.exists( myLocalPath ):
    print "Downloading data"
    myUrl = "http://whatever"
    # save it where it should have been found
    urllib.urlretrieve(myUrl, myLocalPath)
  else:
    print "Reading from local file" + myLocalPath
  # then read the file...

This code only checks if the file exists. If the file downloaded in previous run is outdated, then the newer version must be downloaded. This can be a good task for a cronjob - but it's the topic of another post ;)

Hope this helps!

-- Anne

Hi, I'm Sam. I've been learning a lot here at Linfiniti (thanks to the brilliant team!) Just like to add a quick note on one of the tasks I learnt this week.

I was working on a shapefile of the suburbs in Cape Town. A client required the suburbs to be grouped by region. After the tedious part of manually grouping the suburbs (using a created field, REGION), the unioning (dissolving) of the suburbs proved to be quick and painless through PostgreSQL, using this SQL command that implements the geomunion function:

create table ct_regions as select geomunion(the_geom), "REGION" \
 from "ct_suburbs" group by "REGION";

“ct_suburbs” is the original shapefile that was loaded into the PostGIS database using the Quantum GIS 'SPIT' plugin. “REGION” is the class (attribute) that I wish to union by.  And ct_regions will be the output shapefile. See the result here:

Before dissolving (ct_suburbs shows suburbs)

After dissolving by suburbs (ct_regions shows collections of suburbs that have been merged)

Hopefully this will be of some use when it comes to your own mapping!

Over the last few months I have been working hard to get an internship programme going here at Linfiniti Consulting. Part of our mission is to build up local skills in FOSS GIS. The internship programme is an adjunct to other initiatives like providing free training courses, open days and so on.

We have hired Anne Ghisla (Italian QGIS and GRASS geek) to come over to South Africa and spend two months mentoring our interns.

The interns were offered a two month period during which they would receive a monthly stipend to cover their living expenses, and receive on-the-job experience in the practical use of FOSS GIS to solve real world problems.

Robert Makondo studied computer science in Pretoria and worked for a year as an intern at the State IT Agency (SITA). Robert is a relatively newcomer to FOSS and FOSSGIS but is already showing great enthusiasm in learning the new tools of the trade!

Samantha Pan is from Cape Town where she studied GIS at UCT. Samantha has used GIS during her studies but is new to the world of FOSSGIS (which she is adapting to extremely well!). Unfortunately Samantha is only staying with us for one month but we hope she will use the skills she learns here in her future place of work.

Having the mixture of CS and GIS disciplines in our interns is really nice since they complement each other well and can problem solve together. I think Anne is really enjoying her stay here in South Africa and mentoring our interns. We will be sad to see her go at the end of March :-(.

If any readers out there are interested in coming to South Africa for a few months to help with mentoring interns please let me know! We will cover your costs and provide accommodation. In return you get to help bridge the digitial divide between Africa and the western world, while at the same time having a great experience in a different country!

Since I got a crackberry cellphone that geotags images when I take them using its built in GPS, I have become a photo snapping lunatic. Of course one's lunacy needs some way to manifest itself using FOSSGIS so I am going to decribe here a workflow to enable you to enjoy the thrills of geotagged imagery in QGIS even if you aren't the proud owner of a completely proprietary crackberry phone! The process I describe below will allow you to geotag a directory full of images using a gpx track that you collect using your gps when you are out in the field. If your camera / phonecam already geotags your images, you can do step 1 and then skip straight down to step 10 below!

sudo apt-get install gpscorrelate exiv2 python-pyexiv2

gpscorrelate -r *.jpg;

exiv2 IMG_0594.jpg | grep timestamp

Image timestamp : 2009:07:31 13:02:53 IMG_0594.jpg

2009-07-31T10:58:29Z

13 02 53 <-- photo
10 58 29 <-- gpx
 2  4 24 <-- difference

+ 2:00 hours gmt <-- timezone offset
-264 photo offset <-- difference (4min 24sec) in seconds, negative to show gps is behind the camera

685.07
2009-11-08T06:49:54Z
ACTIVE LOG
22:10 11-Jan-10
eTrex Venture
 0
 0

File name       : IMG_2709.jpg
File size       : 960453 Bytes
MIME type       : image/jpeg
Image size      : 2816 x 1880
Camera make     : Canon
Camera model    : Canon EOS 1000D
Image timestamp : 2009:11:08 08:55:14
Image number    :
Exposure time   : 1/250 s
Aperture        : F8
Exposure bias   : 0 EV
Flash           : No, compulsory
Flash bias      : 0 EV
Focal length    : 70.0 mm
Subject distance: 0
ISO speed       : 100
Exposure mode   : Landscape mode
Metering mode   : Multi-segment
Macro mode      : Off
Image quality   : Normal
Exif Resolution : 2816 x 1880
White balance   : Auto
Thumbnail       : image/jpeg, 7605 Bytes
Copyright       :
Exif comment    :

08 55 14 <-- photo
06 49 54 <-- gpx
 2 05 20 <-- difference
 +2hrs   <-- gmt offset
  320s   <-- second offset

gpscorrelate --timeadd +2:00 -g /tmp/knp_gpx_tim_interp.gpx --degmins -O -264 -m 1 *.jpg

gpscorrelate --timeadd +2:00 -g ../../KZN_tim.gpx --degmins -O -348 -m 30 *.jpg

exiv2 -pa IMG_0700.jpg

Exif.Image.GPSTag                            Long        1  8984
Exif.GPSInfo.GPSVersionID                    Byte        4  2.0.0.0
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2  South
Exif.GPSInfo.GPSLatitude                     Rational    3  25deg 4.26000'
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2  East
Exif.GPSInfo.GPSLongitude                    Rational    3  31deg 12.74000'
Exif.GPSInfo.GPSAltitudeRef                  Byte        1  Above sea level
Exif.GPSInfo.GPSAltitude                     Rational    1  675.5 m
Exif.GPSInfo.GPSTimeStamp                    SRational   3  13:25:59
Exif.GPSInfo.GPSMapDatum                     Ascii       7  WGS-84
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2009:07:31

gpsbabel -i gpx -f knp_gpx_tim.gpx -x interpolate,time=1 -o gpx -F knp_gpx_tim_interp.gpx

By the time a release of QGIS makes it out the door, most of us developers have long since forgotten about it and taken to the green fields of being able to add features again in QGIS trunk (i.e. 'the fun place'). There is however a kind of sigh of relief to have made another milestone in the project and to have reduced the delta between what users want in a desktop GIS and what we provide. This (1.4) release announcement resulted in a bit of a mob rush to the QGIS website which caused quite a bit of downtime for the site over the last 24 hours. Thankfully Chris Schmidt and Frank Warmerdam (and probably others) were on hand to give the server the needed poke with a pointy stick to get it to behave better.

I decided to host the QGIS standalone installer exe on linfiniti.com for this release in order to try to get a better idea of download stats. Despite the server outages (meaning people didn't have the link to download QGIS off my server) we have done around 900 downloads in the 24 hour period since the announcement. Considering that many people will share a download amongst friends and colleagues, especially here in South Africa where bandwidth is limited, and that there are numerous types of QGIS packages which aren't tracked, 900 downloads probably means many times that actually installed onto people's desktops.

QGIS 1.5 should be the final release before we start breaking API compatibility to make way for the 2.0 release. Breaking API compatibility lets use get rid of cruft from the code base and refactor the way we have designed QGIS without the overhead of having to support a large number of existing plugins and custom apps based on QGIS. I am looking forward to the rest of this year and all the new QGIS goodies the developer team will bring to the table!

I just sent out the official release announcement for QGIS 1.4 'Enceladus'....lets hope everyone loves it! The QGIS team has done a brilliant job for this one and I can't wait for 1.5 because things are only going to get better! Read the QGIS Blog for details.

I put the standalone windows installer on my own server this time so I can track downloads - it will be interesting to see just how many people grab it....I'll post some stats here later if there is anything worth reporting...

Marco Hugentobler and I have been working on integrating C++ based GPS integration into QGIS core.

A friend asked me for a presentation about QGIS that provides a quick overview of the project and the software. I quickly knocked this presentation together from various other presentations. Obviously there is a lot more detail that one could give about the project but hopefully it gives a basic introduction well enough.

Today I wanted to batch convert a directory of .tiff images to .ecw (MrSid wavelet compressed). Our server has 8 cores so it would be nice to use them all right?

Here is the quick & dirty way I do this kind of job in parallel.

#!/bin/bash
mkdir ecw
for FILE in *.tif
do
  BASENAME=$(basename $FILE .tif)
  OUTFILE=ecw/${BASENAME}.ecw
  echo "Processing: ${BASENAME}.tif"
  if [ -f $OUTFILE ] #skip if exists
  then
    echo "Skipping: $OUTFILE"
  else
    /usr/local/bin/gdal_translate -of ECW -co LARGE_OK=YES $FILE $OUTFILE
  fi
done

The script is extremely simple and is set up so that you can run it multiple times without problems because if looks to see if the output file already exists before trying to write it. If it does exist, it skips straight on to the next image.

To run 8 parallel processes I simply do this at the command prompt (I did mine in a screen session):

./toecw &
./toecw &
./toecw &
./toecw &
./toecw &
./toecw &
./toecw &
./toecw &

Afterwards you can fire up top and watch 'em go!

top - 18:21:04 up  6:41,  4 users,  load average: 10.29, 9.83, 6.69
Tasks: 216 total,   1 running, 215 sleeping,   0 stopped,   0 zombie
Cpu0  : 56.5%us, 22.5%sy,  0.0%ni, 15.7%id,  4.9%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu1  : 53.3%us, 31.6%sy,  0.0%ni,  8.9%id,  6.2%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  : 50.7%us, 37.5%sy,  0.0%ni,  4.9%id,  6.6%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu3  : 46.6%us, 38.4%sy,  0.0%ni,  4.9%id,  9.8%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu4  : 44.0%us, 29.8%sy,  0.0%ni,  8.7%id, 17.2%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu5  : 30.7%us, 57.4%sy,  0.0%ni,  1.7%id,  9.6%wa,  0.0%hi,  0.7%si,  0.0%st
Cpu6  : 58.3%us, 23.8%sy,  0.0%ni,  9.4%id,  8.5%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  : 46.1%us, 38.6%sy,  0.0%ni, 10.1%id,  4.6%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:  16227956k total, 16144508k used,    83448k free,  1739140k buffers
Swap: 62492832k total,        0k used, 62492832k free, 13383020k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
12717 timlinux  18  -2  197m  85m 5384 D  104  0.5   0:55.49 gdal_translate
12536 timlinux  18  -2  171m  77m 5384 S  102  0.5   1:08.95 gdal_translate
12705 timlinux  18  -2  195m  65m 5384 D  100  0.4   0:52.58 gdal_translate
12737 timlinux  18  -2  194m  64m 5384 D   97  0.4   0:40.78 gdal_translate
12549 timlinux  18  -2  195m 103m 5384 S   95  0.7   1:12.68 gdal_translate
12751 timlinux  18  -2  165m  66m 5384 S   88  0.4   0:37.46 gdal_translate
12561 timlinux  18  -2  166m  67m 5384 D   69  0.4   1:03.91 gdal_translate
12528 timlinux  18  -2  164m  65m 5384 S   16  0.4   0:18.24 gdal_translate

One thing to note - I ran this with the data sitting on a storage array - if your data all lives on a single drive you may have serious IO issues doing the above....

This week I conducted a 3 Day GeoDjango course. We had 11 particpants consisting of SITA (South African State Information Technology Agency) employees and interns, and other non-SITA employees. The course was given free in order to promote skills development in Open Source GIS, and the lab facilities were donated by SITA, Perseus Park, Pretoria, South Africa.

It was really great to meet the attendees and I hope the course sowed a seed of interested in them that will prompt them to go off and make beautiful open source web sites powered by GeoDjango!