Last update:
Sun May 5 02:00:35 2024
•
•
•
•
If you work with raster data in GIS you straddle a kind of wierd world where you deal with imagery but you cannot really manipulate the imagery as you would say with the GIMP or Photoshop. Conversely, if you ever tried to open a large GIS raster dataset with the GIMP you may have noticed... Read more »
If you work with raster data in GIS you straddle a kind of wierd world where you deal with imagery but you cannot really manipulate the imagery as you would say with the GIMP or Photoshop. Conversely, if you ever tried to open a large GIS raster dataset with the GIMP you may have noticed it really sucks at it. In this article I show a technique I used to apply simple image processing techniques commonly used in graphics apps like the GIMP to GIS raster data. Read on after the break...
I recently posted an article on creating coloured rasters with gdal which outlined a procedure for applying colour from polygonal areas to a raster. I used the technique to produce a pretty looking hillshade map with height classes superimposed on it like this:
While the terrain map looks great, the colours are very dominant and it makes it difficult to see the trail data (the real purpose of the map) clearly. There are a lot of great articles out there (e.g. this one) on the importance of contrast in graphic design, and I wanted to apply those principles to the cartographic outputs I was working on. In QGIS it is quite simple to desaturate an image by assigning a global transparency level to it. Here is the same map as above but with 70% transparancy assigned to it using the raster layer properties dialog in QGIS.
Things are never quite as simple as that though. The maps we are producing need to be printable using the QGIS Server (and from within QGIS desktop). For good printable output in QGIS, using raster transparency is a no-no at the moment because of some idiosyncrasies in the way that the print subsystem renders. Basically, the print system chops up the raster into possibly slightly overlapping tiles (due to rounding errors) which can result in a banding effect when two semi -transparent tiles are superimposed. So the long and the short of it is that although display purposes you can obtain a nice effect (as shown above), ideally we need to be able to create a new raster with that effect pre-applied.
Enter my discourse on why GIS are a kind of wierd platform for doing raster editing. In GIMP I would simply overlay a white layer and then use an overlay mode such as 'screen', add etc. in order to get the desired effect. So I decided to write my own tool to do this. I used Frank Warmerdam's hsv_merge.py script as inspiration and created a tool for applying lightening or darkening effects to a raster. Frank's implementation is nice in that it works with really big rasters since it processes the data in blocks - and my tool follows suite. You can download the tool from github:
https://github.com/timlinux/RasterProcessing
To use the tool, simply run it like this:
python gdal_lightener.py -a 180 input.tif output.light180.tif
Here are the general usage notes:
usage: gdal_lightener.py [-q] [-of file_format] -a amount -m mode src_color dst_color where amount is the amount of lightening to apply [0-255] default 180 mode - darken, lighten or screen src_color is a RGB or RGBA dataset dst_color will be a RGB or RGBA dataset
Screen lower amount to get a darker image (closer to original)
In general you probably won't see much difference between 'screen' and 'lighten' modes. And the 'darken' mode obviously makes the image darker - useful if you are trying to overlay light coloured content with good contrast. Afterwards you can add the new layer to your QGIS project and it will have a 'washed out' appearance without having to apply any transparency to achieve the effect.
Eventually I will probably gather more little functions like this and create a QGIS plugin so that they are accessible directly from the GUI. Have fun!
You know the drill right? You just made a beautiful map for your boss using QGIS Server. Then he walks into your office and says ‘Kiepie, I need another one in a different CRS’. Your heart drops in your chest and your jaw drops to the desk. You can’t do that using QGIS without publishing... Read more »
You know the drill right? You just made a beautiful map for your boss using QGIS Server. Then he walks into your office and says 'Kiepie, I need another one in a different CRS'. Your heart drops in your chest and your jaw drops to the desk. You can't do that using QGIS without publishing a whole new virtual host instance right? Wrong...with a little tweaking, you can publish projects in different CRS's quite easily from the same apache virtual host. Read on after the break to see how...
Please note: this article assumes you are already familiar with the basic setup and deployment of QGIS Server and the QGIS Web Client, have a running installation already and are looking to improve your map service offerings...
First, here is the outline of the procedure:
Note that for all the images listed below, you can click on them to enlarge them.
Ok so the first step is to create your project, enable on the fly projection, set the CRS to your desired coordinate reference system (I am going to use a global equal area projection for this example EPSG:3410). This should all be standard fare for a seasoned QGIS user like yourself right? I am going to use the standard helloworld project that we ship by default with the QGIS Web Client. Hopefully the screenshots below give you a few hints if you are getting waylaid:
Take special note of the 'proj' CRS definition shown when you select your CRS:
+proj=cea +lon_0=0 +lat_ts=30 +x_0=0 +y_0=0 +a=6371228 +b=6371228 +units=m +no_defs
Also click the 'Use current canvas extent' button to set your capabilities document to use the current QGIS viewport.
I used File->Save As to save the project in my QGIS Web Client 'projects' directory as 'helloworld-epsg3410.qgs'. This gives me a warm fuzzy feeling because every time I look at the filename, I know exactly which CRS it is set up for.
The QGIS Web Client uses Proj4JS to provide client side projection support and ships with a basic set of definitions under:
qgis-web-client/site/libs/proj4js/defs
To create a new definition, I made a text file called 'EPSG3419.js' containing the following text.
Proj4js.defs["EPSG:3410"] = "+title=World Equal Area +proj=cea +lon_0=0 +lat_ts=30 +x_0=0 +y_0=0 +a=6371228 +b=6371228 +units=m +no_defs"; |Create your custom Proj4JS CRS definition|
You just need to replace the appropriate parts with the details for your CRS.
The QGIS Web Client is normally set up to always load the same GlobalOptions.js. This file contains CRS specific information however - such as which EPSG code to use and what zoom levels to use. Because of this we want to make a separate one for our global CRS. Out of the box, the web client already ships with a couple of templated GlobalOptions.js files, so we can just use one of those as the starting point.
I just cloned GlobalOptions.js.templ-4326 to GlobalOptions-3410.js. There are a number of things you can (and should) edit in this file, but I am not going to cover all of the options here, only the fact that you need to set the EPSG code in accordance with your QGIS project. So:
//EPSG projection code var epsgcode = 4326;
Would become:
//EPSG projection code var epsgcode = 3410; |Configure your GlobalOptions file|
We need to clone this file because it contains a reference to GlobalOptions.js - which wish to replace in favour of our 3410 specfic version. So to do that I simply copied this file to qgis-web-client-3410.html and replaced this line:
<script type="text/javascript" src="js/GlobalOptions.js"></script>
with
<script type="text/javascript" src="js/GlobalOptions-3410.js"></script> |Create your custom web client file|
OK, we are on the home straight now. The last bit of work is simply to add a new entry in index.html to your map (linebreaks added for readability):
<a href="qgiswebclient-3410.html?map=/home/web/qgis-web-client/ projects/helloworld-epsg3410.qgs&format=image/png&visibleLayers= Hello,Country">HelloWorld - 3410<a/> |Add an entry to your index.html|
Good now we should be able to see a new entry if we navigate to our index page:
And when we click on that second link we see something like this:
This whole process could be automated with a but of javascript and cleverness, but for a quick win with the boss, this should at least allow you to deploy diverse QGIS projects within the same apache virtual host. Happy hacking folks!
The Raster Calculator in QGIS allows you to run any expression on a raster or collection of rasters. While it is definitely useful, exactly how to phrase your expression in order to reclassify a raster is not always clear. Let’s say you have a raster with values between 0 and 255, and you want everything [...]
The Raster Calculator in QGIS allows you to run any expression on a raster or collection of rasters. While it is definitely useful, exactly how to phrase your expression in order to reclassify a raster is not always clear. Let's say you have a raster with values between 0 and 255, and you want everything between 100 and 150 set to 1 (and the rest to 0). If you're like me, your first instinct would be to write an expression that looks somewhat like this:
((raster@1 > 100) AND (raster@1 < 150)) = 1 else = 0
However, this does not work. In fact, in my case it returned the opposite of what I wanted (and without the "else ...", it returned nothing at all). Instead, the proper way to do it turns out to be this:
((raster@1 < 100) OR (raster@1 > 150)) = 0
Whatever is not set to zero by this statement will be set to 1 instead.
Just a quick heads up for those of you using OpenLayers. There seem to have been a lot of problems lately with OpenLayers refusing to redraw its layers when panning, where everything was working before. One possible solution turned up in this thread on OpenLayers Users. The credit for solving the problem is therefore not [...]
Just a quick heads up for those of you using OpenLayers. There seem to have been a lot of problems lately with OpenLayers refusing to redraw its layers when panning, where everything was working before. One possible solution turned up in this thread on OpenLayers Users. The credit for solving the problem is therefore not mine, but I found it so useful that I had to share.
Short version: check that you're not using OpenLayers.Control.MouseDefaults - it has been deprecated. Instead, use OpenLayers.Control.Navigation - everything else stays the same.
Quick fix for a silly problem that kept me occupied for far too long.
One of the most popular posts on my blog ever is my article about creating GDAL hillshades: ‘A workflow for creating beautiful relief shaded DEMS using GDAL‘. In the techniques outlined in the aforementioned article, colours are assigned across a continuum to and superimposed on top of a hillshaded model. The GDAL tools in QGIS [...]
One of the most popular posts on my blog ever is my article about creating GDAL hillshades: 'A workflow for creating beautiful relief shaded DEMS using GDAL'. In the techniques outlined in the aforementioned article, colours are assigned across a continuum to and superimposed on top of a hillshaded model. The GDAL tools in QGIS have come along nicely thanks to the work of Guiseppe Sucameli - to the point that you can pretty much do the workflow from the aforementioned article within the QGIS gui environment now (barring some simple text file editing activities).
Sometimes one wants to simply assign arbitrary colours to pixel values of a raster to create a colour map. You can do this with QGIS, but the results are not persistent in the dataset, which I needed because I wanted to create my own colour map and then merge that raster with a shaded relief model. There was one extra requirement I had - I wanted to create my colour map based on polygonal areas at the beginning of my workflow. In this article I describe how I went about creating a hillshaded relief model using a polygonal overlay to define my height classes. I'm also going to show the mis-steps I took to give you a more realistic view of the process of doing such work. Read on for details...
My polygon vector file represents five height classes stored in an attribute called 'minhoyde':
0 500 1000 1500 2000
So to make a raster from this polygon layer, I used the Raster -> Conversion -> Rasterize command in QGIS to create and run this command:
gdal_rasterize -a minhoyde -ts 59620 74579 -l hoyde /tmp/hoyde.shp /tmp/hoyde.tif
This process took rather a long time - as you can see I was generating a fairly large raster and the dataset had a lot of complex polygons to process. If I was smart at that point I would have added these options to the rasterize command before running it:
-co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9 -ot Int16
The first three options would have made for a more optimally compressed file (see Rudi's previous article on this). The last option asks GDAL to output a 16bit integer file - as it turned out, the default in the absence of this is to generate a Float64 file which is massive. Since I missed that step the first go around (and consequently landed up with a 34G file), I used the gdal_translate tool (Raster -> Conversion -> Translate in QGIS) to do this as a separate step:
gdal_translate -of GTiff -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9 -ot Int16 hoyde.tif hoyde2.tif
After doing this my output file shrunk to a much more managable 18mb.
After loading up the layer in QGIS, I realised I made two important mistakes:
This is a normal part of iterative development work - I was trying to create a new workflow and making a few wrong turns is inevitable. So here was the complete revised gdal_rasterize command which I ran:
gdal_rasterize -of GTiff -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9 \ -ot Byte -a_nodata 255 -a minhoyde -ts 59620 74579 -l hoyde /tmp/hoyde.shp /tmp/hoyde.tif
The first interesting thing I noticed when generating the raster was that the output size was no different from the Int16 version I created earlier. Also, you may have been worried reading the above about how gdal would map data values ranging from 0-2000 onto a Byte data set (which should only support pixel values from 0-255). Well take a look at this gdalinfo snippet:
Band 1 Block=59620x1 Type=Byte, ColorInterp=Gray
Min=0.000 Max=2000.000
Minimum=0.000, Maximum=2000.000, Mean=59.377, StdDev=208.406
NoData Value=255
Metadata:
STATISTICS_MAXIMUM=2000
STATISTICS_MEAN=59.377337101373
STATISTICS_MINIMUM=0
STATISTICS_STDDEV=208.40576721446
you can see that gdal still shows the correct range of values event though it is a byte dataset. If you open the file we have produced in QGIS, it will look something like this:
Now if we wanted to we could assign a colour pallette to it in QGIS by setting theLayer -> Properties -> Symbology Tab -> Colour Map option to 'ColourMap' and then defining a colour for each unique value on the ColourMap tab. To illustrate, I have done this to replicate the look I am aiming for when I generate my colours layer in the next step.
When you assign a colour map in QGIS, other applications are not aware of this colour mapping, so what we are aiming to do in this tutorial is to embed the colour map into the image using GDAL, which I show in the next step.
The next step in the process was to assign colours to the rasterized polygon map. To do this, I basically treat the rasterized polygon layer as if it was a DEM and use the gdaldem colour-relief tool to assign colours to it. Before I could do this, I needed to understand the values that had been assigned for the different height classes during the rasterization process above. Doing this was a simple matter of opening my 'hoyde' dataset (the rasterized polygon layer created above) in QGIS and clicking around until I had identified the various unique values the dataset contains. In the process I generated a little table like this in text editor:
Now that I know which pixel values represent which heights, I can assign colour values (which were supplied by my client) to each height class by creating a colour lookup file that looks like this (saved as colours.txt):
The first column is the pixel value and the subsequent columns represent RGB values. Good now everything is in place and I can generate my coloured raster using Raster -> Analysis -> Dem from withing QGIS:
Or run from the command line:
gdaldem color-relief /home/web/utno/qgis-web-client/data/hoyde.tif \ /home/web/utno/qgis-web-client/data/colours.txt \ /home/web/utno/qgis-web-client/data/colours.tif \ -alpha -of GTiff -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9
After this command ran, I had a new tif image called 'colours.tif' - which we can then use as a basis for hsv_merge.py.
The last step in the process is to merge the colour model with our hillshade. This process is the same as described in my previous article - basically you need to download the hsv_merge.py script and run it from the command line) like this:
hsv_merge.py colours.tif norge_hillshade_shpclip.tif colour_relief.tif
Afterwards you should have a generated file called colour_relief.tif which is a composite of the hillshade and the colour map. Here is what it looked like for me when I was done. I put a few vector layers over the top for extra credits :-)
Although we are pretty much done with our workflow, it would be worth your while to try to optimise the output file size a little (in my case the output above was 17GB). Firstly the image we created is an RGB image - but it is a good condidate for conversion to a palletted image using the gdal 'rgb2pct.py' application. Also we could try adding some compression options to the created pct file to make it smaller still. In QGIS you can do this using Raster -> Conversion -> RGB to PCT and filling in the dialog as shown in my example below:
Or on the command line:
rgb2pct.py colour_relief.tif colour_relief_pct.tif
That reduced my file down from 17GB to 4.2GB. As a final step I used gdal_translate to compress the tif (once again, see Rudi's previous article on what these options mean).
gdal_translate -of GTiff -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9 colour_relief_pct.tif colour_relief_pct_compress.tif
This resulted in a file size of 324MB - much more acceptible. Unfortunately that wasn't quite the end of the story for this job. As I mentioned, doing this kind of work is a lot about trial and error. In this case converting the RBG image to a PCT caused the resulting images to have quite a grainy appearance.
I actually quite liked it as it looks a bit 'arty' but on the sum of it, the smooth tones of the original colour chaded DEM looked better to me so I used the translate step above directly on the original to compress it (forgoing the reduced colour pallette).
Or from the command line (I've added a few more options to the command line version below, dropping the alpha channel):
gdal_translate -of GTiff -b 1 -b 2 -b 3 -co COMPRESS=DEFLATE \ -co PREDICTOR=2 -co ZLEVEL=9 colour_relief.tif colour_relief_compress.tif
This resulted in a file size of 1024MB with a much better fidelity to the original. For performance reasons, you will probably want to forgo disk space a little and build pyramids / overviews for the final raster too. A final step you may want to do is to apply a Mask to the output layer - which is left as an exercise to the reader - I've described the process also quite well in my original article. Note that you can now do the masking procedure from within QGIS using the raster tools using gdalwarp with the cutline option.
There is no excuse for having an ugly looking map on your FOSSGIS desktop :-) GDAL and QGIS give you all the tools you need to make something beautiful and to be flexible about how you produce it. I'm not quite done with my workflow yet as I actually intend to merge other thematic layers (landuse in particular) into the colour shaded map so that I can have a single raster backdrop which depicts both landuse and height classes. But hopefully the above procedure will give you some ideas about alternative ways in which to generate shading classes for your hillshade / dem. For example you could use flood risk polygons, demographic data or other polygonal datasets you have kicking around to colour your hillshade with.
Producing a raster backdrop in this way can offer some nice rendering performance gains and help you to simplify your projects. Ultimately this work is going to be published on the web so I am trying to speed things up a little and also work around a banding issue in QGIS's print composer when you work with semi-transparent raster data.
One extra step that I performed that may be worth a mention was to use gdal sieve to remove 'salt and pepper' effect that was caused by small tranparent pixels being introduced between the polygons in the rasterization process. Example usage:
gdal_sieve.py -st 2 -of GTiff /home/web/utno/qgis-web-client/data/mask2.tif \ /home/web/utno/qgis-web-client/data/mask3.tif
This has been happening to me often lately: “Rats, this function is broken when I try to do XYZ. But why? I know it worked last week before I started messing around with it. I wasn’t testing the currently broken aspect of it in the interim, though, and I can’t remember what it looked like [...]
This has been happening to me often lately:
"Rats, this function is broken when I try to do XYZ. But why? I know it worked last week before I started messing around with it. I wasn't testing the currently broken aspect of it in the interim, though, and I can't remember what it looked like two weeks ago. When it was working."
Fear not! Do this:
git checkout some_branch@{"2 weeks ago"}
Yesterday I wrote a new little plugin for QGIS. It is available here: http://plugins.qgis.org/plugins/VectorTransparency/ or if you prefer to get it from GIT then here: https://github.com/timlinux/VectorTransparency. You can also get the plugin by adding the new QGIS official repository (which will be available by default from QGIS 1.8 onwards) to your repository list: http://plugins.qgis.org/plugins/plugins.xml The plugin is very [...]
Yesterday I wrote a new little plugin for QGIS. It is available here: http://plugins.qgis.org/plugins/VectorTransparency/ or if you prefer to get it from GIT then here: https://github.com/timlinux/VectorTransparency. You can also get the plugin by adding the new QGIS official repository (which will be available by default from QGIS 1.8 onwards) to your repository list:
http://plugins.qgis.org/plugins/plugins.xml
The plugin is very simple but saves me hours of work. When I have complex symbologies e.g. a vector layer with many categories and I want to update the transparency level of all symbols, the process is very tedious and involves clicking on each symbol and shifting the transparency slider until I get the correct value. I need to repeat this for each symbol in the layer. My new plugin does all this in one go for you. Here are a couple of screenies that illustrate its use.
Update: It seems doing this in the symbology tab of QGIS is possible - just select all the symbol classes, right click and choose set transparency from the context menu. I'll leave my plugin foating around though since it is probably quicker to use when you want to update many layers. Thanks to the folks on #QGIS irc channel for bringing this to my attention.
A nice article by Aslam Parker, Chief Directorate: National Geo-spatial Information, South Africa was published this month in our local GIS/Surveying magazine ‘Position IT‘ that describes the South African CRS. It probably would be of general interest to those trying to understand CRS concepts too. Direct link to the PDF here.
A nice article by Aslam Parker, Chief Directorate: National Geo-spatial Information, South Africa was published this month in our local GIS/Surveying magazine 'Position IT' that describes the South African CRS. It probably would be of general interest to those trying to understand CRS concepts too. Direct link to the PDF here.
In two previous articles (one one using Eclipse to build QGIS, and the other on using Eclipse as a Django development platform), I mentioned I would delve a little into using the Eclipse environment as a platform for QGIS Plugin Development. In particular we are interested in using Eclipse as a remote debugging platform. What [...]
In two previous articles (one one using Eclipse to build QGIS, and the other on using Eclipse as a Django development platform), I mentioned I would delve a little into using the Eclipse environment as a platform for QGIS Plugin Development. In particular we are interested in using Eclipse as a remote debugging platform. What does that mean? It means that Eclipse/PyDev will be able to attach itself to your plugin running withing QGIS and then step through the plugin code line by line, inspect variables and so on. This is incredibly useful as it allows you to break away from the very bad 'code, launch plugin, hope for the best' development process to one which is much more rigourous and time effecient. This article describes the process I followed to get everything set up for remote debugging with Eclipse/PyDev effectively. I also throw in a couple of other tips on how to use PyDev as an effective plugin development platform. Read more after the break below....
So if you haven't already followed the steps for setting up PyDev from my previous article, you should do that first. The procedures described here will probably work equally well on Windows and OSX, though I haven't tested them. Ok so let's dive in! First thing you need to do is have a plugin to test with. In my case, I simply used the plugin builder plugin (I know its a bit meta!) that Gary Sherman (master of the QGIS universe) wrote in order to create a simple test plugin. To install the plugin builder in QGIS go to Plugins -> Fetch Python Plugins. Ensure that you have the QGIS Contributed Plugins Repository enabled (the simplest way to do that is to use the 'Add 3rd Party Repostiories' button under the Repositories tab). Now filter the plugins list using the word 'Builder' and then install the Plugin Builder plugin.
While you are at it, I also recommend installing the 'Plugin reloader' plugin which allows you to reload a plugin when you have made some code changes without reinstalling all of QGIS.
Ok now the next step is to create a new plugin. If you have an existing plugin that you want to work with, you can skip this step. First ensure the plugin builder and reloader plugins are enabled (Plugins-> Manage Plugins), then launch the plugin builder (Plugins -> Plugin Builder -> Plugin Builder). Now fill in the blanks to create a new plugins - you can change the details as needed.
When you are done, click ok and choose ${HOME}/.qgis/python/plugins as the directory in which to save your plugin. Afterwards you should see a report something like this:
Ok for now, we can close QGIS and jump over to Eclipse. Here is a summary of the steps we will be following:
It's a few steps to follow, but you do it once and then forget about it so let's dive in...
In Eclipse do File -> New -> PyDev Project. In the dialog that appears, fill in the blanks as in the screenshot below (adapting for your system as needed).
Take special note of the fact that I have unticked 'Use default' and instead pointed the project contents folder into my plugin folder under ${HOME}/.qgis/python/plugins/RemoteDebuggerTest. Also note that the Interpreter is set to 'Default' and the 'Don't configure python path' option is set. Click next and then just accept the next screen.
Ok, now you should have the project listed in your project browser like this:
So one thing that you don't get out of the box with Eclipse PyDev is the ability to compile Qt ui's (user interface files) and Qt rcc's (resource files). There are two command line tools that usually do this for you - pyuic4 and pyrcc4. These tools need to be run on any ui and rcc files you may have whenever they are changed. Running these tools will generate python code from the xml documents contained in these files. When you use the plugin builder to generate your plugin, it also creates a makefile which will build these for you, so under Linux, the quickest way to build them is to create an Eclipse helper to launch the make file. To do this pick from the menu Run -> External Tools -> External Tools Configuration. Now right click on 'Program' and choose 'New'. Then set the following details:
Now click on the 'Run' button and the makefile will compile your UI and RCC files. If you right click on the 'TestPlugin' project folder and choose 'Refresh', you will see these resources appear in the project. Any time you change the ui or rcc files you need to rerun the make tool. More specifics on building user interfaces and resource files can be found in the Qt and Qt-Designer documentation. If you are on a windows platform, you probably won't be able to use the makefile (I'm saying this without actually testing it), so you will probably need to create individual helper tools.This article should give you some hints.
Ok so now we have all of the setup legwork out of the way, we can move on to the fun part - running and testing our plugin. It is probably a good idea to try to load your plugin in QGIS at this point and just test that the stub created by the plugin builder works as expected. To add a breakpoint we need to add PyDev to our python path. You can do this in one of two way - updating your PYTHONPATH before launching QGIS so that it can find PyDev, or appending PyDev to your python path at run time. I chose the latter approach. Note that before you can start, you need to understand where your PyDev python module is. On my system it got put into:
/home/timlinux/.eclipse/org.eclipse.platform_3.7.0_155965261/plugins/org.python.pydev.debug_2.2.4.2011110216/pysrc/
Your should be something similar - try running this command if you can't find it easily:
locate pydevd.py
Now to add the breakpoint, we will first drop these few lines into the top of our plugin (for example I put mine in remotedebuggertestdialog.py):
import sys
sys.path.append("/home/timlinux/.eclipse/org.eclipse.platform_3.7.0_155965261/plugins/org.python.pydev.debug_2.2.4.2011110216/pysrc/")
from pydevd import *
And then anywhere in that file where you want a breakpoint to be triggered, simply insert this line:
settrace()
Here is a screenshot of my plugin file ready for debugging:
In order for Eclipse/PyDev to be aware of the breakpoint being reached, you need to run a small server that will listen for a call from settrace() and attach to that process. To do that, switch Eclipse to the Debug perspective:
In the toolbar in this perspective, you will find an icon that will launch the pydev server.
After clicking that icon, the server will run in the background and you will be all set to start debugging.
Now run QGIS. It is important to note that you must start QGIS after starting the PyDev debug server - otherwise the debug traces seem to be ignored. The next thing to do is to launch your plugin in QGIS. As mentioned before, you may want to set up the reloader plugin so that you can reload your plugin in QGIS each time you make a change to it without needing to exit and restart QGIS each time. You don't need to do anything special otherwise, just click the default icon that was created for your plugin.
When the plugin launches, you should see Eclipse/PyDev kick in and halt execution after the trace point. You can now use the normal Eclipse debugging tools to step through the code, examine variables and so on. If you make some changes to the code, just continue the running plugin and close it, then reload the plugin using the plugin reloader, the breakpoint should get hit and so on. I did find at times that the breakpoint was ignored. This can be resolved by stopping and starting the PyDev server and then restarting QGIS. Here is a screenshot of my Eclipse session after the breakpoint was encountered:
In case it isn't obvious, you should remove the debugging stuff from your production code!
One of the great assets of QGIS is the python language environment which makes it very easy for even novices to develop new functionality in QGIS. One of the difficulties with the python platform for me has always been the lack of a nice way to provide a debugging environment for those who prefer a GUI especially for new programmer users and trainees. This article presents one way to address this and make it easy for new users to get started with QGIS plugin development. If you are a windows user and adapt these notes for your platform, it would be great if you could make any notes about what you did in the comments below.
Recently I needed the function ST_Split for various and sundry reasons, only to discover that I couldn’t use it in my database. The reason was that ST_Split is a new function introduced in Postgis 2.0, and the database was still in 1.5.3. Obviously, time for an upgrade. I began following the upgrade process, which I’ll [...]
Recently I needed the function ST_Split for various and sundry reasons, only to discover that I couldn't use it in my database. The reason was that ST_Split is a new function introduced in Postgis 2.0, and the database was still in 1.5.3. Obviously, time for an upgrade.
I began following the upgrade process, which I'll summarize below.
First, the installation of the new PostGIS:
$ tar xvfz postgis-2.0.0SVN.tar.gz cd postgis-2.0.0SVN ./configure --with-topology make sudo make install
Next, following the same instructions, I made a test database which was supposed to be in PostGIS 2.0:
$ createdb test createlang plpgsql test psql -f /usr/share/postgresql/9.1/contrib/postgis-2.0/postgis.sql test
But at this point, it would fail with the following error:
psql:/usr/share/postgresql/9.1/contrib/postgis-2.0/postgis.sql:73: ERROR: type "spheroid" already exists
Turns out that I still had bits and pieces of the old PostGIS 1.5 getting automatically applied via template. In such cases, PostGIS 2.0 will not be able to create a database with new functions, because the old ones are already there and can't be overwritten. Therefore, I had to get rid of the old PostGIS in my template. The simplest way to do this in my case was to restore the original template:
psql# UPDATE pg_database SET datistemplate = false WHERE datname = 'template1'; ALTER DATABASE template1 RENAME TO template1old; CREATE DATABASE template1 WITH TEMPLATE template0; UPDATE pg_database SET datistemplate = true WHERE datname = 'template1';
Then, after dropping and recreating the test database, do the following from the terminal:
$ cd /usr/share/postgresql/9.1/contrib/postgis-2.0/ psql -d test -f postgis.sql psql -d test -f spatial_ref_sys.sql psql -d test -f rtpostgis.sql psql -d test -f topology.sql
To see if it went well, do this:
psql# SELECT PostGIS_full_version();
Which should give you something like this:
POSTGIS="2.0.0SVN" GEOS="3.2.2-CAPI-1.6.2" PROJ="Rel. 4.7.1, 23 September 2009" GDAL="GDAL 1.9dev, released 2011/01/18" LIBXML="2.7.8" USE_STATS
