Page 1 of 77 (1532 posts)

  • talks about »

Tags

Last update:
Thu Jul 2 21:30:09 2015

A Django site.

QGIS Planet

Video tutorial: animated heatmaps with QGIS

Do you like the QGIS heatmap functionality? Did you know that QGIS can also create animated heatmaps?

The following video tutorial shows all necessary steps. To reproduce it, you can get the sample data from my Time Manager workshop at #QGIS2015.


First report

What do I have completed this week?

  • New implementation of the AlgorithmExecutor subclassing QObject.
  • Mechanism to deal with crashing Algorithms: when there is an exception during the algorithm execution, a signal is emitted in order to allow the main thread to make the other thread quit.
  • Use signal to connect the algorithm progress signal to the setPercentage slot in the ProgressBar

 

In the runAlgorithm method of the Processing class I changed the code associated with the runalg in order to support multithreading. Each time that an algorithm is running through the QGIS python console, a new thread and a new instance of the algorithm executor are created. The signals of the AlgorithmExecutor are connected to the thread signal in order to quit the thread and run the algorithm when the thread starts. While the algorithm is running, the main thread waits for the algorithm to finish and then proceed to show the output (first approach)

 

What am I going to achieve for the next week?

Create a non-blocking version of the processing in order to the interface not wait for the algorithm to finish.

Is there any blocking issue?

QGIS seems to crash randomly when starting the new thread and, when it doesn’t crash, the output of the algorithm is None, which may indicate that the algorithm is crashing in the new thread.

 

The complete report about the crashing can be found here.


Second report

What do I have completed this week?

  • A new implementation was designed to take into account several algorithms running. (using bounded signals)

  • Deep analysis of the processing in order to fit this new design into the current implementation of the Processing toolbox

  • Thread/Signal debugging

  • Fix the problem that makes QGIS crash when starting the new thread

 

What am I going to achieve for the next week?

Continue working on the new design of the multithreading support.

Is there any blocking issue?

There is no blocking issue for now.


Fourth report

What do I have completed this week?

  • The progress signal is successfully connected to both progress bar (when running from the console) and AlgorithmDialog (When using the processing toolbox).
  • Fixed the bug that makes the algorithm return “None”.
  • Fixed the bug that makes QGIS crash (sometimes) after the first time we run the algorithm.
  • The multithreading on the AlgorithmDialog seems to work fine.
  • More general implementation that allows to test the multithreading with any algorithm.

What am I going to achieve for the next week?

  • Solve the issue with the outputs.
  • Figure out why the processAlgorithm is not being called.
  • Test the algorithms on python console and AlgorithmDialog.
  • Bug fixes.

Is there any blocking issue?

  • Problem loading the outputs from the algorithm.
  • Have some university deadlines that may affect my work time on the project.

 


Third report

What do I have completed this week?

  • Using signals from the GeoAlgorithm to print the progress on QGIS python console
  • Code refactoring
  • Debugging
  • Multithread when running the algorithm through the AlgorithmDialog

What am I going to achieve for the next week?

  • Finish the multithread on the AlgorithmDialog
  • Connect the progress signal to the progress bar
  • Fix a bug that makes QGIS crash (sometimes) after the first time we run the algorithm
  • Fix a bug that makes the algorithm return “None”

Is there any blocking issue?

Last week I had less time to work on the project due to my university exams and projects. I still have some university projects to finish till the end of the semester that may affect GSoC in the next week.


QGIS 2.10 symbology feature preview

With the release of 2.10 right around the corner, it’s time to have a look at the new features this version of QGIS will bring. One area which has received a lot of development attention is layer styling. In particular, I want to point out the following new features:

1. Graduated symbol size

The graduated renderer has been expanded. Formerly, only color-graduated symbols could be created automatically. Now, it is possible to choose between color and size-graduated styles:

Screenshot 2015-06-21 18.39.25

2. Symbol size assistant

On a similar note, I’m sure you’ll enjoy the size assistant for data-defined size:

Screenshot 2015-06-21 23.16.10 Screenshot 2015-06-21 23.16.01

What’s particularly great about this feature is that it also creates a proper legend for the data-defined sizes:

Screenshot 2015-06-21 23.18.46

3. Interactive class exploration and definition

Another great addition to the graduated renderer dialog is the histogram tab which visualizes the distribution of values as well as the defined class borders. Additionally, the user can interactively change the classes by moving the class borders:

Screenshot 2015-06-21 18.43.09

4. Live layer effects

Since Nyall’s crowd funding initiative for live layer effects was a resounding success, it is now possible to create amazing effects for your vector styles such as shadows, glow, and blur effects:

Screenshot 2015-06-21 18.45.22

I’m very much looking forward to seeing all the new map designs this enables on the QGIS map Flickr group.

Thanks to everyone who was involved in developing and funding these new features!


QField in the wild

QField Experimental is out, after a couple of months of requirements gathering, private early alpha testing and foremost tons of emails requesting access to the testes group we decided today to put the current BETA version in the playstore. This means that from now on you can install QField just like any other android app by using the playstore.

QField app on Google Play
See more ›

Change predefined scales plus new PDOK services

New PDOK services This post is mostly interesting for dutch readers, as our national OWS service ‘PDOK’ added some new services. And we made them (5000 layers) available via the PDOK services plugin. Change predefined scales But I also want to show that you can change the predfined scales that you see in the Scale … Continue reading Change predefined scales plus new PDOK services

Crayfish 2.1: New Features

New features keep being added to Crayfish. Now it is possible to export time variable grid as animation, add AnuGA results and visualise vectors on user-defined grids.

Here are the new features in more detail…

Export to animation

The ground works were done in Crayfish 2.0 for this feature. You can now generate animation from contours and vectors and export them as AVI. There are two methods of exporting to animation: basic and using QGIS print template (qpt).

With the basic option, you can define a title, a legend and a progress clock. Alternatively, for a smarter solution, you can set up a print composer with the benefit of all its rich features. The composer template (qpt) can be used as a frame layout for exporting your animation.

Below is an example of a multi-frame print composer template used to generate animation from a Crayfish layer.

AnuGA support

Crayfish 2.1 now supports SWW file format generated by AnuGA.

SWW files generated by AnuGA in Crayfish (Click to enlarge)

Vector on user-defined grid

With this option, users can define a grid and Crayfish will interpolate values and displays results on the custom grid. Images below show vectors on the outputted mesh and user-defined grid.

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on a user-defined mesh in Crayfish (Click to enlarge)

Crayfish manual

With the ever-growing features in Crayfish, we have decided to dedicate a page on how to use Crayfish in QGIS. From the manual page, users can download a sample data and try the Crayfish features in QGIS.

Sponsors

We’d like to thank Maroondah City Council for sponsoring some of the great features in this release.

QGIS Quick WKT plugin iface edition

Some plugin core functions can now be called from a Python console:

g = QgsGeometry.fromWkt('POINT (9.9 43)')
iface.show_geometry(g)
iface.show_geometry(g.buffer(0.2, 2))
iface.show_wkt('POINT (9 45)')
iface.show_wkb(r'0103...') # cut

All functions accept a layer title as optional argument, if None is given, they are automatically added to a Quick WKT GeometryType (memory) layer, such as Quick WKT Polygon for polygons.

The man with the golden gun ( and some examples of installing and using QGIS on Windows)

I’m not Scaramanga (maybe a little bit of Nick Nack) but anyway, this article could be be a golden bullet for helping you to eliminate a lot of obstacles if you are using QGIS in some kind of class-room setup, networked environment and/or are struggling with the “new versions 3 times a year” policy of QGIS.

man-with-the-golden-gun-008

My interest for a hassle free and fast installation of QGIS stems from an experience last year: Some colleagues and I was giving a QGIS workshop for attendants from all over the world. We were told that QGIS was installed on all the computers in the “out-of-town” computer lab and indeed, the computers we checked had a working installation of QGIS. Next day, 15 minutes before the workshop should start, we discovered that the QGIS installations was either non-existent or badly mangled on a large portion of the rest of the computers. And no person with “local admin” rights within sight. It would be an understatement to say that the first hour of the workshop was an interesting time for us. Well, all’s well that ends well, and in the end the workshop was a success (Lucky us!)

A couple of months later I stumbled on this article “A QGIS class room setup on Windows ” – thanks to Richard Duivenvoorde – and tried the setup by copying the qgis program directory from the initial install to another “clean” computer. Alas, QGIS did work on some – but not on all of the computers, I tried it on. The QGIS installation had some dependencies to dll’s in the windows/system32 directory. If they already were present, the “copy” installation would work but otherwise it would fail. I discovered a somewhat complicated solution to the problem and publicized it in the article “How to make a ‘QGIS on a stick’ implementation

Then, some weeks ago at the “QGIS 2015 conference at Nødebo“, I had a chat with QGIS core developer Jürgen E. Fischer about my installation problems and viola! Two days after the chat, the OSGEO4W network installation contained the missing dll’s, Thanks Jürgen.

So now we have two bits of information:

  • By using the “–configpath” qualifier we can redirect where QGIS is looking for plugins, user selected values for options, values and files normally saved in the .qgis2 directory or in the registry.
  • The OSGeo4W network installation creates QGIS without any dependencies to external dll’s outside the installation directory. You could make an installation by simply copy an existing QGIS program directory to another computer.

This gives us some interesting possibilities:

  • Use case I : “QGIS on a stick”
    You can make a QGIS installation located on a memory stick and completely self contained; simply plug in the stick into a Windows computer and start QGIS by double clicking on the QGIS startup file.
  • Use case II: “Simple ‘xcopy’ installation of QGIS”
    Make an installation procedure for QGIS that consists of a simple copy of the QGIS program directory to a new computer. Like the answer to: “How the heck am I going to install QGIS on 24 computers in half an hour ?”
  • Use case III: “Network install of QGIS with floating profiles”
    Install QGIS on a network file share and put the QGIS setup information into the “Floating profile”. A large part of a user profile including will be cached on a central server in the windows domain. If the user logs on to another computer than the usual, his/her profile will automatically be copied to the new computer, giving the user his “normal” desktop and setups.
  • Use case IV: “QGIS on Citrix”
    Normally it ain’t my cup of tea to install any kind of program on a Citrix server where the program has a “canvas” area with fast changing images/pixels. For example: A GIS program like QGIS showing a map with an orthofoto background with millions of pixels – all with a slightly different hues of greens or browns.
    However, many IT departments swears by Citrix, so….
  • Use Case V: “Smart installation and update of a large amount of QGIS installations”
    If you have to install QGIS on dozens or hundreds of computers and keep the installation up-to-date, this is the use-case for you.

Step 1: Creating the common template..

First you have to create a common template for your future installation(s) whatever use case you implement afterwards. I use the 32-bit edition of the network installation as a basis for this although it’s perfectly possible to use the 64 bit version.

  1. Browse to http://qgis.org/en/site/forusers/download.html and click on OSGeo4W Network installer (32 bit) . This will download the installer.
  2. Start the installer and choose “Express Desktop Install”.
  3. Check “QGIS” and uncheck everything else.
  4. Check “I agree yadda yadda…” to every license message.This will install a default QGIS installation in directory C:\OSGeo4W and create a QGIS user directory in C:\Users\<my username>\.qgis2, i.e. the directory .qgis2 in your home directory. These 2 directories is the basis for the template. You can check the installation by starting QGIS from the “OSGeo4W” program group in “Start” –> “All Programs”. It should work without problems.
  5. Copy the C:\Users\<my username>\.qgis2 directory with contents to C:\OSGeo4W, so you’ll have a C:\OSGeo4W\.qgis2 directory.
  6. Copy the startup file for QGIS: C:\OSGeo4W\bin\QGIS.bat to C:\OSGeo4W\bin\QGIS.ref. This is insurance ! If you utterly mangle QGIS.bat in the next steps you have this file as a backup. Otherwise you are not going to use it for anything.
  7. Edit the file: C:\OSGeo4W\bin\QGIS.bat with a simple text editor like notepad.
    Insert 2 lines between line 2 and 3 in the bat file:

    call “%~dp0\o4w_env.bat”
    call “%OSGEO4W_ROOT%”\apps\grass\grass-6.4.4\etc\env.bat

    so it looks like this:

    call “%~dp0\o4w_env.bat”
    set QGIS_OAS=”%OSGEO4W_ROOT%”\.qgis2
    if exist “%OSGEO4W_ROOT%”\bin\QGIS_OAS.bat call “%OSGEO4W_ROOT%”\bin\QGIS_OAS.bat
    call “%OSGEO4W_ROOT%”\apps\grass\grass-6.4.4\etc\env.bat

    and replace the last line in the bat file:

    start “QGIS” /B “%OSGEO4W_ROOT%”\bin\qgis-bin.exe %*

    with this line:

    start “QGIS” /B “%OSGEO4W_ROOT%”\bin\qgis-bin.exe –configpath %QGIS_OAS% %*

    and save the result.

    What the edits mean:

    • Create an environment variable %QGIS_OAS% which works as a pointer to the location of the .qgis2 directory. Initially it points to the .qgis2 directory inside the OSGeo4W installation directory we created in (5).
    • Execute a command-file “QGIS_OAS.bat” if it exists. This command-file is the basis for all our different use-cases. There will be a different version for each use-case. Initially we don’t use this file.
    • Add the “–configpath %QGIS_OAS% qualifier to the actual start of QGIS. This means, that QGIS will look for all setup information, user-defined options, plugins and whatnot in the directory specified by the %QGIS_OAS% environment variable. And QGIS will not use any settings in the registry.
  8. Start QGIS by double clicking the revised C:\OSGeo4W\bin\QGIS.bat file and make all the changes to the initial setup of QGIS that should be included in your future QGIS installation(s). This could be change of language, default projection, extra plugins, snap options, yadda yadda yadda. Come on, make you dream QGIS setup !!

Thanks to the –configpath qualifier now on the start line for QGIS all these changes will be saved in the C:\OSGeo4W\.qgis2 directory. After this, you have a complete setup of QGIS – with your own changes – all located in the C:\OSGeo4W directory. This is the common template for all use cases.

Step 2: Do one of the use cases…

Use case I : “QGIS on a stick “

You have already done this use case. The template made in step 1 is a ready made “QGIS on a stick” installation:

  1. Copy the entire C:\OSGeo4W template directory from your computer to the root of your memory stick.
  2. Insert the memory stick into any Windows computer and start QGIS by double clicking on \OSGeo4W\bin\qgis.bat on the memory stick.
    QGIS will use \OSGeo4W\.qgis2 on the memory stick as the QGIS user directory.

Use case II : “Simple ‘xcopy’ installation of QGIS “

In this use case we will have to extend the template by creating the QGIS_OAS.bat file. This example will put the QGIS user directory “.qgis2” in the users home directory, just like the normal installation does – with one exception: This installation will not use the registry but save all user options in a ini-file placed in the QGIS user directory.

  1. Create the file C:\OSGeo4W\bin\QGIS_OAS.bat with the following content:
    set QGIS_OAS=”%USERPROFILE%”\.qgis2
    if exist %QGIS_OAS% exit /B
    xcopy “%OSGEO4W_ROOT%”\.qgis2 %QGIS_OAS% /e /i
    nircmd shortcut “””%OSGEO4W_ROOT%””\bin\qgis.bat” “~$folder.desktop$” “Start QGIS” “” “””%OSGEO4W_ROOT%””\bin\qgis-bin.exe” “0” “min” “””%OSGEO4W_ROOT%””\bin” “”

    (First the pointer to the QGIS user directory is changed to the “users home directory”\.qgis2. Secondly, The command file will check if the QGIS user directory exists in the users home directory. If it doesn’t exist, the command file will create a copy of the QGIS user directory template into the users home directory; create a shortcut to start QGIS and place this shortcut on the user’s desktop.)

  2. You can now install QGIS on a new PC by copying the entire C:\OSGeo4W directory to the new pc (using a memory-bird, network drive, zip-copy-unzip, whatever). The directory can be copied to any directory location on the new PC. Please note that some locations will require “local admin” rights for creating the new directory, for example if you copy it to “C:\Program Files”. I usually try the avoid this.

First time use on the new pc:

  • Start QGIS by double clicking on C:\OSGeo4W\bin\qgis.bat (or wherever you placed the OSGeo4W directory). When QGIS starts, the combined command files QGIS.BAT and QGIS_OAS.BAT will automatically create the QGIS user directory on the new pc; create a shortcut to QGIS on the user desktop and start QGIS.

After that you can start QGIS by double clicking on the shortcut placed on the desktop.

 

Use case III: “Network install of QGIS with floating profiles

This usage scenario will have the QGIS installation located in a directory on a common network file-share. A normal user need only to have read and execute rights to this directory. The QGIS user directory will be placed in the users “roaming profile” on the client PC, a special region which is cached to a central server.

The computers and users are members of a AD domain. If the user logs on to another pc in the domain, the “roaming profile”, including the desktop and the QGIS user directory, will automatically be copied the new computer and the user will see the usual desktop and applications.

With this setup, there is no dependencies to any particular client pc after the first start of QGIS by the user. If the user logs on any other client PC in the domain it will be possible for her to start QGIS simply by double clicking on the QGIS shortcut on desktop.

There is some requirements: The user has to have access to the common file share (!) and the network connection has to be stable and relatively fast.

In this use case we will do the following:

  1. Copy the QGIS directory C:\OSGeo4W to the common share: X:\QGIS\Current (The drive letter X: is arbitrary; it could be any drive letter). I will explain the naming scheme of the directory later.
  2. Create the file X:\QGIS\Current\bin\QGIS_OAS.bat with the following content
    set QGIS_OAS=”%APPDATA%”\.qgis2
    if exist %QGIS_OAS% exit /B
    xcopy “%OSGEO4W_ROOT%”\.qgis2 %QGIS_OAS% /e /i
    nircmd shortcut “””%OSGEO4W_ROOT%””\bin\qgis.bat” “~$folder.desktop$” “Start QGIS” “” “””%OSGEO4W_ROOT%””\bin\qgis-bin.exe” “0” “min” “””%OSGEO4W_ROOT%””\bin” “”

    (First, The QGIS_OAS environment variable points to the qgis user directory in the roaming profile. Secondly, the command file will check if the QGIS user directory exists. If it doesn’t, the command file will copy the QGIS user directory template to the users roaming profile; create a shortcut to start QGIS and place this shortcut on the user desktop.)

First time use on the new pc:

  • Start QGIS by double clicking on X:\QGIS\Current\bin\qgis.bat. When QGIS starts, the combined command files QGIS.BAT and QGIS_OAS.bat will automatically create the QGIS user directory on the new pc, create a shortcut on the user desktop and start QGIS.
  • After the first time you start QGIS by double clicking on the shortcut placed on the desktop.

Oh .. and why did I use this particular naming scheme for the directory ? It is used to make it easy to update QGIS:

When you have a new version of QGIS, do the following:

  1. Make a template of the new qgis version as described in step 1 and use case III, but in another directory: X:\QGIS\Testing and leave the X:\QGIS\Current unchanged.
  2. Test the new installation. You might even let some of your users test it!! You can start the “testing” version of QGIS by double clicking on X:\QGIS\Testing\bin\qgis.bat
  3. When you are satisfied with the new installation:
    • Rename the directory X:\QGIS\Current to X:\QGIS\Deprecated
    • Rename the directory X:\QGIS\Testing to X:\QGIS\Current 

After the last rename operation all the networked users will execute the new version using their existing shortcut. This method will probably work with all minor updates (2.8 -> 2.10), but probably not with a major update (2.10 -> 3.0)

Use case IV : “QGIS on Citrix “

If you have a Citrix environment you probably have a personal network-based share for each an every user on the domain. For this use case I call this directory M:\Personal. But the naming convention can be anything.

The idea is to place the QGIS user directory on the personal network share, so there won’t be anything saved or registered on the Citrix server that’s user related:

Do the following:

  1. Create a working installation of QGIS on the Citrix sever using the techniques described in Step 1.
  2. Create the file C:\OSGeo4W\bin\QGIS_OAS.bat on the Citrix server with the following content:
    set QGIS_OAS=M:\Personal\.qgis2
    if exist %QGIS_OAS% exit /B
    xcopy “%OSGEO4W_ROOT%”\.qgis2 %QGIS_OAS% /e /i

    (First, The QGIS_OAS environment variable points to the qgis user directory placed on the users personal network share. In this case “M:\Personal”. Secondly, the command file will check if the QGIS user directory exists. If it doesn’t, the command file will copy the QGIS user directory template to the users personal network share.)

  3. Create the usual application startup definition for Citrix pointing to C:\OSGeo4W\bin\QGIS.bat on the Citrix server.

Bingo ! That’s it. I’ll leave the last use case “Smart installation and update of a large amount of QGIS installations” to a separate article, because it is more complicated than any of the previous use cases. I’ll try to publish that article in a week or so, so stay tuned !!

This article has a zip file “QgisOnAStick.zip” attached. It contains the revised QGIS.BAT file plus all the different types of QGIS_OAS bat files described in the article. The suffix is renamed to “.usecase_1″, “.usecase_2″ and so on. Download the zip file and rename the suffix to “.bat” on the files you’re going to use.

A final note: I’ve used QGIS 2.8.2 in my tests. So the use cases works for this version of QGIS and will probably work for other QGIS 2.x versions.

Regardsbo

Bo Victor Thomsen

bvt@aestas.dk
AestasGIS
Denmark

How to Use Function Editor in QGIS Field Calculator

In QGIS 2.8, there is a new option for users to add their own python function in the Field calculator. This is an extremely useful feature enabling users to populate data within the attribute table using customised python function.

Nathan wrote a blog post about the feature and how to write a python with arguments. But in QGIS 2.8, the function editor does not correctly support functions without arguments.

In the example below, we are going to calculate proportion of area for each SAC (Special Areas of Conservation) in Great Britain to the total area of the layer.

Add GB_SAC_20130918_Dissolved.shp layer to QGIS, right-click on the layer and open attribute table. Make the layer editable and click on the Field calculator. We are now going to create a new column (propArea) and populate proportionate area of each SAC to the total area of the layer.

Under Function Editor tab, click on New file and type area for the name and save the file. For the content, copy and paste the following lines:

"""
A custom function to calculate total area of the GIS layer.
This function has no arguments.
"""

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

@qgsfunction(args='auto', group='Custom')
def total_area(x, feature, parent):
    return sum( f.geometry().area() for f in iface.activeLayer().getFeatures() )

Click on Run Script to add total_area function to your Custom function list.

Now, click on Expression tab and type:

$area / total_area(0)

As you can see, we have passed 0 as an argument. If you click OK, your QGIS will freeze! As there are many features in the layer, the expression, calculates total area for each row.

Lets make the script a bit more elegant. Firstly, we need to add caching, so that area will be calculated only once and cached for the rest of operation. Secondly, we can make the script a bit more generic, so that we can use it to get the area of other loaded layers in QGIS:

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

cache = {}
def _layer_area(layer):
    """ Internal method """
    if layer.name() not in cache:
        area = sum( f.geometry().area() for f in layer.getFeatures() )
        cache[layer.name()] = area
    return cache[layer.name()]

@qgsfunction(args='auto', group='Custom')
def total_area(layer_name, feature, parent):
    for layer in iface.mapCanvas().layers():
        if layer.name() == layer_name:
            return _layer_area(layer)
    return 0

Now, click on Expression tab and type:

$area / total_area('GB_SAC_20130918_Dissolved')

This time, it should be quicker!

QGIS Quality and Testing

I promised that I will write a bit about what I’ve been up to at the last QGIS developer meeting – apart from the social part we also got some work done there. So let me start with something that

QGIS 3.0 future plans

If you follow the QGIS developer mailing list, you’ve probably seen threads about the next major release: 3.0. The topic has been one of the many points we talked about at the latest QGIS developer meeting and Tim Sutton sums up the discussed plan in a post published today:

One hot topic was ‘when will QGIS 3.0 be released’. The short answer to that question is that ‘we don’t know’ – Jürgen Fischer and Matthias Kuhn are still investigating our options and once they have had enough time to understand the implications of upgrading to Qt5, Python 3 etc. they will make some recommendations. I can tell you that we agreed to announce clearly and long in advance (e.g. 1 year) the roadmap to moving to QGIS 3.0 so that plugin builders and others who are using QGIS libraries for building third party apps will have enough time to be ready for the transition. At the moment it is still uncertain if there even is a pressing need to make the transition, so we are going to hang back and wait for Jürgen & Matthias’ feedback.

The take-away message here is that the QGIS team is aware of the current developments around Python and Qt and will keep the community updated about the further development path well before any move.

qgis_keep_calm


Report back on the first QGIS User Conference in Nødebo, Denmark

I finally have some time to sit down and write up some thoughts on the QGIS User Conference and Developer Meeting (aka Hackfest) that we just held in Nødebo, Denmark. First up I need to thank Lene Fischer, who was the organiser and wowed us all with her relaxed and competent organisational approach to the conference. Thanks also to the University of Copenhagen School of Forestry – they sponsored the event by providing the venue and accommodation – and the venue was absolutely awesome with little cottages in the forest and all sorts of interesting diversions scattered around the forest. Lene gave me a list of names of people who helped to organise the event – I am sorry I have only got your first names but a very big thank you to you all!

 

Students: Runner, Shuttlebus, Kitchenaid, Cleaner, Info, Coordinator, Parking, Inn-keeper, Keyholder
Johanne
Thomas
Mikkel M
Steffen
Christian
Mikkel N
Ida
Louise
Anita
Thyge
Rune
Nanna
Peter
Jens
Heidi
Simon
Employees at University of CopenhagenCoordination, Accomodation, Bed&Linnen, Computer, Kitchen, Network, Tent/chairs/, Cookiebaker, Supporter, Cheerleader, Lifgt, Microphones/projector, Webpage, DTP,
Anne
Irene
Aleksander
Klaus
Vivian
Nicolas
Brian
Mike
Bo
Lene
Bent
Henning
Poul
Peter
MereteSusanne

 

 

 

On the first day of the user conference, I got to present a session on ‘the future of QGIS’ (video feed here and continued here) which held more as a town hall style meeting with a few themes (desktop, server, mobile etc.) I think the participants enjoyed the format and it was equally novel for the general user community (who got to have their questions answered directly by developers) and the developers (who got to see what real users look like).

The QGIS User Conference had many interesting talks (you can see the complete programme here – along with links to the video stream for each talk). For me the most interesting things happening at the meetup (both user conference and hackfest parts were:

  • the fact that we had our first ever general users conference (with around 150 attendees)
  • the geometry checking tools developed by Sandro Mani from Sourcepole
  • the huge amount of effort and thought being put into the processing framework – if you haven’t already tried out the QGIS processing tools, do go and try them!
  • The server side plugin framework that Alessandro Pasotti is working on – see his blog post here too http://www.itopen.it/qgis-developer-meeting-in-nodebo/
  • The amount of polish being applied to QGIS – there are probably less ‘gee whizz’ new features and a lot more fixes and improvements – just take a look at the incoming pull requests to get a flavour of the kind of activity going on.
  • The new geometry system by Marco Hugentobler (also from Sourcepole) which will support curves and 3D geometries (z / m). The graphical user interface for working with the new geometries won’t come until a later release, but 2.10 will get the underlying support added (along with shims to provide backwards compatibility to the old geometry classes).
  • There were some interesting talks on using QGIS in a server side / headless / command line context – again check out the talks and video streams in the programme to watch talks by Martin Dobias, Dražen Odobašić.
  • QGIS on mobile is coming – Matthias Kuhn showed off the current state of QField – the Android native interface for field work based on QGIS he has been working on. See his blog post here too for his take on the week. While the Android work shows lots of promise, there are still lots of problems to be resolved – for example missing support for ‘Lollipop’ devices. Please consider sponsoring Matthias’ efforts if you can.
  • There is a heap of interesting stuff coming down the pipeline from Nyall Dawson for the production of print maps and rendering effects for map renderers. Nyall also showed off some other very interesting ideas for context based variables that can be used in expressions – it’s hard to explain it in  a sentance or two – suffice to know that power users are going to have even more awesome tools at their fingertips for producing great maps.

One hot topic was ‘when will QGIS 3.0 be released’. The short answer to that question is that ‘we don’t know’ – Jürgen Fischer and Matthias Kuhn are still investigating our options and once they have had enough time to understand the implications of upgrading to Qt5, Python 3 etc. they will make some recommendations. I can tell you that we agreed to announce clearly and long in advance (e.g. 1 year) the roadmap to moving to QGIS 3.0 so that plugin builders and others who are using QGIS libraries for building third party apps will have enough time to be ready for the transition. At the moment it is still uncertain if there even is a pressing need to make the transition, so we are going to hang back and wait for Jürgen & Matthias’ feedback.

I apologise for not reporting on many of other interesting talks and birds of a feather meetings here – there was so much going on including work on documentation, translations, bug fixing, bug triaging that it is quite difficult to list it all here.

Two initiatives I was involved in at the meetup: the user certification programme and the formation of a QGIS legal entity. I am not going to post details here because things are not finalised yet (watch the mailing lists for details on the legal entity), but if you are interested in the certification programme, please get into contact – we have started drafting a roadmap for the roll out of our official curriculum. The QGIS project also got a huge boost from the QGIS Academy folks who will be contributing all their training resources right into the core of the QGIS documentation project (see Kurte Menke’s presentation notes on the programme).

We (Paolo Cavallini, Alessandre Pasotti, Nyall Dawson and myself) had a little roundtable discussion on the last day of the hackfest where we ran through some of the highlights from the week. You can listen to it here – or subscribe to the podcast at http://podcast.qgis.org (I will try to get back into the swing of making more regular episodes).

Well that wraps up my feedback for the event – I really encourage everyone to come along and join us on the next QGIS User Conference – it was fun, informal and informative!

QGIS developer meeting in Nødebo

During the hackfest I’ve been working on the refactoring of the server component, aimed to wrap the server into a class and create python bindings for the new classes. This work is now in the PR queue and brings a first working python test for the server itself.

The server can now be invoked directly from python, like in the example below:

 

#!/usr/bin/env python
"""
Super simple QgsServer.
"""

from qgis.server import *
from BaseHTTPServer import *

class handler (BaseHTTPRequestHandler):

    server = QgsServer()

    def _doHeaders(self, response):
        l = response.pop(0)
        while l:
            h = l.split(':')
            self.send_header(h[0], ':'.join(h[1:]))
            self.log_message( "send_header %s - %s" % (h[0], ':'.join(h[1:])))
            l = response.pop(0)
        self.end_headers()

    def do_HEAD(self):
        self.send_response(200)
        response = str(handler.server.handleRequestGetHeaders(self.path[2:])).split('\n')
        self._doHeaders(response)

    def do_GET(self):
        response = str(handler.server.handleRequest(self.path[2:])).split('\n')
        i = 0
        self.send_response(200)
        self._doHeaders(response)
        self.wfile.write(('\n'.join(response[i:])).strip())

    def do_OPTIONS(s):
        handler.do_GET(s)

httpd = HTTPServer( ('', 8000), handler)

while True:
    httpd.handle_request()

The python bindings capture the server output instead of printing it on FCGI stdout and allow to pass the request parameters QUERY_STRING directly to the request handler as a string, this makes writing python tests very easy.

QGIS and IPython: the definitive interactive console

Whatever is your level of Python knowledge, when you’ll discover the advantages and super-powers of IPython you will never run the default python console again, really: never!

If you’ve never heard about IPython, discover it on IPython official website, don’t get confused by its notebook, graphics and parallel computing capabilities, it also worth if only used as a substitute for the standard Python shell.

I discovered IPython more than 5 years ago and it literally changed my life: I use it also for debugging instead ofpdb, you can embed an IPython console in your code with:

from IPython import embed; embed()

TAB completion with full introspection

What I like the most in IPython is its TAB completion features, it’s not just like normal text matching while you type but it has full realtime introspection, you only see what you have access to, being it a method of an instance or a class or a property, a module, a submodule or whatever you might think of: it even works when you’re importing something or you are typing a path like in open('/home/.....

Its TAB completion is so powerful that you can even use shell commands from within the IPython interpreter!

Full documentation is just a question mark away

Just type “?” after a method of function to print its docstring or its signature in case of SIP bindings.

Lot of special functions

IPython special functions are available for history, paste, run, include and many more topics, they are prefixed with “%” and self-documented in the shell.

All that sounds great! But what has to do with QGIS?

I personally find the QGIS python console lacks some important features, expecially with the autocompletion (autosuggest). What’s the purpose of having autocompletion when most of the times you just get a traceback because the method the autocompleter proposed you is that of another class? My brain is too small and too old to keep the whole API docs in my mind, autocompletion is useful when it’s intelligent enough to tell between methods and properties of the instance/class on which you’re operating.

Another problem is that the API is very far from being “pythonic” (this isn’t anyone’s fault, it’s just how SIP works), here’s an example (suppose we want the SRID of the first layer):

core.QgsMapLayerRegistry.instance().mapLayers().value()[0].crs().authid()
# TAB completion stops working here^

TAB completion stop working at the first parenthesis :(

What if all those getter would be properties?

registry = core.QgsMapLayerRegistry.instance()
# With a couple of TABs without having to remember any method or function name!
registry.p_mapLayers.values()
[<qgis._core.QgsRasterLayer at 0x7f07dff8e2b0>,
 <qgis._core.QgsRasterLayer at 0x7f07dff8ef28>,
 <qgis._core.QgsVectorLayer at 0x7f07dff48c30>,
 <qgis._core.QgsVectorLayer at 0x7f07dff8e478>,
 <qgis._core.QgsVectorLayer at 0x7f07dff489d0>,
 <qgis._core.QgsVectorLayer at 0x7f07dff48770>]

layer = registry.p_mapLayers.values()[0]

layer.p_c ---> TAB!
layer.p_cacheImage            layer.p_children       layer.p_connect       
layer.p_capabilitiesString    layer.p_commitChanges  layer.p_crs           
layer.p_changeAttributeValue  layer.p_commitErrors   layer.p_customProperty

layer.p_crs.p_ ---> TAB!
layer.p_crs.p_authid               layer.p_crs.p_postgisSrid      
layer.p_crs.p_axisInverted         layer.p_crs.p_projectionAcronym
layer.p_crs.p_description          layer.p_crs.p_recentProjections
layer.p_crs.p_ellipsoidAcronym     layer.p_crs.p_srsid            
layer.p_crs.p_findMatchingProj     layer.p_crs.p_syncDb           
layer.p_crs.p_geographicCRSAuthId  layer.p_crs.p_toProj4          
layer.p_crs.p_geographicFlag       layer.p_crs.p_toWkt            
layer.p_crs.p_isValid              layer.p_crs.p_validationHint   
layer.p_crs.p_mapUnits    

layer.p_crs.p_authid
Out[]: u'EPSG:4326'

This works with a quick and dirty hack: propertize that adds a p_... property to all methods in a module or in a class that

  1. do return something
  2. do not take any argument (except self)

this leaves the original methods untouched (in case they were overloaded!) still allowing full introspection and TAB completion with a pythonic interface.

A few methods are still not working with propertize, so far singleton methods like instance() are not passing unit tests.

IPyConsole: a QGIS IPython plugin

If you’ve been reading up to this point you probably can’t wait to start using IPython inside your beloved QGIS (if that’s not the case, please keep reading the previous paragraphs carefully until your appetite is grown!).

An experimental plugin that brings the magic of IPython to QGIS is now available:
Download IPyConsole

 

Please start exploring QGIS objects and classes and give me some feedback!

 

IPyConsole QGIS plugin

Installation notes

You basically need only a working IPython installation, IPython is available for all major platforms and distributions, please refer to the official documentation.

 

QGIS and QT: getting ready for HiDPI screens

 

A few months ago I changed my laptop for a Dell M3800 with an amazing 3840×2160 15,6″ display, that means a crazy 282 dpi resolution. I won’t discuss the problems I had to face and resolve to make this thing usable with Kubuntu 14.04 because they are yet some useful resource on the net: https://wiki.archlinux.org/index.php/HiDPI. The executive summary is that KDE and QT/GTK applications work pretty well and without serious usability problems: Firefox, Chrome, Thunderbird have no problems at all if we exclude some really minor glitches like pixelated or too big icons in a few occasions (see the example below).

Small problem with a big icon in Firefox

Small problem with a big icon in Firefox @ 282 dpi

 

QGIS on HiDPI

Unfortunately, QGIS has some serious usability problems on such an high resolution screen and I spent a few hours try to understand how is it possible to fix them and what should be done to avoid them in the first place.

The reason behind is that HiDPI screens usage is increasing and we can expect they’ll gain more and more market share expecially among professional users and GIS users are mostly professional users!

To give you a quick taste of the kind of problems that have to be addressed, take a look at the pictures below.

QGIS statusbar: unreadable on HiDPI

QGIS statusbar: unreadable on HiDPI

qgis-hidpi-pluginmanager

Plugin manager rows are too low and left tab column has a 200 px max width

qgis-hidpi-attributetable-issue

Attribute table rows are too low and icons are too small

 

I tried to fix some of the issues shown above and this is the resulting commit: https://github.com/qgis/QGIS/pull/2014/commits, QGIS run on different platforms and I wasn’t able to test it on all of them, I hope the fixes will not cause problems on platforms and screen sizes other than I could test myself.

But the purpose of this notes it to list some humble tips that could be useful to avoid or fix these kind of problem.

Tip #1: Avoid hardcoded pixel values!

Web developers started the fight with HiDPI screens a few years ago, (buzz)words like responsive design and the move towards mobile platforms forced the developers to consider what a pixel really is in a web context, and the obvious conclusion was that a pixel is not a pixel! Please see the following resources if you’re curious and not familiar with these concepts:

The most obvious and efficient solution for the web browser was to change the concept of pixel from a physical to a logical unit.

Unfortunately, at least in QT the pixel is still a physical pixel, this means the we must avoid to specify the size of GUI elements in pixel units.

 

Qt documentation has a page about Developing Scalable UIs.

This blog page also has some useful tips: HighDPI in KDE Applications.

 

A collection of useful tips for QT developers can be found in the HiDPI KDE wiki page:

  • Do not use QFont setPixelSize but use setPointSize, and better, avoid setting a custom size at all.
  • Try to stick with the possibilities from KGlobalSettings (KGlobalSettings::generalFont(), KGlobalSettings::largeFont() etc.)
  • Do not use a fixed QSize with images (QPixmap, QIcon…)
  • Do not use KIconLoader::StdSizes. Even though it would sound like a good idea to use it, it suffers from the same issue: Hardcoded pixel sizes.
  • To get a user’s configured icon size, include and use the IconSize function (note: this is not a member function): IconSize(KIconLoader::Small). However, be aware that the user might be able to mess up your apps look by setting some insane values here.
  • If you use svg images for icons (for example in plasma) get the standard sizes from some theme elements (e.g. buttons, fonts) and scale your images accordingly. (Don’t do this when using pixel based images)

Some of the tips above are clearly related to KDE, but I’ve found them still useful to get the overall picture.

Coming to QGIS specific problems, there are still many that have to be resolved, but the overall application is somewhat already useable. The issues I have addressed were easy picks and most of the times it was enough to remove the hardcoded values and rely to QT default sizing mechanisms to get a good result, such as in this case (from src/app/qgisapp.cpp):

//mCoordsLabel->setMaximumHeight( 20 );

Things were harder with plugin manager, the MVC delegate that draws the plugins rows contained a lot of hardcoded metrics for icon and text placement, the solution was to change the values to be relative to the configured font height, see all details in my PR to solve plugin manager HiDPI issues

 

This is generally a good idea: we must not make any assumption about the size of the fonts the user will be using.

Tip #2: force resize of dialogs

This is something that you should be aware of if you are using an HiDPI screen to build your GUI.

I first encountered this problem when developing my IPyConsole QGIS python plugin: I designed the GUI with designer-qt4 on my HiDPI screen and forget about testing it on “normal” screens, the result is that the window size of the settings dialog (a relatively small dialog) was set by designer at a crazy value of 1115×710. It was impossible from QT Designer to set a small size by dragging the window corner with the mouse: the application automatically reset the dialog size to the calculated minimum size (which on an HiDPI screen was that huge  1115×710).

The solution in that case was to manually alter the Ui_SettingsDialog.ui file to set a small size (say 300×200) and call adjustSize()

self.settingsDlg.show()
# This will adjust the size according to font/dpi:
self.settingsDlg.adjustSize()
result = self.settingsDlg.exec_()

Tip #3: Icons

I haven’t yet come to a solution for the right size for icons (suggestions welcome!). Of course the first point is to move all icons to SVG, the process has already started but many icons have still to be converted.

But having scalable icons is not enough: we must make sure that the icon size is not hardcoded in the GUI, there are some settings for icon size in QGIS options dialog and that might be the right point to start but we probably need more than one size:

  • size of toolbar icons
  • size of plugin icons
  • size for smaller toolbars (attribute table, python console vertical sidebar to cite a few)

Maybe a 3-sizes approach would be fine (big, medium, small), for an easy configuration, some pre-defined values for HiDPI / 96dpi and HD screens would be useful.

To get an idea of what the attribute table looks like right now on an HiDPI screen, look at the picture below, scaled to 96dpi:

 

qgis-hidpi-attributetable-small-icons-issue96dpi

Icons are too small on an HiDPI screen

 

 

Conclusions

There is still much work to do in order to have an usable interface on HiDPI screens, some issues can be easily solved by avoiding hardcoded sizes and leaving to QT the heavy job of calculating GUI widget sizes.

It’s important that developers (not only core developers but plugin developers too) are aware of this kind of issues and how to avoid them in the first place.

HiDPI screens are just around the corner and it’s better to be prepared.

 

A final note about QT5, it’s not clear to me if and when the move to QT5 will be done and moreover if that move will automatically solve HiDPI issues due to a better HiDPI support, but I’m afraid it won’t.

 

QGIS server python plugins

 

Today it’s a great day for QGIS Server: Python plugins, the project that took me busy during the past two months, has been merged to master and will be available starting with the next QGIS release.

The project has been discussed and approved in Essen during the last QGIS HF (see my presentation about server plugins), thanks to the input and suggestions coming from Marco Hugentobler and Martin Dobias it is now implemented in the more complete and flexible way.

In this article I will introduce the core concepts and the main features of python plugins for QGIS server.

QGIS server plugins architecture

QGIS server provides some core services: WFS, WMS, WCS. What we wanted to achieve was a system to easily add new services and modify existing services through python plugins.

Mi first experiments were limited to a 404 handler that intercepts unhandled requests and hooks into python plugins capturing every stdout output, this was indeed not enough flexible for a full fledged plugins implementation.

The main loop

QGIS server is not different from most web services implementations: it listens for incoming requests, parses the URL query string parameters and returns its output accordingly to the incoming request.

The standard loop before introducing python plugins looked like the following:

  • Get the request
    • create GET/POST/SOAP request handler
    • if SERVICE is WMS/WFS/WCS
      • create WMS/WFS/WCS server passing in request handler
        • call server’s executeRequest()
          • call request handler output method
    • else Exception

Plugins come into play

Server python plugins are loaded once when the FCGI application starts and they should register one or more QgsServerFilter (from this point, you might find useful a quick look to the server plugins API docs). Each filter should implement at least one of three callbacks (aka: hooks):

    1. requestReady
    2. sendResponse
    3. responseComplete

All filters have access to the request/response object (QgsRequestHandler) and can manipulate all its properties (input/output) and can raise exceptions (while in a quite particular way as we’ll see below).

Here is a pseudo code showing how and when the filter’s callbacks are called:

  • Get the request
    • create GET/POST/SOAP request handler
    • pass request to serverIface
    • call plugins requestReady filters
    • if there is not a response
      • if SERVICE is WMS/WFS/WCS
        • create WMS/WFS/WCS server
          • call server’s executeRequest and possibily call sendResponse plugin filters when streaming output or store the byte stream output and content type in the request handler
      • call plugins responseComplete filters
    • call plugins sendResponse filters

    • request handler output the response

requestReady

This is called when the request is ready: incoming URL and data have been parsed and before entering the core services (WMS, WFS etc.) switch, this is the point where you can manipulate the input and perform actions like:

  • authentication/authorization
  • redirects
  • add/remove certain parameters (typenames for example)
  • raise exceptions

You could even substitute a core service completely by changing SERVICE parameter and hence bypassing the core service completely (not that this make much sense though).

Implementation details of server plugins will be discussed in depth in a future article, by now please refer to  QGIS HelloServer plugin for a complete implementation of the examples and methods cited in this article.

 

sendResponse

This is called whenever output is sent to FCGI stdout (and from there, to the client), this is normally done after core services have finished their process and after responseComplete hook was called, but in a few cases XML can become so huge that a streaming XML implementation was needed (WFS GetFeature is one of them), in this case, sendResponse is called multiple times before the response is complete (and before responseComplete is called). The obvious consequence is that sendResponse is normally called once but might be exceptionally called multiple times and in that case (and only in that case) it is also called before responseComplete.

SendResponse is the best place for direct manipulation of core service’s output and while responseComplete is typically also an option, sendResponse is the only viable option  in case of streaming services.

responseComplete

This is called once when core services (if hit) finish their process and the request is ready to be sent to the client. As discussed above, this is  normally called before sendResponse except for streaming services (or other plugin filters) that might have called sendResponse earlier.

responseComplete is the ideal place to provide new services implementation (WPS or custom services) and to perform direct manipulation of the output coming from core services (for example to add a watermark upon a WMS image).

Raising exception from a plugin

Some work has still to be done on this topic: the current implementation can distinguish between handled and unhandled exceptions by setting a QgsRequestHandler property to an instance of QgsMapServiceException, this way the main C++ code can catch handled python exceptions and ignore unhandled exceptions (or better: log them).

This approach basically works but it does not satisfy my pythonic way of handle exceptions: I would rather prefer to raise exceptions from python code to see them bubbling up into C++ loop for being handled there.

Conclusions

The new plugin system is very flexible and allows for basic input/output (i.e. request/response) manipulation and for new services implementation while it remains unobtrusive and has negligible impact on performances, in the next article I will discuss server plugin implementation in depth.

 

See also the second part of this article.

See all QGIS Server related posts

Python SIP C++ bindings tutorial

Since QGIS uses QT libraries, SIP is the natural choice for creating the bindings.

Here are some random notes about this journey into SIP and Python bindings, I hope you’ll find them useful!
We will create a sample C++ library, a simple C++ program to test it and finally, the SIP configuration file and the python module plus a short program to test it.

Create the example library

FIrst we need a C++ library, following  the tutorial on the official SIP website  I created a simple library named hellosip:

 

$ mkdir hellosip
$ cd hellosip
$ touch hellosip.h hellosip.cpp Makefile.lib

This is the content of the header file hellosip.h:

#include <string>

using namespace std;

class HelloSip {
    const string the_word;
public:
    // ctor
    HelloSip(const string w);
    string reverse() const;
};

This is the implementation in file hellosip.cpp , the library just reverse a string, nothing really useful.

#include "hellosip.h"
#include <string>

HelloSip::HelloSip(const string w): the_word(w)
{
}

string HelloSip::reverse() const
{
    string tmp;
    for (string::const_reverse_iterator rit=the_word.rbegin(); rit!=the_word.rend(); ++rit)
        tmp += *rit;
    return tmp;
}

 

Compiling and linking the shared library

Now, its time to compile the library, g++ must be invoked with -fPIC option in order to generate Position Independent Code, -g tells the compiler to generate debug symbols and it is not strictly necessary if you don’t need to debug the library:

g++ -c -g -fPIC hellosip.cpp -o hellosip.o

The linker needs a few options to create a dynamically linked Shared Object (.so) library, first -shared which tells gcc to create a shared library, then the -soname which is the library version name, last -export_dynamic that is also not strictly necessary but can be useful for debugging in case the library is dynamically opened (with dlopen) :

g++ -shared -Wl,-soname,libhellosip.so.1  -g -export-dynamic -o libhellosip.so.1  hellosip.o

At the end of this process, we should have a brand new libhellosip.so.1 sitting in the current directory.

For more informations on shared libraries under linux you can read TLDP chapter on this topic.

 

Using the library with C++

Before starting the binding creation with SIP, we want to test the new library with a simple C++ program stored in a new cpp file: hellosiptest.cpp:

#include "hellosip.h"
#include <string>
using namespace std;
// Prints True if the string is correctly reversed
int main(int argc, char* argv[]) {
  HelloSip hs("ciao");
  cout << ("oaic" == hs.reverse() ? "True" : "False") << endl;
  return 0;
}

To compile the program we use the simple command:

g++ hellosiptest.cpp -g -L.  -lhellosip -o hellosiptest

which fails with the following error:

/usr/bin/ld: cannot find -lhellosip
collect2: error: ld returned 1 exit status

For this tutorial, we are skipping the installation part, that would have created proper links from the base soname, we are doing it now with:

ln -s libhellosip.so.1 libhellosip.so

The compiler should now be happy and produce an hellosiptest executable, that can be tested with:

$ ./hellosiptest
True

If we launch the program we might see a new error:

./hellosiptest: error while loading shared libraries: libhellosip.so.1: cannot open shared object file: No such file or directory

This is due to the fact that we have not installed our test library system-wide and the operating system is not able to locate and dynamically load the library, we can fix it in the current shell by adding the current path to the LD_LIBRARY_PATH environment variable which tells the operating system which directories have to be searched for shared libraries. The following commands will do just that:

export LD_LIBRARY_PATH=`pwd`

Note that this environment variable setting is “temporary” and will be lost when you exit the current shell.

 

 

SIP bindings

Now that we know that the library works we can start with the bindings, SIP needs an interface header file with the instructions to create the bindings, its syntax resembles that of a standard C header file with the addition of a few directives, it contains (among other bits) the name of the module and the classes and methods to export.

The SIP header file hellosip.sip contains two blocks of instructions: the class definition that ends around line 15 and an additional %MappedType block that specifies how the std::string type can be translated from/to Python objects, this block is not normally necessary until you stick standard C types. You will notice that the class definition part is quite similar to the C++ header file hellosip.h:

// Define the SIP wrapper to the hellosip library.

%Module hellosip

class HelloSip {

%TypeHeaderCode
#include <hellosip.h>
%End

public:
    HelloSip(const std::string w);
    std::string reverse() const;
};

// Creates the mapping for std::string
// From: http://www.riverbankcomputing.com/pipermail/pyqt/2009-July/023533.html

%MappedType std::string
{
%TypeHeaderCode
#include 
%End

%ConvertFromTypeCode
    // convert an std::string to a Python (unicode) string
    PyObject* newstring;
    newstring = PyUnicode_DecodeUTF8(sipCpp->c_str(), sipCpp->length(), NULL);
    if(newstring == NULL) {
        PyErr_Clear();
        newstring = PyString_FromString(sipCpp->c_str());
    }
    return newstring;
%End

%ConvertToTypeCode
    // Allow a Python string (or a unicode string) whenever a string is
    // expected.
    // If argument is a Unicode string, just decode it to UTF-8
    // If argument is a Python string, assume it's UTF-8
    if (sipIsErr == NULL)
        return (PyString_Check(sipPy) || PyUnicode_Check(sipPy));
    if (sipPy == Py_None) {
        *sipCppPtr = new std::string;
        return 1;
    }
    if (PyUnicode_Check(sipPy)) {
        PyObject* s = PyUnicode_AsEncodedString(sipPy, "UTF-8", "");
        *sipCppPtr = new std::string(PyString_AS_STRING(s));
        Py_DECREF(s);
        return 1;
    }
    if (PyString_Check(sipPy)) {
        *sipCppPtr = new std::string(PyString_AS_STRING(sipPy));
        return 1;
    }
    return 0;
%End
};

At this point we could have run the sip command by hand but the documentation suggests to use the python module sipconfig that, given a few of configuration variables, automatically creates the Makefile for us, the file is by convention named configure.py:

import os
import sipconfig

basename = "hellosip"

# The name of the SIP build file generated by SIP and used by the build
# system.
build_file = basename + ".sbf"

# Get the SIP configuration information.
config = sipconfig.Configuration()

# Run SIP to generate the code.
os.system(" ".join([config.sip_bin, "-c", ".", "-b", build_file, basename + ".sip"]))

# Create the Makefile.
makefile = sipconfig.SIPModuleMakefile(config, build_file)

# Add the library we are wrapping.  The name doesn't include any platform
# specific prefixes or extensions (e.g. the "lib" prefix on UNIX, or the
# ".dll" extension on Windows).
makefile.extra_libs = [basename]

# Search libraries in current directory
makefile.extra_lflags= ['-L.']

# Generate the Makefile itself.
makefile.generate()

We now have a Makefile ready to build the bindings, just run make to build the library. If everything goes right you will find a new hellosip.so library which is the python module. To test it, we can use the following simple program (always make sure that LD_LIBRARY_PATH contains the directory where libhellosip.so is found).

import hellosip
print hellosip.HelloSip('ciao').reverse() == 'oaic'

Download

The full source code of this tutorial can be downloaded from this link.

  • Page 1 of 77 ( 1532 posts )
  • >>

Back to Top

Sponsors