Page 1 of 98 (1948 posts)

  • talks about »


Last update:
Mon Jul 23 04:25:14 2018

A Django site.

QGIS Planet

Movement data in GIS #15: writing a PL/pgSQL stop detection function for PostGIS trajectories

Do you sometimes start writing an SQL query and around at line 50 you get the feeling that it might be getting out of hand? If so, it might be useful to start breaking it down into smaller chunks and wrap those up into custom functions. Never done that? Don’t despair! There’s an excellent PL/pgSQL tutorial on to get you started.

To get an idea of the basic structure of a PL/pgSQL function and to proof that PostGIS datatypes work just fine in this context, here’s a basic function that takes a trajectory geometry and outputs its duration, i.e. the difference between its last and first timestamp:

CREATE OR REPLACE FUNCTION AG_Duration(traj geometry) 
RETURNS numeric LANGUAGE 'plpgsql'
RETURN ST_M(ST_EndPoint(traj))-ST_M(ST_StartPoint(traj));

My end goal for this exercise was to implement a function that takes a trajectory and outputs the stops along this trajectory. Commonly, a stop is defined as a long stay within an area with a small radius. This leads us to the following definition:

   traj geometry, 
   max_size numeric, 
   min_duration numeric)
RETURNS TABLE(sequence integer, geom geometry) 
-- implementation follows here!

Note how this function uses RETURNS TABLE to enable it to return all the stops that it finds. To add a line to the output table, we need to assign values to the sequence and geom variables and then use RETURN NEXT.

Another reason to use PL/pgSQL is that it enables us to write loops. And loops I wanted for my stop detection function! Specifically, I wanted to go through all the points in the trajectory:

FOR pt IN SELECT (ST_DumpPoints(traj)).geom LOOP
-- here comes the magic!

Eventually the function should go through the trajectory and identify all segments that stay within an area with max_size diameter for at least min_duration time. To test for the area size, we can use:

IF ST_MaxDistance(segment,pt) <= max_size THEN is_stop := true; 

Putting everything together, my current implementation looks like this:

   traj geometry,
   max_size numeric,
   min_duration numeric)
RETURNS TABLE(sequence integer, geom geometry) 
LANGUAGE 'plpgsql'
   pt geometry;
   segment geometry;
   is_stop boolean;
   previously_stopped boolean;
   stop_sequence integer;
   p1 geometry;
segment := NULL;
sequence := 0;
is_stop := false;
previously_stopped := false;
p1 := NULL;
FOR pt IN SELECT (ST_DumpPoints(traj)).geom LOOP
   IF segment IS NULL AND p1 IS NULL THEN 
      p1 := pt; 
   ELSIF segment IS NULL THEN 
      segment := ST_MakeLine(p1,pt); 
      p1 := NULL;
      IF ST_Length(segment) <= max_size THEN is_stop := true; END IF; 
      segment := ST_AddPoint(segment,pt); 
      -- if we're in a stop, we want to grow the segment, otherwise we remove points to the specified min_duration 
      IF NOT is_stop THEN 
         WHILE ST_NPoints(segment) > 2 AND AG_Duration(ST_RemovePoint(segment,0)) >= min_duration LOOP
            segment := ST_RemovePoint(segment,0); 
         END LOOP;
      END IF;
      -- a stop is identified if the segment stays within a circle of diameter = max_size
      IF ST_Length(segment) <= max_size THEN is_stop := true; 
      ELSIF ST_Distance(ST_StartPoint(segment),pt) > max_size THEN is_stop := false;
      ELSIF ST_MaxDistance(segment,pt) <= max_size THEN is_stop := true; 
      ELSE is_stop := false; 
      END IF; 
      -- if we found the end of a stop, we need to check if it lasted long enough 
      IF NOT is_stop AND previously_stopped THEN 
         IF ST_M(ST_PointN(segment,ST_NPoints(segment)-1))-ST_M(ST_StartPoint(segment)) >= min_duration THEN
            geom := ST_RemovePoint(segment,ST_NPoints(segment)-1); 
            RETURN NEXT;
            sequence := sequence + 1;
            segment := NULL;
            p1 := pt;
         END IF;
      END IF;
   END IF;
   previously_stopped := is_stop;
IF previously_stopped AND AG_Duration(segment) >= min_duration THEN 
   geom := segment; 

While this function is not really short, it’s so much more readable than my previous attempts of doing this in pure SQL. Some of the lines for determining is_stop are not strictly necessary but they do speed up processing.

Performance still isn’t quite where I’d like it to be. I suspect that all the adding and removing points from linestring geometries is not ideal. In general, it’s quicker to find shorter stops in smaller areas than longer stop in bigger areas.

Let’s test! 

Looking for a testing framework for PL/pgSQL, I found plpgunit on Github. While I did not end up using it, I did use its examples for inspiration to write a couple of tests, e.g.

CREATE OR REPLACE FUNCTION test.stop_at_beginning() RETURNS void LANGUAGE 'plpgsql'
DECLARE t0 integer; n0 integer;
WITH temp AS ( SELECT AG_DetectStops(
   ST_GeometryFromText('LinestringM(0 0 0, 0 0 1, 0.1 0.1 2, 2 2 3)'),
   1,1) stop 
SELECT ST_M(ST_StartPoint((stop).geom)), 
       ST_NPoints((stop).geom) FROM temp INTO t0, n0;	
IF t0 = 0 AND n0 = 3
   THEN RAISE INFO 'PASSED - Stop at the beginning of the trajectory';
   ELSE RAISE INFO 'FAILED - Stop at the beginning of the trajectory';

Basically, each test is yet another PL/pgSQL function that doesn’t return anything (i.e. returns void) but outputs messages about the status of the test. Here I made heavy use of the PERFORM statement which executes the provided function but discards the results:

Movement data in GIS #14: updates from GI_Forum 2018

Last week, I traveled to Salzburg to attend the 30th AGIT conference and co-located English-speaking GI_Forum. Like in previous year, there were a lot of mobility and transportation research related presentations. Here are my personal highlights:

This year’s keynotes touched on a wide range of issues, from Sandeep Singhal (Google Cloud Storage) who – when I asked about the big table queries he showed – stated that they are not using a spatial index but are rather brute-forcing their way through massive data sets, to Laxmi Ramasubramanian @nycplanner (Hunter College City University of New York) who cautioned against tech arrogance and tendency to ignore expertise from other fields such as urban planning:

One issue that Laxmi particularly highlighted was the fact that many local communities are fighting excessive traffic caused by apps like Waze that suggest shortcuts through residential neighborhoods. Just because we can do something with (mobility) data, doesn’t necessarily mean that we should!

Not limited to mobility but very focused on open source, Jochen Albrecht (Hunter College City University of New York) invited the audience to join his quest for a spatial decision support system based on FOSS only at and

The session Spatial Perspectives on Healthy Mobility featured multiple interesting contributions, particularly by Michelle P. Fillekes who presented a framework of mobility indicators to assess daily mobility of study participants. It considers both spatial and temporal aspects of movement, as well as the movement context:

Figure from Michelle Pasquale Fillekes, Eleftheria Giannouli, Wiebren Zijlstra, Robert Weibel. Towards a Framework for Assessing Daily Mobility using GPS Data. DOI: 10.1553/giscience2018_01_s177 (under cc-by-nd)

It was also good to see that topics we’ve been working on in the past (popularity routing in this case) continue to be relevant and have been picked up in the German-speaking part of the conference:

Of course, I also presented some new work of my own, specifically my research into PostGIS trajectory datatypes which I’ve partially covered in a previous post on this blog and which is now published in Graser, A. (2018) Evaluating Spatio-temporal Data Models for Trajectories in PostGIS Databases. GI_Forum ‒ Journal of Geographic Information Science, 1-2018, 16-33. DOI: 10.1553/giscience2018_01_s16.

My introduction to GeoMesa talk failed to turn up any fellow Austrian GeoMesa users. So I’ll keep on looking and spreading the word. The most common question – and certainly no easy one at that – is how to determine the point where it becomes worth it to advance from regular databases to big data systems. It’s not just about the size of the data but also about how it is intended to be used. And of course, if you are one of those db admin whizzes who manages a distributed PostGIS setup in their sleep, you might be able to push the boundaries pretty far. On the other hand, if you already have some experience with the Hadoop ecosystem, getting started with tools like GeoMesa shouldn’t be too huge a step either. But that’s a topic for another day!

Since AGIT&GI_Forum are quite a big event with over 1,000 participants, it was not limited to movement data topics. You can find the first installment of English papers in GI_Forum 2018, Volume 1. As I understand it, there will be a second volume with more papers later this year.

This post is part of a series. Read more about movement data in GIS.

QGIS Server certified as official OGC reference implementation

We are very excited to announce that QGIS Server has been successfully certified as a compliant WMS 1.3 server against the OGC certification platform, and moreover, it is even considered as a reference implementation now!

This is the first step on our roadmap of having a fast, compliant and bullet proof web map server that is straightforward to publish from a classical QGIS project.

What does it mean?

Having a certified server means that QGIS Server successfully passes the automated and semi automated tests that ensure we are 100% compliant with the standards. That means you can trust QGIS to be used by any WMS client seamlessly.
Moreover, that certification is now powered by a continuous integration system that checks every night in developement versions if we still pass the tests.

Daily compliance reports are available on the new website.

What’s next?

Building the automated testing platform and getting officially certified was only the first step. We now are starting to certify the WFS services, thanks to the latest grant application program support.

We also want QGIS server development to be performance-driven. The following projects are particularly relevant:

  • MS-Perf produces benchmark reports with MapServer and GeoServer.
  • graffiti  and PerfSuite tools have been designed to create a really light tool, easy to enrich with new datasets and performance tests, and easy to integrate in continuous integration systems. It compares QGIS-ltr, QGIS-rel and QGIS-dev nightlies for the same scenarios in details and produces html reports. It can also graph performance history for the development version to track regressions or improvements.

Many thanks to the supporters and voting members that helped bootstrap all those testing platforms and offer them to the community.

If you want to support or give a hand on the QGIS desktop client side, we think that area would deserve some love too!

QGIS 3.2 Bonn is released!

We are pleased to announce the release of QGIS 3.2 ‘Bonn’. The city of Bonn was the location of our 16th developer meeting.


This is the second release in the 3.x series. It comes with tons of new features (see our visual changelog).

Packages and installers for all major platforms are available from

We would like to thank the developers, documenters, testers and all the many folks out there who volunteer their time and effort (or fund people to do so). From the QGIS community we hope you enjoy this release! If you wish to donate time, money or otherwise get involved in making QGIS more awesome, please wander along to and lend a hand!

QGIS is supported by donors and sponsors. A current list of donors who have made financial contributions large and small to the project can be seen on our donors list. If you would like to become and official project sponsor, please visit our sponsorship page for details. Sponsoring QGIS helps us to fund our six monthly developer meetings, maintain project infrastructure and fund bug fixing efforts. A complete list of current sponsors is provided below – our very great thank you to all of our sponsors!

QGIS is Free software and you are under no obligation to pay anything to use it – in fact we want to encourage people far and wide to use it regardless of what your financial or social status is – we believe empowering people with spatial decision making tools will result in a better society for all of humanity.





QGIS Grant Programme 2018 Results

We are extremely pleased to announce the winning proposals for our 2018 QGIS.ORG grant programme. Funding for the programme was sourced by you, our project donors and sponsorsNote: For more context surrounding our grant programme, please see:

The QGIS.ORG Grant Programme aims to support work from our community that would typically not be funded by client/contractor agreements, and that contributes to the broadest possible swathe of our community by providing cross-cutting, foundational improvements to the QGIS Project.

Voting to select the successful projects was carried out by our QGIS Voting Members. Each voting member was allowed to select up to 6 of the 14 submitted proposals by means of a ranked selection form. The full list of votes are available here (on the first sheet). The second sheet contains the calculations used to determine the winner (for full transparency). The table below summarizes the voting tallies for the proposals:


A couple of extra notes about the voting process:

  • The PSC has an ongoing program to fund documentation so elected to fund the QGIS Training Manual update even if this increases the total funded amount beyond the initial budget.
  • Although the budget for the grant programme was €25,000, the total amount for the winning proposals is €35,500. This increase is possible thanks to the generous support by our donors and sponsors this year.
  • Voting was carried out based on the technical merits of the proposals and the competency of the applicants to execute on these proposals.
  • No restrictions were in place in terms of how many proposals could be submitted per person / organization, or how many proposals could be awarded to each proposing person / organization.
  • Voting was ‘blind’ (voters could not see the existing votes that had been placed).

Of the 45 voting members, 29 registered their votes 17 community representatives and 12 user group representatives.

On behalf of the QGIS.ORG project, I would like to thank everyone who submitted proposals for this call!

A number of interesting and useful proposal didn’t make it because of our limited budget; we encourage organizations to pick up one of their choice and sponsor it.


Pomysł na zorganizowanie tego spotkania kiełkował od lat, ale zawsze brakowało czasu na jego realizację. Główny problem polegał na określeniu docelowej grupy uczestników oraz formuły. Wiemy oczywiście, jak szerokie jest grono użytkowników programu QGIS, ale ta wiedza wcale nie ułatwiała zadania. Ostatecznie po przeprowadzeniu kilku ankiet na polskim forum QGIS postanowiliśmy zaserwować wszystkiego po trochu. Potrzebni byli tylko prelegenci, miejsce i termin. Zdecydowaliśmy się na 19 czerwca 2018…


Pomysł na zorganizowanie tego spotkania kiełkował od lat, ale zawsze brakowało czasu na jego realizację. Główny problem polegał na określeniu docelowej grupy uczestników oraz formuły. Wiemy oczywiście, jak szerokie jest grono użytkowników programu QGIS, ale ta wiedza wcale nie ułatwiała zadania. Ostatecznie po przeprowadzeniu kilku ankiet na polskim forum QGIS postanowiliśmy zaserwować wszystkiego po trochu. Potrzebni byli tylko prelegenci, miejsce i termin. Zdecydowaliśmy się na 19 czerwca 2018…

Using Threads in PyQGIS3

While porting a plugin to QGIS3 I decided to also move all it’s threading infrastructure to QgsTasks. Here three possible variants to implement this. the first uses the static method QgsTask.fromFunction and is simpler to use. A great quick solution.

PyQGIS for non-programmers

If you’re are following me on Twitter, you’ve certainly already read that I’m working on PyQGIS 101 a tutorial to help GIS users to get started with Python programming for QGIS.

I’ve often been asked to recommend Python tutorials for beginners and I’ve been surprised how difficult it can be to find an engaging tutorial for Python 3 that does not assume that the reader already knows all kinds of programming concepts.

It’s been a while since I started programming, but I do teach QGIS and Python programming for QGIS to university students and therefore have some ideas of which concepts are challenging. Nonetheless, it’s well possible that I overlook something that is not self explanatory. If you’re using PyQGIS 101 and find that some points could use further explanations, please leave a comment on the corresponding page.

PyQGIS 101 is a work in progress. I’d appreciate any feedback, particularly from beginners!

Create a QGIS vector data provider in Python is now possible


Why python data providers?

My main reasons for having Python data provider were:

  • quick prototyping
  • web services
  • why not?


This topic has been floating in my head for a while since I decided to give it a second look and I finally implemented it and merged for the next 3.2 release.


How it’s been done

To make this possible I had to:

  • create a public API for registering the providers
  • create the Python bindings (the hard part)
  • create a sample Python vector data provider (the boring part)
  • make all the tests pass


First, let me say that it wasn’t like a walk in the park: the Python bindings part is always like diving into woodoo and black magic recipes before I can get it to work properly.

For the Python provider sample implementation I decided to re-implement the memory (aka: scratch layers) provider because that’s one of the simplest providers and it does not depend on any external storage or backend.


How to and examples

For now, the main source of information is the API and the tests:

To register your own provider (PyProvider in the snippet below) these are the basic steps:

metadata = QgsProviderMetadata(PyProvider.providerKey(), PyProvider.description(), PyProvider.createProvider)

To create your own provider you will need at least the following components:

  • the provider class itself (subclass of QgsVectorDataProvider)
  • a feature source (subclass of QgsAbstractFeatureSource)
  • a feature iterator (subclass of QgsAbstractFeatureIterator)

Be aware that the implementation of a data provider is not easy and you will need to write a lot of code, but at least you could get some inspiration from the existing example.


Enjoy wirting data providers in Python and please let me know if you’ve fond this implementation useful!

QGIS User Conference 2018

This year’s QGIS User Conference will be embedded within the FOSS4G conference. That means that you should refer to the FOSS4G web site for all registration and event details with the exception of the QGIS Hackfest (22-26 August) which follows our normal organisation procedure on the QGIS Wiki.

There are five main activities for QGIS Users to enjoy at the conference:

  1. The QGIS Hackfest: This is our twice-yearly meet up to work on and improve the QGIS code base. Come along to this event if you want contribute documentation, bug fixes, bug reports, features and other ideas for improvements to the QGIS project. The hackfest is a side event to be held on the island of Zanzibar and will happen before the main conference. Again refer to the QGIS Hackfest wiki page for details and to register.
  2. The Workshops: FOSS4G is always popular for the wide range of workshops presented. The workshop presentations are very often by the developers of the software being explained, which gives you a unique opportunity to talk face to face with the people using the software you are using. There will be a number of QGIS specific workshops at FOSS4G – you will be able to identify them easily on the mobile app that provides the conference programme by filtering on ‘QGIS’ in the workshops section. For a current list of all the workshops at the event please refer to: 
  3. The Presentations: There will be a wide range of QGIS related presentations during the main conference. 17 QGIS related presentations have been registered. See for a full list of the conference presentations.
  4. The Academic Track: This year a large number of academic papers have been submitted, a good number of which relate to QGIS. Watch the conference web site for more details to be published soon!
  5. The QGIS Plenary / Panel Discussion: During the conference we will have an ‘Ask Me Anything’ session with QGIS developers and community members forming a panel to field any questions you might want to ask! This is a unique opportunity to find out more about the project, its roadmap or just come along to say ‘thank you’ to the dedicated team of community members who put so much work into making QGIS!

If you have not already registered for FOSS4G2018, do so now to enjoy early bird rates! Visit the Registration Page for more details! We are looking forward to meeting you in Dar es Salaam and Zanzibar!

About the FOSS4G2018 Conference

The annual Free and Open Source Software 4 Geospatial. (FOSS4G) conference is the largest community gathering focused on open source geospatial software. FOSS4G brings together developers, users, decision-makers and observers from a broad spectrum of organizations and fields of operation. Through six days of workshops, presentations, discussions, and code sprints, FOSS4G participants, learn, exchange, and create new collaborations, geospatial products, standards.

FOSS4G is a global conference, with attendees from around the world! Last year, Boston, USA was the host city, before that in 2016 it Bonn, Germany. Now is the time for Dar es Salaam, Tanzania! FOSS4G 2018 will host numerous workshops over 27 – 28 August with exciting keynotes and presentations at the Julius Nyerere International Convention Centre 29 – 31 August, and finally Code Sprints and community building events September 1 & 2. In addition to bringing together the FOSS4G community in Dar, the 21st QGIS Developer meeting will take place in Zanzibar (a short 60min ferry trip away!) the 22 – 26 of August – the Geospatial Savanna is on the rise in East Africa!

The vision for the conference is to make it as accessible and inclusive as possible, bridging the gaps between the various communities that make us so strong, to make us stronger. This will be achieved through a Travel Grant Program that will enable economically disadvantaged participants the opportunity to attend, covering travel, lodging and conference costs. This will be bolstered by an exciting conference program, taking the very best that the open source geospatial community can offer. This ranges from locating schools to the instrumentation of communities with cheap, open source, 3D printed weather stations that improve community resilience to climate change.

The theme for FOSS4G 2018 is “Leave No-One Behind”. We live in a world where our users and developers have the opportunity to make a difference to the lives of everyone, whether it’s those in extreme poverty looking for access to clean water, to those using routing algorithms to calculate the fastest route to work. Location and geography are at the heart of these challenges and a FOSS4G 2018 in Dar es Salaam will invigorate our existing projects, bringing them to new users and developers while supporting and nurturing the existing community.

Locatieserver: about parcels and roadnumbers

For those interested in dutch OpenData: our national OpenData service PDOK has a Solr based geocoding service available. Since this week it is possible to search for ‘parcel’ codes to find cadastral parcels, but also to search on so called ‘hectometer-paaltjes': the little green number signs you see when you drive the dutch highways. So … Continue reading Locatieserver: about parcels and roadnumbers

How to filter features in QGIS using the graphical processing modeler

This article describes a new algorithm for the processing modeler called feature filter algorithm. If you are already familiar with ETL concepts and the graphical modeler, you can directly jump to the section the feature filter algorithm. Building workflows for

New PyQGIS documentation

We are proud to announce our new dedicated documentation of the QGIS Python API (also called PyQGIS) which is now available at

While the QGIS API has long been documented, Python developers in the past had to work with the general C++ documentation that wasn’t always straightforward to use. The new PyQGIS documentation presents the API in an accessible pythonic manner.

Of course, creating a good API documentation from source code in an automated way, is not trivial. A key challenge was to automatically create Python bindings files (or SIP files). A custom Perl script known as “sipify” now enables us to automatically integrate the C++ documentation into the Python bindings and keep them up to date. Another challenge was to create the documentation itself using Sphinx. Two detailed reports containing all the technical details of the first and second generation of the documentation are available if you want to learn more about the underlying architecture.

This has been a really important infrastructure project for QGIS that has been made possible by support from our donors and sponsors, as well as the generous in-kind contributions of our community members.

Scalable spatial vector data processing

Working with movement data analysis, I’ve banged my head against performance issues every once in a while. For example, PostgreSQL – and therefore PostGIS – run queries in a single thread of execution. This is now changing, with more and more functionality being parallelized. PostgreSQL version 9.6 (released on 2016-09-29) included important steps towards parallelization, including parallel execution of sequential scans, joins and aggregates. Still, there is no parallel processing in PostGIS so far (but it is under development as described by Paul Ramsey in his posts “Parallel PostGIS II” and “PostGIS Scaling” from late 2017).

At the FOSS4G2016 in Bonn, I had the pleasure to chat with Shoaib Burq who ran the “An intro to Apache PySpark for Big Data GeoAnalysis” workshop. Back home, I downloaded the workshop material and gave it a try but since I wanted a scalable system for storing, analyzing, and visualizing spatial data, it didn’t really seem to fit the bill.

Around one year ago, my search grew more serious since we needed a solution that would support our research group’s new projects where we expected to work with billions of location records (timestamped points and associated attributes). I was happy to find that the fine folks at LocationTech have some very promising open source projects focusing on big spatial data, most notably GeoMesa and GeoWave. Both tools take care of storing and querying big spatio-temporal datasets and integrate into GeoServer for publication and visualization. (A good – if already slightly outdated – comparison of the two has been published by Azavea.)

My understanding at the time was that GeoMesa had a stronger vector data focus while GeoWave was more focused on raster data. This lead me to try out GeoMesa. I published my first steps in “Getting started with GeoMesa using Geodocker” but things only really started to take off once I joined the developer chats and was pointed towards CCRI’s cloud-local “a collection of bash scripts to set up a single-node cloud on your desktop, laptop, or NUC”. This enabled me to skip most of the setup pains and go straight to testing GeoMesa’s functionality.

The learning curve is rather significant: numerous big data stack components (including HDFS, Accumulo, and GeoMesa), a most likely new language (Scala), as well as the Spark computing system require some getting used to. One thing that softened the blow is the fact that writing queries in SparkSQL + GeoMesa is pretty close to writing PostGIS queries. It’s also rather impressive to browse hundreds of millions of points by connecting QGIS TimeManager to a GeoServer WMS-T with GeoMesa backend.

Spatial big data stack with GeoMesa

One of the first big datasets I’ve tested are taxi floating car data (FCD). At one million records per day, the three years in the following example amount to a total of around one billion timestamped points. A query for travel times between arbitrary start and destination locations took a couple of seconds:

Travel time statistics with GeoMesa (left) compared to Google Maps predictions (right)

Besides travel time predictions, I’m also looking into the potential for predicting future movement. After all, it seems not unreasonable to assume that an object would move in a similar fashion as other similar objects did in the past.

Early results of a proof of concept for GeoMesa based movement prediction

Big spatial data – both vector and raster – are an exciting challenge bringing new tools and approaches to our ever expanding spatial toolset. Development of components in open source big data stacks is rapid – not unlike the development speed of QGIS. This can make it challenging to keep up but it also holds promises for continuous improvements and quick turn-around times.

If you are using GeoMesa to work with spatio-temporal data, I’d love to hear about your experiences.


Via twitter: Jan Hartman’s and WebMappers hard work of georeferencing a set of Old Amsterdam maps: 6 XYZ-Map services with maps old as 1625 to have a look into history, off course also to be loaded in QGIS Wanna see ‘the red light district’ in 1625? Or see Dutch 17th century glory on … Continue reading AmsterdamTimeMachine

Coding a QgsLocator (Plugin)

What is a Locator (plugin) Some months ago, Nyall Dawson silently dropped a nice widget into the lower left corner of your QGIS screen: People familiar with QtCreator (the Qt-development environment) should recognize it as a QtCreator Locator look-a-like: a way to (very) quickly search in your project for words, classes, bookmarks, help topics, files … Continue reading Coding a QgsLocator (Plugin)

Movement data in GIS #13: Timestamp labels for trajectories

In Movement data in GIS #2: visualization I mentioned that it should be possible to label trajectory segments without having to break the original trajectory feature. While it’s not a straightforward process, it is indeed possible to create timestamp labels at desired intervals:

The main point here is that we cannot use regular labels because there would be only one label for the whole trajectory feature. Instead, we are using a marker line with a font marker:

By default, font markers only display one character from a given font but by using expressions we can make it display longer text, including datetime strings:

If you want to have a label at every node of the trajectory, the expression looks like this:

         segments_to_lines( $geometry ),
      ))||' seconds'

You probably remember those parts of the expression that extract the m value from previous posts. Note that – compared to 2016 – it is now necessary to add the segments_to_lines() function.

The m value (which stores time as seconds since Unix epoch) is then converted to datetime and finally formatted to only show time. Of course you can edit the datetime format string to also include the date.

If we only want a label every 30 seconds, we can add a case statement around that:

   segments_to_lines( $geometry ),
)) % 30 = 0
         segments_to_lines( $geometry ),
      ))||' seconds'

This works well if the trajectory sampling interval is fairly regular. This is not always the case and that means that the above case statement wouldn’t find many nodes with a timestamp that ends in :30 or :00. In such a case, we could resort to labeling nodes based on their order in the linestring:

 @geometry_part_num  % 30 = 0

Thanks a lot to @JuergenEFischer for providing a solution for converting seconds since Unix epoch to datetime without a custom function!

Note that expressions using @geometry_part_num currently suffer from the following issue: Combination of segments_to_lines($geometry) and @geometry_part_num gives wrong segment numbers

This post is part of a series. Read more about movement data in GIS.

Reports from the winning grant proposals 2017

While we are waiting for this year’s grant proposals to come in, it is time to look back at last year’s winning proposals and their results. These are the reports on the work that has been done within the individual projects:

QGIS 3D – Martin Dobias

Results are included in the QGIS 3.0 release. As proposed in the grant, a new 3D map view has been added together with GUI for easy configuration of 3D rendering. The 3D view displays terrain (either from a DEM raster layer or a simple flat area) with 2D map rendered on top of the terrain. In addition to that, vector layers can be rendered as true 3D entities: points may be visualized as simple geometric shapes or as 3D models (loaded from a file), polygons and linestrings are tessellated into 3D geometries. 2D polygons can be turned into 3D objects using extrusion, possibly with data-defined height – an easy way how to display buildings, for example. Data with 3D coordinates have the Z values in geometries respected. Although the 3D view is still in its early stages, it is already usable for many use cases. Hopefully this functionality will help to attract even more users to QGIS!

More details:

Improvements to relations – Régis Haubourg

Various improvements for deep relations with PostgreSQL were successfully added in QGIS 3.0:

Add consistency to UI controls – Nyall Dawson

We’ve unified all the various opacity, rotation and scale controls to use the same terminology and numeric scales. We’ve also updated ALL methods for setting opacity, rotation and scale within the PyQGIS API to use consistent naming and arguments, making the API more predictable and easy to use. Lastly, we’ve also added a new reusable opacity widget (QgsOpacityWidget) to the GUI library so that future code can (and 3rd party scripts and plugins) can follow the new UI conventions for opacity handling.

Extend unit test coverage for geometry classes – Nyall Dawson

We’ve extended the unit testing coverage for all the underlying geometry primitive classes (points, lines, polygons, curves, collections, etc) so that all these classes have as close to 100% unit test coverage as possible. In the process, we identified and fixed dozens of bugs in the geometry library, and naturally added additional unit tests to avoid regressions in future releases. As a result QGIS’ core geometry engine is much more stable. Furthermore, we utilised the additional test coverage to allow us to safely refactor some of the slower geometry operations, meaning that many geometry heavy operations will perform much faster in QGIS 3.0.

Processing algorithm documentation – Matteo Ghetta & Alexander Bruy

The new Help system is landed and already available: when opening a Processing algorithm and clicking on the Help button, the guide of the algorithm will be showed in the default browser.

Many of the QGIS Processing algorithm guides have been enhanced with pictures and new or enhanced descriptions. A consistency number of Pull Requests have been already merged and many others are in review. Just a few descriptions need to be still enhanced.

Currently all the QGIS algorithms have been described and all the PR in the doc repository have been merged (kudos to Harrissou for all the reviews!).

Right now the Help button of each Processing dialog will open the related page of the algorithm, BUT:

  • if the name of the algorithm is made by only ONE word (e.g. clip, intersection…), the help button will open the browser to also the correct section (that is, the user will see directly the description of the related algorithm)
  • if the name of the algorithm has >1 words (e.g. split polygon with lines, lines to polygon, ecc.) the Help button will open the correct page (so the algorithm GROUP) but is not able to go to the correct algorithm anchor. This is because sphinx converts “split with lines” in “split-with-lines” while QGIS system will always cast the words “split-with-lines” in “splitwithlines”. Not a big deal, but IMHO a pity.
    We are really too close to the solution.

So Processing Help system right now consists of:

  • QGIS algs -> documented
  • GDAL algs -> documented
  • GRASS -> documented (own docs)
  • Orfeo -> documented (own docs)
  • SAGA -> nothing documented

Thanks to QGIS Grants to provide this chance to give a big improvement to the Processing framework even if not in a coding way!

Last but not least, we had another project that was not part of the grant programme but was also funded by QGIS.ORG in 2017:

Python API documentation – Denis Rouzaud

QGIS Python API Documentation is created using Sphinx and this work is available on Github. The repo is a fork of QGIS’ one and has been merged in the meantime. The docs are available at It uses a new theme (sphinx_rtd_theme aka ReadTheDocs theme). Some improvements were brought in (not exhaustive):

  • QGIS theming with colors and icon
  • Foldable toctree
  • Summary of methods and attributes for classes
  • Module index (not available before)
  • Correct display of overloaded methods

Full Python signature in Docstring

In former SIP versions, it was not possible to use the auto generated signature if a Docstring already existed. This means any documented method could not have a signature created. Unfortunately for this project, the vast majority of methods in QGIS API are documented!

The source code of SIP was modified and theses changes got merged upstream. See rev 1788 to 1793 in SIP changelog. It will be released in upcoming 4.19.7 version. QGIS source code was modified accordingly to prepend auto generated Python signatures to existing Docstrings. Using a CMake configuration file for each module (,, etc.) was required to avoid syntax errors when using former version of SIP (since bumping minimum version is not realistic).

Sipify adjustments

Many things were fixed in sipify script :

  • Creation of links to classes, methods
  • Handling/fixing of Doxygen annotations \see, \note, \param
  • Handling of code snippets: c++ vs Python. Only Python are shown.

Thank you to everyone who participated and made this round of grants a great success and thank you to all our sponsor and donors who make this initiative possible!


Drill down (cascading) forms in QGIS crowdfund – final stretch!

Update: donations are now closed, with the outcome of the campaign pending!

We’re nearing the final hours of our crowd funding campaign to implement a drill-down (cascading) field support within QGIS forms, and thanks to numerous generous backers we’re very close to hitting the funding goal! This is a really exciting new feature which would help add greater flexibility and power to QGIS feature forms, but in order to implement it for QGIS 3.2 we need to hit the funding target by 11 May 2018.

As a result, we’re dropping the minimum contribution amount and throwing open the campaign for payments of any amount. These smaller payment will be treated as direct donations to the campaign, so unlike the standard campaign backing these are payable up front. In the case that the campaign IS NOT successful, the donations will not be refunded and will instead be reinvested back into the QGIS (via bug fixing and maintenance efforts). Of course, if you’d prefer to pledge using the standard crowdfunding “no payment if campaign unsuccessful” model you’re more than welcome to! (Full details are available on the campaign page).

Donations closed – outcome pending!

Full details are available on the campaign page.

  • Page 1 of 98 ( 1948 posts )
  • >>

Back to Top