I’ve previously written about Movement data in GIS and the AI hype and today’s post is a follow-up in which I want to share with you a new review of the state of the art in deep learning from trajectory data.

Our review covers 8 use cases:

1. Location classification
2. Arrival time prediction
3. Traffic flow / activity prediction
4. Trajectory prediction
5. Trajectory classification
6. Next location prediction
7. Anomaly detection
8. Synthetic data generation

We particularly looked into the trajectory data preprocessing steps and the specific movement data representation used as input to train the neutral networks:

On a completely subjective note: the price for most surprising approach goes to natural language processing (NLP) Transfomers for traffic volume prediction.

The paper was presented at BMDA2023 and you can watch the full talk recording here:

References

Graser, A., Jalali, A., Lampert, J., Weißenfeld, A., & Janowicz, K. (2023). Deep Learning From Trajectory Data: a Review of Neural Networks and the Trajectory Data Representations to Train Them. Workshop on Big Mobility Data Analysis BMDA2023 in conjunction with EDBT/ICDT 2023.

This post is part of a series. Read more about movement data in GIS.

Changes

🐛 Bug Fixes

• Significant improvement in Bluetooth connection reliability
• Dark theme tweaks

The QGIS Cloud Support Team is often asked if it is possible to integrate data from external databases into the QGIS Cloud Web Map, so that the map information in the QGIS Cloud Web Map is always up-to-date. In this blog article we would like to show how this is possible with PostgreSQL. To publish data in the QGIS Cloud GDI, the data must always be stored in the personal QGIS Cloud database.

Get your QGIS-ass to ‘s-Hertogenbosch!

Changes

🐛 Bug Fixes

• Slightly higher rotation kickstart threshold to avoid accidental rotation while pinch zooming
• Fix moving of features on rotated map canvas
• Fix 'invalid relation' on fields with valid relations
• Fix WFS provider on iOS

With the QGIS Grant Programme 2022, we were able to support four proposals that are aimed to improve the QGIS project, including software, infrastructure, and documentation. The following reports summarize the work performed in the proposals. 

1. Support building QGIS application on Qt 6 (#243) – Report
   In addition to the original plan of porting the “gui” and “app” libraries to Qt 6, it was possible to complete also the “3d” and “server” libraries. We now are at a stage where the majority of QGIS builds and runs without any significant issues on Qt 6. The Github CI setup has been updated to also run the C++ tests for gui, app, server and 3d, and the majority of these have been fixed so that they pass on the Qt 6 builds too. In addition, some tests which were failing under Qt 6 revealed some real QGIS bugs which have been fixed in the process of this work. (So there’s a direct benefit for the existing Qt 5 builds too!).
2. Add SQL Logging in the debugging/development panel (#242) – Report
   When debugging or developing a QGIS algorithm or a QGIS plugin and when investigating performances of a particular layer it is often useful to view the SQL commands that QGIS sends to the backend. The SQL logging was implemented for Postgres, GeoPackage, Spatialite and Oracle data providers.
3. QGIS setting registry follow-up (#245) – Report
   The work can mainly be seen here PR qgis/QGIS#51295 with the proposed approach to register settings in a hierarchical and organized way, without too much complexity in the API to actually use the settings.
   To have a clean approach, some keys have been renamed. There is a compatibility handling (both forward and backward). The GUI implementation will be worked on during the HF in NL this spring.
4. Fix handling of provider default value clauses/Autogenerate/nextval(…) handling (#247) – Report
   The bulk of these changes landed in the QGIS 3.28 release. A quick way to demonstrate on of the issues fixed is: open a Geopackage file, start editing the layer, add some features to it, but don’t save the edits, then right click the edited layer and try to save it to a different file. On older QGIS releases you’ll be spammed with a number of error messages because we tried to write a string value of “Autogenerate” into a number field for all the newly created features. On QGIS 3.28 this all just works as expected, with no errors encountered.

Thank you to everyone who participated and made this round of grants a great success and thank you to all our sustaining members and donors who make this initiative possible!

It is with great pleasure that we would like to welcome Felt as our first flagship sustaining member!

Felt plays a revolutionary role for those who work with maps daily by making maps and spatial analysis more accessible across organizations through modern web-based collaboration features. Teams across planning, tech, infrastructure, consulting, environmental sciences and more are using QGIS & Felt to access the power of desktop GIS alongside the ease of modern web-based collaboration features, including:

• Customize your maps with browser-based tools: draw, drop pins, leave notes, trace boundaries, find routes — even add a video — all just point and click
• Collaborate with your team on the same map in the moment, or hours and miles apart
• Invite one, a few, or the whole world to see your work with precise access controls

Have a look yourself and discover the inspiring maps shared by Felt users.

We are particularly excited about this collaboration since Felt users regularly pair the analytical power of QGIS with the team-based sharing and collaboration features of Felt. Together, we contribute to a winning ecosystem of easy-to-use map-making tools for modern teams and organizations.

Felt’s support for QGIS helps us reach the goal of a stable financial basis to ensure another 20+ years of sustainable development and keep bringing the most user-friendly GIS to users worldwide.

Changes

🚀 Features

• New action to import a dataset of compressed project from a remote URL (see documentation)

🐛 Bug Fixes

• Fix digitizing of 3D points on shapefile datasets when not locked to current GNSS position

Sourcepole hat an der FOSSGIS 2023 in Berlin verschiedene Themen mit Vorträgen abgedeckt:

• QGIS Web Client 2 (QWC2) - Neues aus dem Projekt
• COPC, das neue cloudoptimierte Format für Point Clouds
• Cloudoptimierte Formate für Kacheln und multidimensionale Rasterdaten
• Geodatenverarbeitung mit Workflow-Engines

Changes

🚀 Features

• Map canvas rotation support (via pinch rotation gesture and mouse middle button)
• Audio and video attachment recording within the feature form
• Audio and video attachment playback within the feature form
• File attachment support added to the feature form
• User interface dark theme styling (defaulting to current system default on Android)
• User interface font scaling
• A completely revamped non-native camera featuring zoom support, flash toggle, GPS location metadata, and a much better interface

✨ Improvements

• Default value expressions relying on aggregate() now work properly within the feature form
• Image metadata values are now preserved when setting a maximum image size setting (via qfieldsync)
• Feature form’s field labels will now respect the color and style overrides defined in project files
• Upstream libraries updated, including brand new QGIS 3.30

🐛 Bug Fixes

• Vertical grid datasets containing spaces in their paths now working

QGIS 3.30 was released late last week and we are pleased to announce the new features introduced as a part of our latest crowdfunding campaign to improve 3D, point cloud and elevation data.

Thank you

First and foremost, thanks to the generous support from the community to fund our work. Here is the list of our contributors in no particular order:

IGN (INSTITUT NATIONAL DE L’INFORMATION GEOGRAPHIQUE ET FORESTIERE), National Land Survey of Finland, Danish Agency for Data Supply and Infrastructure, K2 Engineering GmbH, ProScape Consulting, Västra Götalandsregionen, Kristianstads kommun, IGN FI, L’Arrière Guichet, Septima, QWAST-GIS, ATAPEX s.r.o., REDcatch GmbH, F.A.R.M. Facilitazioni Agroecologiche Regionali Mobili, EPYMA TERRITORIO Y MEDIO AMBIENTE, SL, GEO EXPLORATION LTD, Bohannan Huston, Inc., Lidar Guys, Neuchâtel- Service de la géomatique, Wooding Geospatial Solutions, Ville de Vevey, QGIS User Group Switzerland, Ecophylla Consulting, Refactor, Locate Press, Alta ehf, Oester Messtechnik GmbH, RUDAZ+PARTNER AG, BayesMap Solutions LLC, GEOACE, Natalie Gyles, Andreas Neumann, Dougal Munro, Spatial Thoughts, Cicada Systems GIS Consulting, Cori Hermle, Powell Asset Mapping LLC, Darren Farmer, Greg Hall, Ecothought Pty Ltd, Gabriel Diosan, Bhutan QGIS Group, Ultimatum Finesse, Balanced Risk Strategies, Ltd, Concordia University, Burmis Studio Inc., Nicholas Hadaller, Angello Villatoro, Yoichi Kayama, Hennessy Amor Becerra Ayala, Flow Design Limited, BNHR.XYZ, Roberto Moyano, Benjamin Kuster, Goldspot, North River Geographic Systems, Inc, David W. Wormuth, Victor Graphics, Valley Spatial, Stephen Mather, SANTIAGO AURELIO MOTA, Kelly Crowell, Brian Duhan, Paddy Fisher, OSGEO:UK, Christian Gugl, GIP CRAIG - Centre Régional Auvergne-Rhône-Alpes de l’Infromation Géographqiue, Raphael Mabit, Tibor Lieskovský, Kerstin Fohlert, Zhan Li, Bernd Vogelgesang, Marlin Müller, Johannes Bonekamp, Stefan Giese, Fabian Faßnacht, QGIS Sweden user group, Falo, DAVID GARCIA HERNANDEZ, Lint Data and Geospatial, Cliff Sin Wai Lau, Grzegorz Sapijaszko, Łukasz Rapa, Alessandro Pintucci and Maarten Pronk.

Our gratitude also goes to those who want to remain anonymous.

Global map shading

To see this feature in action, you can open QGIS project properties and under Terrain, there should be an option for Global Map Shading. You will need to first add a raster as your DEM under the Terrain section.

Elevations of all these layers are combined and the considered elevation is chosen depending on one of two methods:

1. the highest elevation between raster, mesh or point cloud layers will be selected.
2. elevation will be selected based on the order of layers in the layer panel.

Depending on the context and the use of the map the user can choose the more appropriate method.

For now, the shading methods implemented are the Eye Dome Lighting and the hill-shade. More methods could be added in the future - such as ambient occlusion.

The user can choose the elevation shading settings in a specific UI widget that can be found in two places:

1. under the Project Properties, within the same tab of the project elevation settings.
2. under the Styling Panel, a new tab is added for quick access to the user.

Profile elevation within print composer

This work was carried out by our collaborator North Road. The profile tool can embed elevation profiles within print layouts. It is possible to add beautifully styled profiles in your print outputs.

To use elevation profiles in print layouts, simply click the Add Elevation Profile button in the toolbar, and then to initialize it, copy profile configuration from an existing elevation profile from QGIS main window using the “Copy From Profile” button.

3D navigation improvements

With the new improvements, you can:

1. move camera vertically using ctrl+shift+left mouse button
2. keep zooming with the wheel while moving the mouse
3. continue right mouse button zooming when pointer exits the viewport

Improve zoom-extents in 3D

The terrain’s and point cloud layers’ elevation range are taken into account so that the camera is not positioned below the scene’s contents, which was the case when using the terrain’s vertical scale setting to exaggerate the elevation differences.

Limit 3D scenes’ 2D extent

3D Views can now be limited to a specific 2D extent. The terrain is clipped and no 3D features beyond that extent are loaded, making it easy to render specific areas of big QGIS projects. The project’s 2D extent is used by default which can then be adjusted in each 3D view separately using the new General tab in 3D configuration.

Future updates

There will be more features planned for QGIS 3.32. We are finalising the Processing framework for point cloud data and it should be available on QGIS master in coming weeks.

Please do not hesitate to contact us if you have any suggestions to improve QGIS 3D and point cloud support.

We are pleased to announce the release of QGIS 3.30 ‘s-Hertogenbosch!

Installers for all supported operating systems are already out. QGIS 3.30 comes with tons of new features, as you can see in our visual changelog. QGIS 3.30 ‘s-Hertogenbosch is named after this year’s QGIS user conference and contributor meeting host city.

We would like to thank the developers, documenters, testers and all the many folks out there who volunteer their time and effort (or fund people to do so). From the QGIS community we hope you enjoy this release! If you wish to donate time, money or otherwise get involved in making QGIS more awesome, please wander along to qgis.org and lend a hand!

QGIS is supported by donors and sustaining members. A current list of donors who have made financial contributions large and small to the project can be seen on our donors list. If you would like to become a sustaining member, please visit our page for sustaining members for details. Your support helps us fund our six monthly developer meetings, maintain project infrastructure and fund bug fixing efforts.

QGIS is Free software and you are under no obligation to pay anything to use it – in fact we want to encourage people far and wide to use it regardless of what your financial or social status is – we believe empowering people with spatial decision making tools will result in a better society for all of humanity.

In a few weeks, our 25th Contributor Meeting and International QGIS User Conference uc2023.qgis.nl will take off on 18 April.

Register or become a speaker yourself. For details on all the exciting opportunities to get involved, check out the introduction by our awesome organizing team:

Today’s post is a geeky deep dive into how to leverage DVC (not just) data version control to track QGIS geoprocessing workflows.

“Why is this great?” you may ask.

DVC tracks data, parameters, and code. If anything changes, we simply rerun the process and DVC will figure out which stages need to be recomputed and which can be skipped by re-using cached results.

This can lead to huge time savings compared to re-running the whole model

You can find the source code used in this post on my repo https://github.com/anitagraser/QGIS-resources/tree/dvc

I’m using DVC with the DVC plugin for VSCode but DVC can be used completely from the command line, if you prefer this appraoch.

Basically, what follows is a proof of concept: converting a QGIS Processing model to a DVC workflow. In the following screenshot, you can see the main stages

1. The QGIS model in the upper left corner
2. The Python script exported from the QGIS model builder in the lower left corner
3. The DVC stages in my dvc.yaml file in the upper right corner (And please ignore the hello world stage. It’s a left over from my first experiment)
4. The DVC DAG visualizing the sequence of stages. Looks similar to the QGIS model, doesn’t it ;-)

Besides the stage definitions in dvc.yaml, there’s a parameters file:

random-points:
  n: 10
buffer-points:
  size: 0.5

And, of course, the two stages, each as it’s own Python script.

First, random-points.py which reads the random-points.n parameter to create the desired number of points within the polygon defined in qgis3/data/test.geojson:

import dvc.api

from qgis.core import QgsVectorLayer
from processing.core.Processing import Processing
import processing

Processing.initialize()

params = dvc.api.params_show()
pts_n = params['random-points']['n']

input_vector = QgsVectorLayer("qgis3/data/test.geojson")
output_filename = "qgis3/output/random-points.geojson"

alg_params = {
    'INCLUDE_POLYGON_ATTRIBUTES': True,
    'INPUT': input_vector,
    'MAX_TRIES_PER_POINT': 10,
    'MIN_DISTANCE': 0,
    'MIN_DISTANCE_GLOBAL': 0,
    'POINTS_NUMBER': pts_n,
    'SEED': None,
    'OUTPUT': output_filename
}
processing.run('native:randompointsinpolygons', alg_params)

And second, buffer-points.py which reads the buffer-points.size parameter to buffer the previously generated points:

import dvc.api
import geopandas as gpd
import matplotlib.pyplot as plt

from qgis.core import QgsVectorLayer
from processing.core.Processing import Processing
import processing

Processing.initialize()

params = dvc.api.params_show()
buffer_size = params['buffer-points']['size']

input_vector = QgsVectorLayer("qgis3/output/random-points.geojson")
output_filename = "qgis3/output/buffered-points.geojson"

alg_params = {
    'DISSOLVE': False,
    'DISTANCE': buffer_size,
    'END_CAP_STYLE': 0,  # Round
    'INPUT': input_vector,
    'JOIN_STYLE': 0,  # Round
    'MITER_LIMIT': 2,
    'SEGMENTS': 5,
    'OUTPUT': output_filename
}
processing.run('native:buffer', alg_params)

gdf = gpd.read_file(output_filename)
gdf.plot()

plt.savefig('qgis3/output/buffered-points.png')

With these things in place, we can use dvc to run the workflow, either from within VSCode or from the command line. Here, you can see the workflow (and how dvc skips stages and fetches results from cache) in action:

If you try it out yourself, let me know what you think.

An amazing 30 new sustaining members have already answered our crowd-funding call raising total member contributions to €158,000 annually.

We are particularly happy to welcome our first medium-level university sustaining member, the University of Zurich, Department of Geography:

New medium sustaining members

New small sustaining members

• Pacific Geomatics Limited, Canada
• Helix Resources Limited, Australia
• Sand Hill Geographic, Virginia, United States
• GIS Pro Western Australia
• The Spatial Distillery Company, Victoria, Australia
• Qwast-GIS, The Netherlands
• CEICOL, Colombia
• QGIS user group Norway
• Robex resources, Quebec, Canada
• analyGIS GmbH, Switzerland
• CartoExpert, France
• addresscloud, UK
• Baugeologie und Geo-Bau-Labor AG, Switzerland
• Spatial Thoughts, India
• Centremaps, UK
• Geoideal, Colombia
• theworksLA, California, United States
• SoftWater s.r.l., Italy
• menz umweltplanung,Germany
• Oy Arbonaut Ltd, Finland
• ZevRoss Spatial Analysis, New York, United States
• Ecophylla Consulting, Ontario, Canada
• DeBeer&DeVos BV, Belgium
• Reuther NetConsulting, Germany
• H13, Denmark
• Rockwater Pty Ltd, Australia

Even with this impressive list of new members, we also realize that we have not reached the campaign goals yet and that potential large and flagship members may need more time to respond.

Our funding progress so far:

Therefore, we are extending the campaign until the end of March 2023.

Become a sustaining member or donor. Every contribution counts.

Details: Sustaining Membership Program Overview

This month has been busy with 20 new plugins in the QGIS plugin repository.

Since it can be challenging to stay up to date, our monthly plugin update provides you a quick overview of the newest plugins. If any of the names or short descriptions piques your interest, you can find the direct link to the plugin page in the table below the screenshot.

Changes

🐛 Bug Fixes

• Fix freeze occurring when zooming to a 1-point vector layer
• Fix UUID editor widget not working when editable state is unchecked
• Fix parenting of serial port receiver
• Fix blurry 'show feature' button within the relation reference editor widget

It’s only been a few weeks into the new year, but we’ve got great news for you: a brand new QField 2.6 “Geeky Gecko ” has been released with a focus on positioning improvements, including Bluetooth support for Windows. And with that, we are delighted to remove the ‘beta’ status from QField for Windows.

New positioning features

Let’s open with a bang: QField 2.6 now supports NMEA streaming from external GNSS devices over TCP, UDP, and serial ports, in addition to preexisting Bluetooth connectivity. This new functionality means that QField is now compatible with a much larger range of GNSS devices out there.

These new receivers unlock NTRIP-driven centimetre accuracy for devices that use the Bluetooth connection to a manufacturer’s application to connect to NTRIP servers. In this scenario, QField could not initiate a Bluetooth connection since it was already taken. With the new TCP and UDP receivers – provided the manufacturer’s application offers NMEA streaming over either of those Internet protocols – QField can connect and consume high-accuracy positioning.

The presence of a serial port receiver provides support for external GNSS devices using Bluetooth on Windows via the virtual serial port created by the operating system. The lack of Bluetooth support on Windows was a long-wanted enhancement from QField users on that platform and was the last blocker for the ‘beta’ status to go away.

In addition, QField 2.6 allows users to pick from half a dozen metrics a value to attach to the measure (M) dimension of geometries being digitized when locked to the current position. This functionality is available to both users digitizing and the positioning tracker. The measurement values available as of 2.6 are timestamp, ground speed, bearing, horizontal accuracy and vertical accuracy, as well as PDOP, HDOP and VDOP values.

Growing Continuous Integration (CI) testing framework now covers positioning

Starting with version 2.6, QField ships with increased quality assurances thanks to the addition of tests covering positioning functionalities in its growing CI framework.

Practically speaking, this means that every single line of QField code changed is now being tested against positioning-related regression, significantly decreasing the risks of shipping a new version of QField with broken functionality in the area of antenna height, vertical grid shift, and ellipsoidal height adjustments.

We would like to commend Deutsche Bahn for funding the required work here. This could not have come in soon enough as more and more people are opting for QField and relying on it for their crucial day-to-day fieldwork.

🚀 New features

• A new external GNSS devices management system that allows users to save device 'presets' making it easier to switch from one device to another (#3768, funded by Deutsche Bahn)
• A new pair of TCP and UDP receivers to support streaming of NMEA positioning data over IP (#3770, funded by Deutsche Bahn)
• Ability to choose the measurement (M) value to attach to vertices when tracking positioning or when digitizing while locked to position (#3731)

✨ Improvements

• Greatly improved external GNSS device stability and reliability with new automated tests acting as safeguards against future regressions (#3771, #3781, funded by Deutsche Bahn)
• Points created on 3D layers are now always 3D (with Z set to NaN if missing)
• Shortcut to go to a QR Code encoded point with the 'geo:' prefix when scanning through the search bar
• Printing of map layouts and atlases on iOS now works