New functions to add a timedelta column and get the trajectory sampling interval #233
As always, all tutorials are available from the movingpandas-examples repository and on MyBinder:
The new distance measures are covered in tutorial #11:
Computing distances between trajectories, as illustrated in tutorial #11Computing distances between a trajectory and other geometry objects, as illustrated in tutorial #11
But don’t miss the great features covered by the other notebooks, such as outlier cleaning and smoothing:
Trajectory cleaning and smoothing, as illustrated in tutorial #10
If you have questions about using MovingPandas or just want to discuss new ideas, you’re welcome to join our discussion forum.
We talked about Mergin Maps in the MapScaping podcast: QGIS Offline And In The Field
Peter Petrik was a guest in the episode of QGIS Offline And In The Field. He talked with Daniel O’Donohue about collection of spatial data in the field.
Mergin Maps is a field data collection app based on QGIS. It makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows.
Listen to the latest developments in point clouds and QGIS from Martin Dobias: MapScaping podcast.
Martin Dobias, our CTO and the lead developer of 3D and point clouds integration in QGIS sat down with Daniel O’Donohue from Mapscaping to talk about point clouds and QGIS.
Martin discusses his early involvment with QGIS back in 2005 and how he started his journey to become a QGIS developer.
When configuring the vector tiles in QGIS, we specify the desired tile and style URLs, for example:
For example, this is the “gis” style:
And this is the “basic” style:
The second vector tile source I want to mention is basemap.at. It has been around for a while, however, early versions suffered from a couple of issues that have now been resolved.
The basemap.at project provides extensive documentation on how to use the dataset in QGIS and other GIS, including manuals and sample projects:
Here’s the basic configuration: make sure to set the max zoom level to 16, otherwise, the map will not be rendered when you zoom in too far.
The level of detail is pretty impressive, even if it cannot quite keep up with the basemap raster tiles:
Vector tile details at Resselpark, ViennaRaster basemap details at Resselpark, Vienna
Building on its predecessor, QField 2.2 continues to improve its navigation functionality with a brand new precise view (i.e. stakeout) panel as well as adding the capability to cycle through feature vertices to set a destination.
We would like to thank the developers, documenters, testers and all the many folks out there who volunteer their time and effort (or fund people to do so). From the QGIS community we hope you enjoy this release! If you wish to donate time, money or otherwise get involved in making QGIS more awesome, please wander along to qgis.org and lend a hand!
QGIS is supported by donors and sustaining members. A current list of donors who have made financial contributions large and small to the project can be seen on our donors list. If you would like to become a sustaining member, please visit our page for sustaining members for details. Your support helps us fund our six monthly developer meetings, maintain project infrastructure and fund bug fixing efforts.
QGIS is Free software and you are under no obligation to pay anything to use it – in fact we want to encourage people far and wide to use it regardless of what your financial or social status is – we believe empowering people with spatial decision making tools will result in a better society for all of humanity.
In Summer 2022 the Swiss QGIS User community finally got together physically again to meet at the University of Berne, after 3 years of online meetings. Up to 90 QGIS users and contributors out of academia and engineering enjoyed and discussed the newest QGIS features and use cases.
After a warm welcome and introduction by Isabel Kiefer from OPENGIS.ch the presentations started.
QGIS Update
Marco Bernasocchi (OPENGIS.ch CEO and Qgis.org Chair) presented recent QGIS features out of the changelogs of current long term release 3.22, followed by versions 3.24 and 3.26. Among the enhancements are the new curve conversion vertex tool and improvements to the mesh editing, 3D-mode, WMS server and SQL logging, to name a few.
QGIS Animation Workbench
The real world is not static. Thus, often information can be understood more easily in animated form, like visualizing traffic on a map with moving vehicles. QGIS now supports dynamic renderings with the Animation Workbench Plugin. Tim Sutton (Kartoza) led through a Youtube Video showing the underlying mechanisms of the plugin and how to use it.
QGIS Model Baker Update
Starting with the new logo, Romedi Filli (GIS-Fachstelle, Kt. Schaffhausen) showed the latest improvements to the QGIS Model Baker plugin. Especially the data validator and UsabILIty Hub make QGIS project generation out of Interlis data even easier. Even more there is now a python package for those who prefer to python script it all together.
Using QGIS Model Baker for OEREB Cadastre
Adrian Weber (Dütschler + Partner) followed up presenting the use of QGIS Model Baker in migrating their management of municipal usage plans from proprietary software to QGIS driven workflow. Though they see the potential in this approach, they lack time and money, thus doing it step by step. In providing this public service the difficulty is that data is legally binding and system components need to meet this requirements.
Dynamic Forms and Widgets with QGIS Expressions
After a coffee break, Andreas Neumann (Amt für Geoinformation, Kt. Solothurn) gave an interesting technical speech on more dynamic QGIS forms and widgets. Form values can now be defined via expressions so they update automatically depending on other form values. Furthermore, action buttons can be included into forms and defined which can call external web-services, data dependent constraints be defined and more.
Analysis of Flight Trajectories
Driven by technical ambition and will to set some factual basis for political discussion, Yvo Weidmann (Geoidee) carried out a sophisticated analysis of descents to Zurich airport based on Open Source Flight Trajectories and swisstopo data. Therefore he processed data from opensky-network.org, the Aeronautical Information Publication after a lot of initial data validation and cleaning. He finally visualized the results in a nice QGIS driven animation of flight descents.
Teksi utilities application modules
Alexandre Bosshard (Ville de Pully) presented TEKSI, an association that has set itself the task of providing the operators of public infrastructure with decision-making support in the form of professional modules for controlling their activities, namely QGEP and QWAT for the moment with more to follow. Therefore they work on open-source software built mainly on top of QGIS and PostgresSQL/PostGIS.
QGEP (by Teksi) and hydraulic analysis with SWMM
Timothée Produit (Alpnetsystem SA (IG-Group)) gave a more technical speech on their approach of managing a central database to serve both, Teksi’s waste water management tool and QGIS extension QGEP and Stormwater Management Software SWMM to carry out hydraulic analysis in Swiss Romandie. He showed the necessary database and infrastructure setup and workflow steps to create the desired product.
The new Profile Tool in QGIS Core
Nyall Dawson (North Road) led through his Youtube video about QGIS project terrain settings and how they interact with 3d maps and the new elevation profile tool, which is only possible from version 3.26. Powerfull new possibilities to process and visualize elevation and 3D geodata worth watching. Nyall joined the conference virtually after the video to answer questions from the impressed audience.
Cool Maps made with QGIS
Finally Marco Bernasocchi closed the presentations with a collection of incredibly creative QGIS results including Xmas wishes, sports statistics and human facial topology
Workshops
After a tasty lunch including a delicious cheese selection and fruitful conversations the lucky subscribers where asked to take action themselves in the four afternoon workshops. Among other interesting topics the users could get hands on working with QField and QFieldCloud or could get started with QGIS Model Baker and data validation, all tought by the experts and developers of OPENGIS.ch.
Speed and direction column names can now be customized
If you have questions about using MovingPandas or just want to discuss new ideas, you’re welcome to join our recently opened discussion forum.
As always, all tutorials are available from the movingpandas-examples repository and on MyBinder:
Besides others examples, the movingpandas-examples repo contains the following tech demo: an interactive app built with Panel that demonstrates different MovingPandas stop detection parameters
To start the app, open the stopdetection-app.ipynb notebook and press the green Panel button in the Jupyter Lab toolbar:
Here’s a quick preview of the resulting app in action:
To create this app, I defined a single function called my_plot which takes the address and desired buffer size as input parameters. Using Panel’s interact and servable methods, I’m then turning this function into the interactive app you’ve seen above:
To open the Panel preview, press the green Panel button in the Jupyter Lab toolbar:
I really enjoy building spatial data exploration apps this way, because I can start off with a Jupyter notebook and – once I’m happy with the functionality – turn it into a pretty app that provides a user-friendly exterior and hides the underlying complexity that might scare away stakeholders.
Give it a try and share your own adventures. I’d love to see what you come up with.
Understanding which regions QGIS is being used in, which versions are in active use, which platforms it is being used on, and how many users we have is hugely beneficial to our ability as a project to serve our users. Back in 2017 at the bi-annual QGIS hackfest in Nødebo, Denmark, we had a long discussion about key project goals and the need to better understand our user base in order to plan the future direction of the project, and allocate funding and resources to where they are needed most
Typically proprietary software vendors have ready access to detailed user data through telemetry code which they embed in their software. This telemetry code ‘phones home’ key metrics, which together with other techniques such as license sales analysis gives them a very detailed insight into their user base. The data these vendors collect is typically not shared, so their users do not benefit from being able to understand how their data is used.
For QGIS.org, having to resort to what are generally considered to be nefarious and privacy-invading techniques of siphoning user data from our users goes against the ethos we try to promote as an open project. Further, since QGIS is freely available and doesn’t require any self-registration, we do not have a user database we can consult for such analytics. Additional factors make understanding usage levels hard. For example, a single user can download a copy of a QGIS installer and distribute it to many other users, and conversely web crawlers and bots can download many copies of QGIS installers and never install them. Because of this, simply counting the number of downloads from our website does not give a useful picture of our user base.
So we needed to come up with an approach that:
Does not invade our user’s privacy
Does not require including telemetry code in QGIS which exfiltrates user information from their system
Does not store any user-identifiable data on our servers
Is open and transparent in the data collection methodology
Openly shares the insights we gain from our analytics to the broader community
The most obvious privacy-respecting way we could find to understand more about our users was to collect metrics of access to the QGIS News Feed. In order to display the latest news on startup, QGIS Desktop makes a request to https://feed.qgis.org when it is opened. On the server that hosts the feed, we can then use the web server logs to understand which operating system and version of QGIS made the news feed request. Additionally, using the GeoIP library we can resolve each request to the country from which it originated. These pieces of information are included in the User-Agent headers sent by QGIS when it makes a request to the QGIS News Feed.
This process is anonymous, transparent, and simple to disable. It does not identify unique machines. Only one event is logged per unique network per hour. Only one event is logged per QGIS installation per day, and the event is only triggered when the user opens the QGIS Desktop application.
Operating system statistics are derived from QGIS version information, and no system fingerprinting or telemetry is implemented.
Location information is derived from the request source IP address, which is immediately discarded on the server after resolving it to the country of origin.
No logging on the QGIS News Feed server occurs with legacy installations that do not have the news feed feature, offline usage of QGIS, and installations for which feed collection is disabled (see below for info on how to disable it). It will also have statistics skewed in scenarios where atypical networking infrastructure is in effect, such as using a virtual private network.
Despite these caveats, the statistics should provide a good high-level overview of how QGIS is being used, such as the breakdown of QGIS across operating systems and versions – information that is incredibly useful to the QGIS developer team. Only the following four pieces of information are collected:
The date (aggregated by day)
The QGIS version
The Operating System
Country (based on IP which is immediately discarded)
Opting out
If you wish to opt-out of this data collection, simply disabling the feed retrieval, using QGIS offline, or blocking access to the QGIS RSS feed address (feed.qgis.org) on your network will exclude you from this process. QGIS Desktop provides options for disabling version checking and feed access under Settings ➔ Options ➔ General ➔ Application. Note that by default this setting is specific to each individual user profile.
Viewing the analytics
We have made a public dashboard publicly available at https://analytics.qgis.org. The dashboard was made using the fantastic open-source Metabase analytics package.
Credits: This post was written by Charles Dixon-Paver and Tim Sutton
With the new integrated profile tool, you can generate cross sections of point clouds, raster, vector and mesh data. For more information on this tool, you can see the excellent video introduction by North Road who implemented this part of the project.
To be able to view profiles from different data types, there is now a dedicated Elevation settings under layer properties. Users can set the elevation source, style and some other configurations. You can then enable elevation profile widget window by going to the main menu in QGIS, View > Elevation Profile.
Support for COPC
Cloud Optimized Point Cloud (COPC) is a new format for point cloud data and QGIS 3.26 comes with support for it (for both local files and data hosted on remote servers).
COPC is a very exciting addition to the ecosystem, because it is “just” a LAZ file (a format well established in the industry) that brings some interesting extra features. This means all software supporting LAZ file format will also be able to read COPC files without any extra development. If you are familiar with Cloud Optimized GeoTIFF (COG) for rasters, COPC is an extension of the same concept for point cloud data. Read more at https://copc.io/
Ordinary LAS/LAZ files have an issue that it is not possible to efficiently read a subset of data without reading the entire file. This is less of an issue when processing point cloud data, but much more important for point cloud viewers, which typically show only a small portion of the data (e.g. zoomed in to a particular object or zoomed out to show the entire dataset). For that reason, viewers need to index (pre-process) the data before being able to show it - QGIS also needs to do the indexing when a point cloud file is first loaded. The new feature that COPC brings is that data is re-organized in a way that reading just some parts of data is efficient and easy. Therefore when loading COPC files, QGIS can immediately show them without any indexing (that takes time and extra storage).
In addition to that, COPC files can be efficiently used also directly from remote servers - clients such as QGIS can only request small portions of data needed, without the need to download the entire file (that can have size of many gigabytes). This makes dissemination of point cloud data easier than before - just make COPC files available through a static server and clients are ready to stream the data.
A small note: until now, QGIS indexed point cloud files to EPT format upon first load. From QGIS 3.26 we have switched to indexing to COPC - it has the advantage of being just a single file rather than lots of small files in a directory. If you have point cloud data indexed in EPT format already, QGIS will keep using EPT index (rather than indexing also to COPC).
Classified renderer improvements
Classified renderer for point clouds has been improved to:
Show only classes that are in the dataset (instead of hard-coded list) & show also non-standard classes
Show percentage of points for each class
Work also for other attributes (return number, number of returns, point source and few other classes)
Vector transparency in 3D scene
This improvement is not part of the crowdfunding campaign and was exclusively funded by the Swedish QGIS user group, but it is somehow relevant to the audience of this blog post!
With this feature, you can set polygon transparency in 3D scenes.
Want to see more features?
We are trying to improve QGIS to handle point clouds for visualisation and analysis. If you would like certain features to be added to QGIS, do not hesitate to contact us on [email protected] with your idea(s).
With the new integrated profile tool, you can generate cross sections of point clouds, raster, vector and mesh data. For more information on this tool, you can see the excellent video introduction by North Road who implemented this part of the project.
To be able to view profiles from different data types, there is now a dedicated Elevation settings under layer properties. Users can set the elevation source, style and some other configurations. You can then enable elevation profile widget window by going to the main menu in QGIS, View > Elevation Profile.
Support for COPC
Cloud Optimized Point Cloud (COPC) is a new format for point cloud data and QGIS 3.26 comes with support for it (for both local files and data hosted on remote servers).
COPC is a very exciting addition to the ecosystem, because it is “just” a LAZ file (a format well established in the industry) that brings some interesting extra features. This means all software supporting LAZ file format will also be able to read COPC files without any extra development. If you are familiar with Cloud Optimized GeoTIFF (COG) for rasters, COPC is an extension of the same concept for point cloud data. Read more at https://copc.io/
Ordinary LAS/LAZ files have an issue that it is not possible to efficiently read a subset of data without reading the entire file. This is less of an issue when processing point cloud data, but much more important for point cloud viewers, which typically show only a small portion of the data (e.g. zoomed in to a particular object or zoomed out to show the entire dataset). For that reason, viewers need to index (pre-process) the data before being able to show it - QGIS also needs to do the indexing when a point cloud file is first loaded. The new feature that COPC brings is that data is re-organized in a way that reading just some parts of data is efficient and easy. Therefore when loading COPC files, QGIS can immediately show them without any indexing (that takes time and extra storage).
In addition to that, COPC files can be efficiently used also directly from remote servers - clients such as QGIS can only request small portions of data needed, without the need to download the entire file (that can have size of many gigabytes). This makes dissemination of point cloud data easier than before - just make COPC files available through a static server and clients are ready to stream the data.
A small note: until now, QGIS indexed point cloud files to EPT format upon first load. From QGIS 3.26 we have switched to indexing to COPC - it has the advantage of being just a single file rather than lots of small files in a directory. If you have point cloud data indexed in EPT format already, QGIS will keep using EPT index (rather than indexing also to COPC).
Classified renderer improvements
Classified renderer for point clouds has been improved to:
Show only classes that are in the dataset (instead of hard-coded list) & show also non-standard classes
Show percentage of points for each class
Work also for other attributes (return number, number of returns, point source and few other classes)
Vector transparency in 3D scene
This improvement is not part of the crowdfunding campaign and was exclusively funded by the Swedish QGIS user group, but it is somehow relevant to the audience of this blog post!
With this feature, you can set polygon transparency in 3D scenes.
Want to see more features?
We are trying to improve QGIS to handle point clouds for visualisation and analysis. If you would like certain features to be added to QGIS, do not hesitate to contact us on [email protected] with your idea(s).
The 8.2.0 release of GRASS GIS is now available with results from the GSoC 2021 and many other additions. A new grass.jupyter package is now included for interacting with Jupyter notebooks. Single window graphical user interface is available in GUI settings. r.series and three other modules are newly parallelized. Additionally, the release includes a series […]
