The latest version of QField is out, featuring as its main new feature sensor handling alongside the usual round of user experience and stability improvements. We simply can’t wait to see the sensor uses you will come up with!

The main highlight: sensors

QField 2.8 ships with out-of-the-box handling of external sensor streams over TCP, UDP, and serial port. The functionality allows for data captured through instruments – such as geiger counter, decibel sensor, CO detector, etc. – to be visualized and manipulated within QField itself.

Things get really interesting when sensor data is utilized as default values alongside positioning during the digitizing of features. You are always one tap away from adding a point locked onto your current position with spatially paired sensor readings saved as point attribute(s).

Not wowed yet? Try pairing sensor readings with QField’s tracking capability! Head over QField’s documentation on this as well as QGIS’ section on sensor management to know more.

The development of this feature involved the addition of a sensor framework in upstream QGIS which will be available by the end of this coming June as part of the 3.32 release. This is a great example of the synergy between QField and its big brother QGIS, whereas development of new functionality often benefits the broader QGIS community. Big thanks to Sevenson Environmental Services for sponsoring this exciting capability.

Notable improvements

A couple of refinements during this development cycle are worth mentioning. If you ever wished for QField to directly open a selected project or reloading the last session on app launch, you’ll be happy to know this is now possible.

For heavy users of value relations in their feature forms, QField is now a tiny bit more clever when displaying string searches against long lists, placing hits that begin with the matched string first as well as visually highlighting matches within the result list itself.

Finally, feature lists throughout QField are now sorted. By default, it will sort by the display field or expression defined for each vector layer, unless an advanced sorting has been defined in a given vector layer’s attribute table. It makes browsing through lists feel that much more natural.

Changes

🐛 Bug Fixes

• Insure layer visibility is always properly reflected in the legend widget
• Fix checkbox editor widget not reflecting feature values
• Fix default value when tracking against point layers
• Fix HTML code in HTML-enabled multiline text editor widget

Did you know that MovingPandas also supports local image coordinates? Indeed, it does.

In today’s post, we will explore how we can use this feature to analyze bicycle tracks extracted from video footage published by Michael Szell @mszll:

• Dataset: https://zenodo.org/record/7288616
• Data description: https://arxiv.org/abs/2211.01301

The bicycle trajectory coordinates are stored in two separate lists: xs_640x360 and ys640x360:

This format is kind of similar to the Kaggle Taxi dataset, we worked with in the previous post. However, to use the solution we implemented there, we need to combine the x and y coordinates into nice (x,y) tuples:

df['coordinates'] = df.apply(
    lambda row: list(zip(row['xs_640x360'], row['ys_640x360'])), axis=1)
df.drop(columns=['xs_640x360', 'ys_640x360'], inplace=True)

Afterwards, we can create the points and compute the proper timestamps from the frame numbers:

def compute_datetime(row):
    # some educated guessing going on here: the paper states that the video covers 2021-06-09 07:00-08:00
    d = datetime(2021,6,9,7,0,0) + (row['frame_in'] + row['running_number']) * timedelta(seconds=2)
    return d
def create_point(xy):
    try: 
        return Point(xy)
    except TypeError:  # when there are nan values in the input data
        return None
new_df = df.head().explode('coordinates')
new_df['geometry'] = new_df['coordinates'].apply(create_point)
new_df['running_number'] = new_df.groupby('id').cumcount()
new_df['datetime'] = new_df.apply(compute_datetime, axis=1)
new_df.drop(columns=['coordinates', 'frame_in', 'running_number'], inplace=True)
new_df

Once the points and timestamps are ready, we can create the MovingPandas TrajectoryCollection. Note how we explicitly state that there is no CRS for this dataset (crs=None):

trajs = mpd.TrajectoryCollection(
    gpd.GeoDataFrame(new_df), 
    traj_id_col='id',  t='datetime', crs=None)

Plotting trajectories with image coordinates

Similarly, to plot these trajectories, we should tell hvplot that it should not fetch any background map tiles (’tiles’:None) and that the coordinates are not geographic (‘geo’:False):

If you want to explore the full source code, you can find my Github fork with the Jupyter notebook at: https://github.com/anitagraser/desirelines/blob/main/mpd.ipynb

The repository also contains a camera image of the intersection, which we can use as a background for our trajectory plots:

bg_img = hv.RGB.load_image('img/intersection2.png', bounds=(0,0,640,360)) 

One important caveat is that speed will be calculated in pixels per second. So when we plot the bicycle speed, the segments closer to the camera will appear faster than the segments in the background:

To fix this issue, we would have to correct for the distortions of the camera lens and perspective. I’m sure that there is specialized software for this task but, for the purpose of this post, I’m going to grab the opportunity to finally test out the VectorBender plugin.

Georeferencing the trajectories using QGIS VectorBender plugin

Let’s load the five test trajectories and the camera image to QGIS. To make sure that they align properly, both are set to the same CRS and I’ve created the following basic world file for the camera image:

1
0
0
-1
0
360

Then we can use the VectorBender tools to georeference the trajectories by linking locations from the camera image to locations on aerial images. You can see the whole process in action here:

After around 15 minutes linking control points, VectorBender comes up with the following georeferenced trajectory result:

Not bad for a quick-and-dirty hack. Some points on the borders of the image could not be georeferenced since I wasn’t always able to identify suitable control points at the camera image borders. So it won’t be perfect but should improve speed estimates.

This post is part of a series. Read more about movement data in GIS.

Kaggle’s “Taxi Trajectory Data from ECML/PKDD 15: Taxi Trip Time Prediction (II) Competition” is one of the most used mobility / vehicle trajectory datasets in computer science. However, in contrast to other similar datasets, Kaggle’s taxi trajectories are provided in a format that is not readily usable in MovingPandas since the spatiotemporal information is provided as:

• TIMESTAMP: (integer) Unix Timestamp (in seconds). It identifies the trip’s start;
• POLYLINE: (String): It contains a list of GPS coordinates (i.e. WGS84 format) mapped as a string. The beginning and the end of the string are identified with brackets (i.e. [ and ], respectively). Each pair of coordinates is also identified by the same brackets as [LONGITUDE, LATITUDE]. This list contains one pair of coordinates for each 15 seconds of trip. The last list item corresponds to the trip’s destination while the first one represents its start;

Therefore, we need to create a DataFrame with one point + timestamp per row before we can use MovingPandas to create Trajectories and analyze them.

But first things first. Let’s download the dataset:

import datetime
import pandas as pd
import geopandas as gpd
import movingpandas as mpd
import opendatasets as od
from os.path import exists
from shapely.geometry import Point

input_file_path = 'taxi-trajectory/train.csv'

def get_porto_taxi_from_kaggle():
    if not exists(input_file_path):
        od.download("https://www.kaggle.com/datasets/crailtap/taxi-trajectory")

get_porto_taxi_from_kaggle()
df = pd.read_csv(input_file_path, nrows=10, usecols=['TRIP_ID', 'TAXI_ID', 'TIMESTAMP', 'MISSING_DATA', 'POLYLINE'])
df.POLYLINE = df.POLYLINE.apply(eval)  # string to list
df

And now for the remodelling:

def unixtime_to_datetime(unix_time):
    return datetime.datetime.fromtimestamp(unix_time)

def compute_datetime(row):
    unix_time = row['TIMESTAMP']
    offset = row['running_number'] * datetime.timedelta(seconds=15)
    return unixtime_to_datetime(unix_time) + offset

def create_point(xy):
    try: 
        return Point(xy)
    except TypeError:  # when there are nan values in the input data
        return None
 
new_df = df.explode('POLYLINE')
new_df['geometry'] = new_df['POLYLINE'].apply(create_point)
new_df['running_number'] = new_df.groupby('TRIP_ID').cumcount()
new_df['datetime'] = new_df.apply(compute_datetime, axis=1)
new_df.drop(columns=['POLYLINE', 'TIMESTAMP', 'running_number'], inplace=True)
new_df

And that’s it. Now we can create the trajectories:

trajs = mpd.TrajectoryCollection(
    gpd.GeoDataFrame(new_df, crs=4326), 
    traj_id_col='TRIP_ID', obj_id_col='TAXI_ID', t='datetime')
trajs.hvplot(title='Kaggle Taxi Trajectory Data', tiles='CartoLight')

That’s it. Now our MovingPandas.TrajectoryCollection is ready for further analysis.

By the way, the plot above illustrates a new feature in the recent MovingPandas 0.16 release which, among other features, introduced plots with arrow markers that show the movement direction. Other new features include a completely new custom distance, speed, and acceleration unit support. This means that, for example, instead of always getting speed in meters per second, you can now specify your desired output units, including km/h, mph, or nm/h (knots).

This post is part of a series. Read more about movement data in GIS.

April wasn’t just the month of our wonderful user conference and contributor meeting, it was also a month with a whopping 23 new plugins that have been published in the QGIS plugin repository.

Here’s the quick overview in reverse chronological order. If any of the names or short descriptions piques your interest, you can find the direct link to the plugin page in the table below the screenshot.

Sorry, this entry is only available in Dutch.

After a beautiful map has been created with QGIS and published on https://qgiscloud.com, the desire quickly arises to integrate this map into one’s own website. This way, visitors to the website could access the map directly and without having to go via https://qgiscloud.com. With a little HTML, this is easy to do. But let’s start from the beginning. Make a QGIS Cloud Map As always with QGIS Cloud, a map must first be published.

Sorry… no English text as of yet. However, you’re welcome to have a look at the Dutch version (and use some automagic translation device)…

Last week, we had our 25th Contributor Meeting in ‘s-Hertogenbosch, The Netherlands. Prior to the meeting, the International QGIS User Conference, brought together 200 participants from all around the world who came to learn about QGIS and exchange ideas and best practices.

The conference programme was jam-packed with great content. Over 50 presentations were given during the conference, with the participants enjoying the Dutch atmosphere whilst building up their knowledge of QGIS and sharing their ideas. With more than 20 years of development behind us, QGIS has become a stable platform that supports geospatial decision making, analysis and decision making in all sorts of endeavours. We saw lots of talks that demonstrated this maturity, for example a presentation John Holden and Blake Esselsteyn showed how QGIS is being used to determine voting districts to support the democratic process, Angelina Savchuk showed how QGIS is being used to support the work of the Red Cross, whilst Nick Vervaal showed us how QGIS is used to make high tech artistic laser cut maps from wood.

We were honoured to be joined by Lucho Ferrer who came all the way from Peru! Ujaval Gandhi travelled all the way from India and wowed the audience with his workshop on QGIS Actions.

There are two mobile offerings for QGIS users who want to take their work out to the field. Both QField and MerginMaps had a strong presence at the User Conference, causing a lot of excitement and buzz in the audience.

Contributor Meeting

As is traditional, the contributor meeting (which was held directly after the two day user conference) was full of workshops, sessions, and spontaneous get-togethers to facilitate the exchange between community members, including sessions on:

• Onboarding Day
• Plans for the certification programme
• Getting around in the QGIS community
• Translating QGIS 
• Documentation system and process
• Editing the documentation online on GitHub
• Building documentation on your local machine and using Git to make a pull request (submission)
• Compiling QGIS
• Making your first pull request to QGIS
• Create QGIS issues and feature requests
• OGC API in QGIS
• QGIS vision
• QGIS website update

From the QGIS community we would like to extend a huge ‘thank you!’ to the local organising team who hosted a flawless event and worked so hard to make sure everyone had the best experience possible. Thank you and see you next year!

Changes

🐛 Bug Fixes

• Yet more bug fixing and stability improvements.

Changes

🐛 Bug Fixes

• More bug fixing and stability improvements.

Changes

🐛 Bug Fixes

• Many, many fixes.

Dear QGIS Community,

We are very pleased to announce that this year’s round of grants is now available. The call is open to anybody who wants to make a contribution to QGIS funded by our grant fund, subject to the call conditions outlined in the application form.

The deadline for this round is in four weeks, on 2nd May 2023.

There are no new procedures in 2023. Please note the following guidelines established in previous years: 

• The proposal must be submitted as a ‘QEP’ (QGIS Enhancement Proposal) issue in the repo: https://github.com/qgis/QGIS-Enhancement-Proposals (tagged as Grant-YEAR). Following this approach will allow people to ask questions and provide public feedback on individual proposals.
• Proposals must clearly define the expected final result so that we can properly assess if the goal of the proposal has been reached.
• The project budgets should account for PR reviewing expenses to ensure timely handling of the project-related PRs and avoid delays caused by relying on reviewer volunteer time. 
• In the week after the QEP discussion period, the proposal authors are expected to write a short summary of the discussion that is suitable for use as a basis on which voting members make their decisions. 

The PSC of QGIS.ORG will examine the proposals and has veto power in case a proposal does not follow guidelines or is not in line with project priorities.

For more details, please read the introduction provided in the application form.

We look forward to seeing all your great ideas for improving QGIS!

A brand new version of QField has been released, packed with features that will make you fall in love with this essential open source tool all over again with a focus on capturing more while you are in the field. QField 2.7 nicknamed “Heroic Hedgehog” also includes a number of worthy fixes making it a crucial update to get.

New recording capabilities

The highlight of QField 2.7 is the new audio and video recording capability straight from the feature form. In addition to preexisting still photo capture, this functionality allows for video motion and audio clips to be added as attachments to feature attributes.

The audio recording capability can come in handy in the field when typing on a keyboard-less device can be challenging. Simply record an audio note of observations to process later.

The experience wouldn’t be complete without audio and video playback support, which we took care of in this version too. Playback of such media content within the feature form gives an immediate feedback and saves time. For those interested in full screen immersion, simply click on the video frame to open the attached in your favorite media player. We also took the opportunity to implement audio and video playback on QGIS so people can easily consume the fruits of their labor in the field at their workstation.

We would be remiss if we didn’t mention map canvas rotation functionality added in this version. This is a long-requested functionality which we are happy to have packed into QField now. Pro-tip: when positioning is enabled, double tapping on the lower-left positioning button will have the map canvas follow both the device’s current location as well as the compass orientation.

Finally – some would argue “most importantly” – QField is now equipped with a beautiful dark theme which users can activate in the settings panel. By default on Android and iOS, QField will follow the system’s dark theme setting. In addition to the new color scheme, users can also adjust the user interface font size.

Big thanks to Deutsches Archäologisches Institut who funded the majority of the new features in this release cycle. Their investment in making QField the perfect tool for them has benefited the community as a whole.

A ton of bug fixing across all platforms

Important stability improvements and fixes to serious issues are also part of this release. Noteworthy fixes include WFS layer support on iOS, much better Bluetooth connectivity on Android, and vertical grid improvement on Windows.

For users facing reliability issues with the native camera on Android, we have spent time supersizing the camera we ship within QField itself. During this development cycle, it has gained zoom and flash controls, as well as a ton of usability improvements, including geo-tagging.

To know more about this release, read the full change log on QField’s github release page.

On January 16th, we launched the largest crowd-funding call for QGIS.ORG so far. Our goal was to increase sustaining membership contributions and bring the total member contributions up to €200k. And today, we are happy to announce that, thanks to all our new and returning sustaining members, we have succeeded and even exceeded our goal: raising a total of €206.5k in contributions for 2023.

In total, 42 new sustaining members have answered our call – these are in addition to already existing sustaining members, who started their memberships in previous years – some of them already many years in a row.

We are particularly happy that we are able to welcome our new and first Flagship sustaining member Felt.

A huge thank you also goes to our returning sustaining member, the Office of Public Works (Ireland) who have increased their contribution to the large membership level.

We are also delighted to welcome seven new medium sustaining members, including the first two university institutes supporting us on this level:

• Institut Dr. Nowak GmbH, Germany
• Bohannan Huston, New Mexico, USA
• University of Zurich, Department of Geography, Switzerland
• Idrostudi srl, Italy
• ETH Zurich, Institute of Cartography and Geoinformation, Switzerland
• IC Infraconsult AG, Switzerland
• BBOÖ Breitband Oberösterreich GmbH, Austria

And finally, our thanks go out to the stunning 33 new small sustaining members from all over the world:

• Pacific Geomatics Limited, Canada
• Helix Resources Limited, Australia
• Sand Hill Geographic, Virginia, United States
• GIS Pro Western Australia
• The Spatial Distillery Company, Victoria, Australia
• Qwast-GIS, The Netherlands
• CEICOL, Colombia
• QGIS user group Norway
• Robex resources, Quebec, Canada
• analyGIS GmbH, Switzerland
• CartoExpert, France
• addresscloud, UK
• Baugeologie und Geo-Bau-Labor AG, Switzerland
• Spatial Thoughts LLP, India
• Centremaps, UK
• Geoideal, Colombia
• theworksLA, California, United States
• SoftWater s.r.l., Italy
• menz umweltplanung, Germany
• Oy Arbonaut Ltd, Finland
• ZevRoss Spatial Analysis, New York, United States
• Ecophylla Consulting, Ontario, Canada
• DeBeer&DeVos BV, Belgium
• Reuther NetConsulting, Germany
• H13, Denmark
• Rockwater Pty Ltd, Australia
• BGU Dr. Brehm & Grünz GbR, Germany
• Julie’s Data, Ukraine
• LökPlan Conze & Cordes GbR, Germany
• OSGeo Oceania, Australia
• b-data GmbH, Switzerland
• PMA (Property Market Analysis LLP), UK
• ROBVQ (Regroupement des organismes de bassins versants du Québec), Canada

So February came and went without us sharing our monthly plugin update. That means, this post lists a whopping 32 new plugins that have been published in the QGIS plugin repository in February and March.

Here’s the quick overview in reverse chronological order. If any of the names or short descriptions piques your interest, you can find the direct link to the plugin page in the table below the screenshot.

I’ve previously written about Movement data in GIS and the AI hype and today’s post is a follow-up in which I want to share with you a new review of the state of the art in deep learning from trajectory data.

Our review covers 8 use cases:

1. Location classification
2. Arrival time prediction
3. Traffic flow / activity prediction
4. Trajectory prediction
5. Trajectory classification
6. Next location prediction
7. Anomaly detection
8. Synthetic data generation

We particularly looked into the trajectory data preprocessing steps and the specific movement data representation used as input to train the neutral networks:

On a completely subjective note: the price for most surprising approach goes to natural language processing (NLP) Transfomers for traffic volume prediction.

The paper was presented at BMDA2023 and you can watch the full talk recording here:

References

Graser, A., Jalali, A., Lampert, J., Weißenfeld, A., & Janowicz, K. (2023). Deep Learning From Trajectory Data: a Review of Neural Networks and the Trajectory Data Representations to Train Them. Workshop on Big Mobility Data Analysis BMDA2023 in conjunction with EDBT/ICDT 2023.

This post is part of a series. Read more about movement data in GIS.

Changes

🐛 Bug Fixes

• Significant improvement in Bluetooth connection reliability
• Dark theme tweaks

The QGIS Cloud Support Team is often asked if it is possible to integrate data from external databases into the QGIS Cloud Web Map, so that the map information in the QGIS Cloud Web Map is always up-to-date. In this blog article we would like to show how this is possible with PostgreSQL. To publish data in the QGIS Cloud GDI, the data must always be stored in the personal QGIS Cloud database.

Get your QGIS-ass to ‘s-Hertogenbosch!