The GRASS GIS community recognises the long-term contributions of Roger Bivand for the development of the rgrass package.

The post A big thank you to Roger Bivand! appeared first on Markus Neteler Consulting.

Explore Mergin Maps' 2024 highlights: app redesigns, new features like Editor permissions, and a preview of what's coming in 2025!

Today, I want to point out a blog post over at

https://carto.com/blog/urban-mobility-insights-with-movingpandas-carto-in-snowflake

written together with my fellow co-authors and EMERALDS project team member Argyrios Kyrgiazos.

For the technically inclined, the highlight are the presented UDFs in Snowflake to process and transform the trajectory data. For example, here’s a TemporalSplitter UDF:

CREATE OR REPLACE FUNCTION CARTO_DATABASE.CARTO.TemporalSplitter(geom ARRAY, t ARRAY, mode STRING)
RETURNS ARRAY
LANGUAGE PYTHON
RUNTIME_VERSION = 3.11
PACKAGES = ('numpy','pandas', 'geopandas','movingpandas', 'shapely')
HANDLER = 'udf'
AS $$
import numpy as np
import pandas as pd
import geopandas as gpd
import movingpandas as mpd
import shapely
from shapely.geometry import shape, mapping, Point, Polygon
from shapely.validation import make_valid
from datetime import datetime, timedelta

def udf(geom, t, mode):
    valid_df = pd.DataFrame(geom, columns=['geometry'])
    valid_df['t'] = pd.to_datetime(t)
    valid_df['geometry'] = valid_df['geometry'].apply(lambda x:shapely.wkt.loads(x))
    gdf = gpd.GeoDataFrame(valid_df, geometry='geometry', crs='epsg:4326')
    gdf = gdf.set_index('t')
    traj = mpd.Trajectory(gdf, 1)
    traj_sm = mpd.TemporalSplitter(traj).split(mode=mode)
    if len(traj_sm.trajectories)>0:
        res = traj_sm.to_point_gdf()
        res['geometry'] = res['geometry'].apply(lambda x: shapely.wkt.dumps(x))
        return res.reset_index().values
    else:
        return []
$$;

You can find the full code here: https://github.com/anitagraser/carto-research-public/tree/master/movingpandas_carto_in_snowflake

Discover how the MOSPREMA project, launched in 2022 in Czechia, uses open-source tools like Mergin Maps and PostgreSQL to monitor and manage mosquito populations. Learn how real-time data, sensors, and mapping technology improve efficiency for local municipalities.

November was a really productive month, with a remarkable total of 43 new plugins published in QGIS plugin repository. In addition there are 3 more plugins from October listed here, which somehow were missed, and for that we apologize to their authors.

Here follows the quick overview in reverse chronological order. If any of the names or short descriptions catches your attention, you can find the direct link to the plugin page in the table below:

FOSS4G is the annual global event of free and open source geographic technologies and open geospatial data hosted by OSGeo. In 2024 it took place in Belém, Brasil.

What's Changed

• Fix crash on older devices

What's Changed

• Fix sketching over image attachments sometimes fails to save
• Fix import URL eating tons of memory on large file downloads
• Avoid infinite sort loop when gathering feature lists for the value relation editor widgets

tldr; Tired of working with large CSV files? Give GeoParquet a try!

“Parquet is a powerful column-oriented data format, built from the ground up to as a modern alternative to CSV files.” https://geoparquet.org/

(Geo)Parquet is both smaller and faster than CSV. Additionally, (Geo)Parquet columns are typed. Text, numeric values, dates, geometries retain their data types. GeoParquet also stores CRS information and support in GIS solutions is growing.

I’ll be giving a quick overview using AIS data in GeoPandas 1.0.1 (with pyarrow) and QGIS 3.38 (with GDAL 3.9.2).

File size

The example AIS dataset for this demo contains ~10 million rows with 22 columns. I’ve converted the original zipped CSV into GeoPackage and GeoParquet using GeoPandas to illustrate the huge difference in file size: ~470 MB for GeoParquet and zipped CSV, 1.6 GB for CSV, and a whopping 2.6 GB for GeoPackage:

Reading performance

Pandas and GeoPandas both support selective reading of files, i.e. we can specify the specific columns to be loaded. This does speed up reading, even from CSV files:

	Whole file	Selected columns
CSV	27.9 s	13.1 s
Geopackage	2min 12s	20.2 s
GeoParquet	7.2 s	4.1 s

Indeed, reading the whole GeoPackage is getting quite painful.

Here’s the code I used for timing the read times:

As you can see, these times include the creation of the GeoPandas.GeoDataFrame.

If we don’t need a GeoDataFrame, we can read the files even faster:

Non-spatial DataFrames

GeoParquet files can be read by non-GIS tools, such as Pandas. This makes it easier to collaborate with people who may not be familiar with geospatial data stacks.

And reading plain DataFrames is much faster than creating GeoDataFrames:

But back to GIS …

GeoParquet in QGIS

In QGIS, GeoParquet files can be loaded like any other vector layer, thanks to GDAL:

Loading the GeoParquet and GeoPackage files is pretty quick, especially if we zoom into a small region of interest (even though, unfortunately, it doesn’t seem possible to restrict the columns to further speed up loading). Loading the CSV, however, is pretty painful due to the lack of spatial indexing, which becomes apparent very quickly in the direct comparison:

As far as I can tell, my QGIS 3.38 ‘Grenoble’ does not support writing to or editing of GeoParquet files. So I’m limited to reading GeoParquet for now.

However, seeing how much smaller GeoParquets are compared to GeoPackages (and also faster to write), I hope that we will soon get the option to export to GeoParquet.

For now, I’ll start by converting my large CSV files to GeoParquet using GeoPandas.

More reading

If you’re into GeoJSON and/or PyGeoAPI, check out Joana Simoes’ post: “Navigating GeoParquet: Lessons Learned from the eMOTIONAL Cities Project”

And if you want to see a global dataset example, have a look at Matt Travis’ presentation using Overture data:

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.

What's Changed

• Fix user interface language not respecting system locale when available
• Work around an upstream Qt crash when a value relations editor combo box pick leads to its container to be hidden
• Fix cloud attachments not uploaded immediately when hitting the synchronize button
• Fix layer login popup getting into the top / bottom system margins
• Fix initiating a tracking session with a feature form containing relations
• Fix geometry digitizing step missing when adding a relationship parent via the relationship combo box
• To avoid interference with snapping results when creating features using a mouse/stylus, do not reset coordinate cursor when rubber band is frozen

Discover how Camptocamp, in collaboration with Zurich, used Mergin Maps and QGIS to map bird nesting sites, empowering volunteers to collect accurate survey data.

In the last month of October, 18 new plugins were published in the QGIS plugin repository.

Here follows the quick overview in reverse chronological order. If any of the names or short descriptions catches your attention, you can find the direct link to the plugin page in the table below:

It’s been a while since my post on geo and the AI hype in 2019. Back then, I didn’t use the term “GeoAI”, even though it has certainly been around for a while (including, e.g., with dedicated SIGSPATIAL workshops since 2017).

GeoAI isn’t one single thing. It’s an umbrella term, including: “AI for Geo” (using AI methods in Geo, e.g. deep learning for object recognition in remote sensing images) and “Geo for AI” (integrating geographic concepts into AI models, e.g. by building spatially explicit models). [Zhang 2020] [Li et al. 2024]

Today’s post is a collection of key GeoAI developments I’m aware of. If I missed anything you are excited about, please let me know here in the comments or over on Mastodon.

Background

A week ago, I had the pleasure to attend a “Specialist Meeting” on GeoAI here in Vienna, meeting over 40 researchers from around the world, from Master students to professor emeritus. Huge props to Jano (Prof. Krzysztof Janowicz) and his team at Uni Wien for bringing this awesome group of people together.

The elephant in the room: LLMs

Unsurprisingly, LLMs and the claims they make about geography are a mayor issue due to the mistakes they make and the biases behind them. An infamous example is AI’s issue with understanding topology:

Even if recent versions of ChatGPT (such as GTP 4o) do a better job with this specific example, this doesn’t make their answers reliable. So between the trustworthiness, reproducibility, explainability, and sustainability issues … LLMs have a long way to go. And it’s not clear whether they are going in the right direction right now.

Geospatial foundation models

Prithvi, a model developed by NASA, IBM, et al. in 2023, is one of the first geospatial foundation models. Like much of GeoAI, Prithvi deals with remote sensing data. Specifically, it is trained on Landsat and Sentinel-2 (HLS) imagery, with applications in flood mapping and wildfire prediction. And maybe best of all: the model is open-source and publicly available.

Spatiotemporal machine learning model specifications

In the general AI community, model cards have become a common way to share information about models. However, identifying the right model for spatiotemporal tasks is hard since there are no standardized descriptions in existing model catalogs (e.g. Hugging Face, DLHub or MLFlow). To address this issue, [Charette-Migneault et al. 2024] have proposed the Machine Learning Model (MLM) extension for the SpatioTemporal Asset Catalogs (STAC). But, yet again, this development is targeting models trained with remote sensing imagery.

Similarly, the OGC Training Data Markup Language for Artificial Intelligence (TrainingDML-AI) and its ISO equivalent are limited to EO as well …

Spatial knowledge graphs

For those among us working mostly with vector data, the KnowWhereGraph is an interesting development. It’s the first geo-enriched knowledge graph [Janowicz et al. 2022] that helps answer geospatial questions by integrating a variety of spatial datasets through hierarchical grids, standard region boundaries and appropriate ontology and knowledge graph schema development. However, so far, the KnowWhereGraph is mostly limited to the United States.

Explainable AI (XAI) and geo

While answers from knowledge graphs are intrinsically explainable, many other (Geo)AI solutions are built on AI approaches that result in black box models.

Graph neural networks (GNNs) have become very popular in GeoAI (including in urban analytics and mobility [Jalali et al. 2023] [Liu et al. 2024]) but their black box nature limits their practical usefulness in domains where transparency and trustworthiness are crucial. To offer insights into how model predictions are made, [Liu et al. 2024] propose a spatially explicit GeoAI-based method that combines a graph convolutional network and a graph-based XAI method, called GNNExplainer to explore the correlation between urban objects.

Reproducibility et al.

The AI hype in geo is still going strong. Journals are being flooded with paper submissions and good reviewers are hard to come by. In many geo-related venues, it is still acceptable to present an AI paper without making code or model available. (We recently discussed this issue for mobility AI specifically [Graser et al. 2024].)

I’m convinced we can and should do better: quality over quantity, moving steadily, building and fixing things.

References

• Charette-Migneault, F., Avery, R., Pondi, B., Omojola, J., Vaccari, S., Membari, P., … & Sundwall, J. (2024). Machine Learning Model Specification for Cataloging Spatio-Temporal Models. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Searching and Mining Large Collections of Geospatial Data (pp. 36-39).
• Graser, A., Jalali, A., Lampert, J., Weißenfeld, A., & Janowicz, K. (2024). MobilityDL: a review of deep learning from trajectory data. GeoInformatica, 1-33.
• Jakubik, J., Roy, S., Phillips, C., Fraccaro, P., Godwin, D., Zadrozny, B., … & Edwards, B. (2023) Foundation models for generalist geospatial artificial intelligence. arXiv preprint arXiv:2310.18660.
• Jalali, A., Graser, A., & Heistracher, C. (2023). Towards eXplainable AI for Mobility Data Science. arXiv preprint arXiv:2307.08461.
• Janowicz, K., Hitzler, P., Li, W., Rehberger, D., Schildhauer, M., Zhu, R., … & Currier, K. (2022). Know, Know Where, KnowWhereGraph: A densely connected, cross-domain knowledge graph and geo-enrichment service stack for applications in environmental intelligence. AI Magazine, 43(1), 30-39.
• Li, W., Arundel, S., Gao, S., Goodchild, M., Hu, Y., Wang, S., & Zipf, A. (2024). GeoAI for Science and the Science of GeoAI. Journal of Spatial Information Science, (29), 1-17.
• Liu, P., Zhang, Y., & Biljecki, F. (2024). Explainable spatially explicit geospatial artificial intelligence in urban analytics. Environment and Planning B: Urban Analytics and City Science, 51(5), 1104-1123.
• OGC Training Data Markup Language for Artificial Intelligence (TrainingDML-AI) Part 1: Conceptual Model Standard (2023)
• Zhang, L. (2020). Explainable Machine Learning for Activity Modeling in GeoAI. Doctoral dissertation, George Mason University.

We’ve added measurement tools to Mergin Maps, allowing you to measure both distance and area directly within the app. The new "Measure" action is accessible from the "more" menu, where you can easily add points to calculate lengths and areas in your projects. The process is similar to creating a line feature, with a live display of the measured distance and options to undo, add points, or complete the measurement. Measurements are following your QGIS distance and area project units.

Read more about the measurement tool in our documentation: https://merginmaps.com/docs/field/measure/

What's Changed

• Improve handling of captured photo's ratio and resolution
• Improve feature attribute pasting
• Optimize addition of child feature in a parent feature being edited
• Guard from potential crash when loading a project that requires authentication credentials
• Fix large files constantly being re-downloaded when synchronizing cloud projects

How to contribute to GRASS GIS development: Guidance for new developers in the GRASS GIS Project.

The post How to contribute to GRASS GIS development appeared first on Markus Neteler Consulting.

You can now preview your project data directly from the project details page—no need to open QGIS or the mobile app!

Maps in dashboard are currently in beta, and we’re more than happy to hear your feedback. Share your thoughts and help us improve it here: https://wishlist.merginmaps.com/p/let-s-get-maps-in-dashboard-out-of-beta.

Sorry, this entry is only available in French.