The GRASS GIS 8.4.0 release provides more than 520 improvements and fixes with respect to the release 8.3.2.

The post GRASS GIS 8.4.0 released appeared first on Markus Neteler Consulting.

The GRASS GIS 8.4.0RC1 release provides more than 515 improvements and fixes with respect to the release 8.3.2. Please support us in testing this release candidate.

The post GRASS GIS 8.4.0RC1 released appeared first on Markus Neteler Consulting.

This year’s user conference took place in Bratislava, organized by the Slovak QGIS User Group and hosted at the Slovak University of Technology Faculty of Civil Engineering. The event was sold out early and the venue was full of energy and excitement.

If you missed the conference or couldn’t make it to a talk due to too many great overlapping sessions, you now have the opportunity to catch up with everything on the UC 2024 playing list:

Thanks to the organizers, speakers, sponsors, and all participants for the wonderful event.

If you want to read some personal reports from community members, here’s Ujaval Gandhi’s report: https://spatialthoughts.com/2024/09/14/qgis-user-conference-2024; and here’s Hans van der Kwast’s: https://www.qwast-gis.com/l/qgis-user-conference-2024/

Last but not least, we want you to save the date for next year’s user conference:

The QGIS User Conference will be in Norrköping, Sweden from 2-3 June 2025, and contributor meeting there after 4-7.

What's Changed

• Fix variable scoping error with relation editor widgets
• Enable WebSockets support in QField to benefit plugin authors
• Fix regression whereas feature multi-editing can lead to crash
• Some (micro) optimizations of feature form
• Fix crash occurring when the internal receiver is request permission while immediately replaced with external receiver on launch
• Insure that QField toasts appear above the virtual keyboard
• Fix missing + IMU in the quality description
• Allow for users to see and send NMEA logs produced by QField

Visit maps in a project detail to experience improved performance. We're continuously working to make them more efficient and reliable.

Today marks the release of Trajectools 2.3 which brings a new set of algorithms, including trajectory generalizing, cleaning, and smoothing.

To give you a quick impression of what some of these algorithms would be useful for, this post introduces a trajectory preprocessing workflow that is quite general-purpose and can be adapted to many different datasets.

We start out with the Geolife sample dataset which you can find in the Trajectools plugin directory’s sample_data subdirectory. This small dataset includes 5908 points forming 5 trajectories, based on the trajectory_id field:

We first split our trajectories by observation gaps to ensure that there are no large gaps in our trajectories. Let’s make at cut at 15 minutes:

This splits the original 5 trajectories into 11 trajectories:

When we zoom, for example, to the two trajectories in the north western corner, we can see that the trajectories are pretty noisy and there’s even a spike / outlier at the western end:

If we label the points with the corresponding speeds, we can see how unrealistic they are: over 300 km/h!

Let’s remove outliers over 50 km/h:

Better but not perfect:

Let’s smooth the trajectories to get rid of more of the jittering.

(You’ll need to pip/mamba install the optional stonesoup library to get access to this algorithm.)

Depending on the noise values we chose, we get more or less smoothing:

Let’s zoom out to see the whole trajectory again:

Feel free to pan around and check how our preprocessing affected the other trajectories, for example:

Changes

🚀 Features

• A nifty processing toolbox has been added into QField, with countless algorithms available to edit features and geometries while in the field
• Geofencing functionality has landed in QField, allowing users to use vector layers to define geofenced areas to warn users or restrict areas where digitizing can occur
• Project variables are now shown and customizable within the variable editor
• The ratio and resolution settings for photos captured by the QField camera
• Date and positioning details can be imprinted onto photos captured by the QField camera

✨ Improvements

• A visual revamp of the variables panel provides more space for variable content and easier interaction
• QField’s growing documentation is now searchable directly within QField through the search bar, simply type ? followed by keyword(s)
• Users can now flag folders as ‘favorites’ when browsing local datasets and projects
• The cloud projects list now comes with a nifty search bar
• Significant improvements in attachment upload to the QFieldCloud servers (if this was a problem in the past, try again!)
• Improvements to the QField camera geotagging functionality
• On mobile devices, the home screen has a shutdown button to insure authentication details are cleared
• Projects can now have a default active layer set for each map theme

🪟 Windows improvements

• QField is now installable via Microsoft Store

What's Changed

• Add orientation (aka compass direction) detail when watermarking/stamping photos
• Fix typo in stamping toaster notification

In last August, there were 24 new plugins published in the QGIS plugin repository.

Highlight

“Help us create the world’s most advanced open database on litter, brands & plastic pollution.”

This sentence welcomes us to the website of the OpenLitterMap project, which aims at providing tools for any and all citizens to capture data on litter worldwide. This data can now be directly accessed in QGIS for visualisation and analysis purposes, thanks to the efforts of the plugin’s author NaturalGIS. Well done to everyone involved, and we wish all the luck to this great project.

Overview

Here follows the quick overview in reverse chronological order. If any of the names or short descriptions catches your attention, you can find the direct link to the plugin page in the table below:

Sorry, this entry is only available in French.

Earlier this year, I shared my experience using ChatGPT’s Data Analyst web interface for analyzing spatiotemporal data in the post “ChatGPT Data Analyst vs. Movement Data”. The Data Analyst web interface, while user-friendly, is not equipped to handle all types of spatial data tasks, particularly those involving more complex or large-scale datasets. Additionally, because the code is executed on a remote server, we’re limited to the libraries and tools available in that environment. I’ve often encountered situations where the Data Analyst simply doesn’t have access to the necessary libraries in its Python environment, which can be frustrating if you need specific GIS functionality.

Today, we’ll therefore start to explore alternatives to ChatGPT’s Data Analyst Web Interface, specifically, the OpenAI Assistant API. Later, I plan to dive deeper into even more flexible approaches, like Langchain’s Pandas DataFrame Agents. We’ll explore these options using spatial analysis workflow, such as:

1. Loading a zipped shapefile and investigate its content
2. Finding the three largest cities in the dataset
3. Selecting all cities in a region, e.g. in Scandinavia from the dataset
4. Creating static and interactive maps

To try the code below, you’ll need an OpenAI account with a few dollars on it. While gpt-3.5-turbo is quite cheap, using gpt-4o with the Assistant API can get costly fast.

OpenAI Assistant API

The OpenAI Assistant API allows us to create a custom data analysis environment where we can interact with our spatial datasets programmatically. To write the following code, I used the assistant quickstart and related docs (yes, shockingly, ChatGPT wasn’t very helpful for writing this code).

Like with Data Analyst, we need to upload the zipped shapefile to the server to make it available to the assistant. Then we can proceed to ask it questions and task it to perform analytics and create maps.

from openai import OpenAI

client = OpenAI()

file = client.files.create(
  file=open("H:/ne_110m_populated_places_simple.zip", "rb"),
  purpose='assistants'
)

Then we can hand the file over to the assistant:

assistant = client.beta.assistants.create(
  name="GIS Analyst",
  instructions="You are a personal GIS data analyst. Write and rund code to answer geospatial analysis questions",
  tools=[{"type": "code_interpreter"}],
  model="gpt-3.5-turbo",  # or "gpt-4o"
  tool_resources={
    "code_interpreter": {
      "file_ids": [file.id]
    }
  }  
)

Then we can start asking questions and giving our assistant tasks:

thread = client.beta.threads.create()

message = client.beta.threads.messages.create(
  thread_id=thread.id,
  role="user",
  content="Please load the zipped shapefile and describe the content"
)

assistant > The shapefile has been successfully loaded as a GeoDataFrame with the following details:

- The GeoDataFrame has 243 entries.
- It contains a total of 38 columns with various data types including integer, float, and object types.
- One of the columns is a geometry column representing the spatial information.

If you have any specific analysis or visualizations you would like to perform on this data, feel free to let me know!

The docs have a nice example for how to stream the answers back, e.g. into a Jupyter notebook. I’ve used it to create a small convenience function:

def ask_another_question(question):
    message = client.beta.threads.messages.create(
        thread_id=thread.id,
        role="user",
        content=question
    )
    with client.beta.threads.runs.stream(
        thread_id=thread.id,
        assistant_id=assistant.id,
        event_handler=EventHandler(),
    ) as stream:
        stream.until_done()

ask_another_question("Please list the three largest cities")

assistant > code_interpreter

# Sort the GeoDataFrame by the population in descending order
largest_cities = gdf.sort_values(by='pop_max', ascending=False).head(3)[['name', 'pop_max']]
largest_cities
assistant > The three largest cities in the dataset, along with their maximum population numbers, are as follows:

1. Tokyo: 35,676,000
2. New York: 19,040,000
3. Mexico City: 19,028,000

These cities have the highest recorded maximum population numbers in the dataset. Let me know if you need any further information or analysis!

ask_another_question("extract all scandinavian cities ")

assistant > To extract all the Scandinavian cities from the dataset, we can filter the GeoDataFrame based on the countries typically considered part of Scandinavia. These countries include Norway, Sweden, Denmark, Finland, and sometimes Iceland. Let's filter the GeoDataFrame to extract the Scandinavian cities.
assistant > code_interpreter

# List of countries typically considered part of Scandinavia
scandinavian_countries = ['Norway', 'Sweden', 'Denmark', 'Finland', 'Iceland']

# Filter the GeoDataFrame for Scandinavian cities
scandinavian_cities = gdf[gdf['adm0name'].isin(scandinavian_countries)][['name', 'adm0name']]

scandinavian_cities.head()  # Displaying the first few Scandinavian cities
assistant > Here are some of the Scandinavian cities extracted from the dataset:

1. Reykjavík, Iceland
2. Oslo, Norway
3. Helsinki, Finland
4. København (Copenhagen), Denmark
5. Stockholm, Sweden

These are just a few examples of Scandinavian cities present in the dataset. If you would like the complete list or perform any specific analysis on these cities, feel free to let me know!

ask_another_question(
    "please create a world map with all cities (use gray circles) "
    "with scandinavian cities highlighted as red circles. "
)

assistant > To create a world map with all cities represented as gray circles and Scandinavian cities highlighted as red circles, we will use the GeoPandas library for mapping. Let's go ahead and create the map.
assistant > code_interpreter

import matplotlib.pyplot as plt

# Plot all cities with gray circles and Scandinavian cities with red circles
fig, ax = plt.subplots(figsize=(12, 8))

# Plot all cities in gray circles
gdf.plot(ax=ax, color='lightgrey', markersize=5)

# Plot Scandinavian cities in red circles
scandinavian_cities.plot(ax=ax, color='red', markersize=50, edgecolor='black', label='Scandinavian Cities')

# Set the title
ax.set_title('World Map with Scandinavian Cities Highlighted', fontsize=16)

# Add legend
ax.legend()

# Display the map
plt.show()
assistant > It seems that there was an error while plotting the map because the GeoDataFrame `scandinavian_cities` does not have the necessary numeric data to plot the map directly.
...
plt.show()

output >

assistant > Here is the world map with all cities represented as gray circles and Scandinavian cities highlighted as red circles. The map provides a visual representation of the locations of the Scandinavian cities in relation to the rest of the cities around the world. If you need any further assistance or modifications, feel free to let me know!

To load and show the image, we can use:

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

def show_image():
    messages = client.beta.threads.messages.list(thread_id=thread.id)

    for m in messages.data:
        if m.role == "user":
            continue
        if m.content[0].type == 'image_file':
            m.content[0].image_file.file_id
            image_data = client.files.content(messages.data[0].content[0].image_file.file_id)
            image_data_bytes = image_data.read()
            with open("./out/my-image.png", "wb") as file:
                file.write(image_data_bytes)
            image = mpimg.imread("./out/my-image.png")
            plt.imshow(image)
            plt.box(False)
            plt.xticks([])
            plt.yticks([])
            plt.show() 
            break

Asking for an interactive map in an html file works in a similar fashion.

You can see the whole analysis workflow it in action here:

This way, we can use ChatGPT to perform data analysis from the comfort of our Jupyter notebooks. However, it’s important to note that, like the Data Analyst, the code we execute with the Assistant API runs on a remote server. So, again, we are restricted to the libraries available in that server environment. This is an issue we will address next time, when we look into Langchain.

Conclusion

ChatGPT’s Data Analyst Web Interface and the OpenAI Assistant API both come with their own advantages and disadvantages.

The results can be quite random. In the Scandinavia example, every run can produce slightly different results. Sometimes the results just use different assumptions such as, e.g. Finland and Iceland being part of Scandinavia or not, other times, they can be outright wrong.

As always, I’m interested to hear your experiences and thoughts. Have you been testing the LLM plugins for QGIS when they originally came out?

This release is the first to support GeoPandas 1.0.

Additionally, this release adds multiple new features, including:

• New explore() function adds interactive folium / leaflet maps
• New support for mfjson trajectory

For the full change log, check out the release page.

We have also revamped the documentation at https://movingpandas.readthedocs.io/ using the PyData Sphinx Theme:

On a related note: if you know what I need to change to get all Trajectory functions listed in the TOC on the right, please let me know.

Sorry, this entry is only available in French.

With the QGIS Grant Programme 2023, we were able to support six proposals (four in the first round and two in the second round) that are aimed to improve the QGIS project, including software, infrastructure, and documentation. The following reports summarize the work performed in the first four proposals:  

1. QGIS Bug Tracker cleanup (#266)  – Report
    We have identified and closed ~291 tickets, among them:
   • 162 bugreports and feature requests which were already fixed or implemented
   • 29 bugreports and feature requests which are invalid (data issues, wrong use of functionality, etc)
   • 57 duplicate bugreports and feature requests
   • 5 won’t fix bugreports
   • 5 bugreports were converted to feature requests
   • 33 tickets were closed (does not contain steps to reproduce, test data and no feedback was provided within several month)
   • Additionally we ensured that all tickets has correct tags assigned to to make them easier to find.
2. Porting to C++ and harmonization of Processing algorithms (#271) – Report
   The QGIS Porting to C++ and Harmonisation of Processing Algorithms grant is now complete.
   • Existing Processing algorithms Voronoi Polygons and Delaunay Triangulation have been ported to C++ and now use GEOS instead of the unmaintained Python module.
   • Two algorithms for generating XYZ tiles (directory and MBTiles variants) have been ported to C++ using a safer and cleaner multi-threading approach.
   • The Align Rasters tool, which was not exposed to Processing, has been removed and a new Processing algorithm with the same functionality has been added.
   • The existing Raster Calculator algorithm has been ported to C++. The algorithm now has two variants: a toolbox version that works the same way as before, and a modeler version that uses the same approach to input naming as the GDAL raster calculator.
3. Add vertical CRS handling to QGIS (#267) – Report
   • As of QGIS 3.34, QGIS can now create and handle vertical and compound CRSes.
   • In QGIS 3.34 coordinate transforms were reworked so that they function correctly with vertical transformation, if both the source and destination CRS have vertical components.
   • In QGIS 3.36 the coordinate reference selection widgets were updated to offer choices of 2d only, compound, or vertical only CRSes.
   • In version 3.38, we introduced a new setting for QGIS projects, for their vertical reference system. Users can control this through project properties, and it’s accessible via PyQGIS and via associated expression variables (eg @project_vertical_crs) for use in print layouts.
   • Similarly, in 3.38 we introduced the API support for map layers to have a vertical CRS. (This was not exposed to users in 3.38 though)
   • In QGIS 3.40 so far we have exposed the vertical CRS setting for vector layers to users (via the layer properties dialog), allowing users to specify the associated vertical CRS for these layers. The vertical CRS is respected in elevation profile plots, in Identify tool results, and in 3D Map views (assuming the 3D map is created with an associated vertical CRS).
   • There is an open pull-request for 3.40 to expose the vertical CRS for point cloud layers in a similar way, with the vertical CRS being respected in elevation profiles, identify tool results, and 3D map views
   • We have open pull requests for 3.40 to show layer vertical CRS information in the layer properties “information” pages, and add expression variables at the layer scope (eg @layer_vertical_crs).
4. Improve test result handling on QGIS CI (#268) – Report
   Any tests which fail a rendering comparison will write a descriptive comment to the PR. The comment details which render tests failed, where they are in the code, and includes some helpful pointers to downloading the full test report and the QGIS developer documentation. We also implemented lots of improvements in running the tests locally and how the render test reports are generated and presented to developers.

Thank you to everyone who participated and made this round of grants a great success and thank you to all our sustaining members and donors who make this initiative possible!

What's Changed

• Fix snapped coordinate cursors disappear under certain project/layer CRS conditions
• Fix value map and value relation editor widgets' handling of special characters (such as < and >)
• Harmonize handling of QField's multi-value relation editor widget with that of QGIS
• Insure that the orientation value is between 0 and 360 (instead of ~-90 to 270)

Full Changelog: v3.3.10...v3.3.11

What's Changed

• Multiple improvements to make cloud project attachments (photos, videos, audio clips, etc.) upload smoother
• Generated print and atlas layouts as well as photos can be pushed to QFieldCloud via the project folder panel

Full Changelog: v3.3.9...v3.3.10

For the last month of July, one short of 30 new plugins were published in the QGIS plugin repository.

Here follows the quick overview in reverse chronological order. If any of the names or short descriptions catches your attention, you can find the direct link to the plugin page in the table below:

What's Changed

• Fix handling of auto-generated FIDs in feature forms
• Fix current_parent_* expression values for (embedded) feature model
• Improve robustness of file path handling

Full Changelog: v3.3.8...v3.3.9

Sorry, this entry is only available in French.

What's Changed

• Fix Value relation editor widget not auto-completing on second feature
• Fix settings' panel tabs not translatable
• Fix untranslatable map canvas menu item and install/upgrade message
• Fix value relation handling of group name display when multi-selection is enabled
• Make the About QField data directory location a clickable hyperlink when relevant

Full Changelog: v3.3.7...v3.3.8