Related Plugins and Tags

QGIS Planet

Movement data in GIS #29: power your web apps with movement data using mobilitydb-sqlalchemy

This is a guest post by Bommakanti Krishna Chaitanya @chaitan94

Introduction

This post introduces mobilitydb-sqlalchemy, a tool I’m developing to make it easier for developers to use movement data in web applications. Many web developers use Object Relational Mappers such as SQLAlchemy to read/write Python objects from/to a database.

Mobilitydb-sqlalchemy integrates the moving objects database MobilityDB into SQLAlchemy and Flask. This is an important step towards dealing with trajectory data using appropriate spatiotemporal data structures rather than plain spatial points or polylines.

To make it even better, mobilitydb-sqlalchemy also supports MovingPandas. This makes it possible to write MovingPandas trajectory objects directly to MobilityDB.

For this post, I have made a demo application which you can find live at https://mobilitydb-sqlalchemy-demo.adonmo.com/. The code for this demo app is open source and available on GitHub. Feel free to explore both the demo app and code!

In the following sections, I will explain the most important parts of this demo app, to show how to use mobilitydb-sqlalchemy in your own webapp. If you want to reproduce this demo, you can clone the demo repository and do a “docker-compose up –build” as it automatically sets up this docker image for you along with running the backend and frontend. Just follow the instructions in README.md for more details.

Declaring your models

For the demo, we used a very simple table – with just two columns – an id and a tgeompoint column for the trip data. Using mobilitydb-sqlalchemy this is as simple as defining any regular table:

from flask_sqlalchemy import SQLAlchemy
from mobilitydb_sqlalchemy import TGeomPoint

db = SQLAlchemy()

class Trips(db.Model):
   __tablename__ = "trips"
   trip_id = db.Column(db.Integer, primary_key=True)
   trip = db.Column(TGeomPoint)

Note: The library also allows you to use the Trajectory class from MovingPandas as well. More about this is explained later in this tutorial.

Populating data

When adding data to the table, mobilitydb-sqlalchemy expects data in the tgeompoint column to be a time indexed pandas dataframe, with two columns – one for the spatial data  called “geometry” with Shapely Point objects and one for the temporal data “t” as regular python datetime objects.

from datetime import datetime
from shapely.geometry import Point

# Prepare and insert the data
# Typically it won’t be hardcoded like this, but it might be coming from 
# other data sources like a different database or maybe csv files
df = pd.DataFrame(
   [
       {"geometry": Point(0, 0), "t": datetime(2012, 1, 1, 8, 0, 0),},
       {"geometry": Point(2, 0), "t": datetime(2012, 1, 1, 8, 10, 0),},
       {"geometry": Point(2, -1.9), "t": datetime(2012, 1, 1, 8, 15, 0),},
   ]
).set_index("t")

trip = Trips(trip_id=1, trip=df)
db.session.add(trip)
db.session.commit()

Writing queries

In the demo, you see two modes. Both modes were designed specifically to explain how functions defined within MobilityDB can be leveraged by our webapp.

1. All trips mode – In this mode, we extract all trip data, along with distance travelled within each trip, and the average speed in that trip, both computed by MobilityDB itself using the ‘length’, ‘speed’ and ‘twAvg’ functions. This example also shows that MobilityDB functions can be chained to form more complicated queries.

mobilitydb-sqlalchemy-demo-1

trips = db.session.query(
   Trips.trip_id,
   Trips.trip,
   func.length(Trips.trip),
   func.twAvg(func.speed(Trips.trip))
).all()

2. Spatial query mode – In this mode, we extract only selective trip data, filtered by a user-selected region of interest. We then make a query to MobilityDB to extract only the trips which pass through the specified region. We use MobilityDB’s ‘intersects’ function to achieve this filtering at the database level itself.

mobilitydb-sqlalchemy-demo-2

trips = db.session.query(
   Trips.trip_id,
   Trips.trip,
   func.length(Trips.trip),
   func.twAvg(func.speed(Trips.trip))
).filter(
   func.intersects(Point(lat, lng).buffer(0.01).wkb, Trips.trip),
).all()

Using MovingPandas Trajectory objects

Mobilitydb-sqlalchemy also provides first-class support for MovingPandas Trajectory objects, which can be installed as an optional dependency of this library. Using this Trajectory class instead of plain DataFrames allows us to make use of much richer functionality over trajectory data like analysis speed, interpolation, splitting and simplification of trajectory points, calculating bounding boxes, etc. To make use of this feature, you have set the use_movingpandas flag to True while declaring your model, as shown in the below code snippet.

class TripsWithMovingPandas(db.Model):
   __tablename__ = "trips"
   trip_id = db.Column(db.Integer, primary_key=True)
   trip = db.Column(TGeomPoint(use_movingpandas=True))

Now when you query over this table, you automatically get the data parsed into Trajectory objects without having to do anything else. This also works during insertion of data – you can directly assign your movingpandas Trajectory objects to the trip column. In the below code snippet we show how inserting and querying works with movingpandas mode.

from datetime import datetime
from shapely.geometry import Point

# Prepare and insert the data
# Typically it won’t be hardcoded like this, but it might be coming from 
# other data sources like a different database or maybe csv files
df = pd.DataFrame(
   [
       {"geometry": Point(0, 0), "t": datetime(2012, 1, 1, 8, 0, 0),},
       {"geometry": Point(2, 0), "t": datetime(2012, 1, 1, 8, 10, 0),},
       {"geometry": Point(2, -1.9), "t": datetime(2012, 1, 1, 8, 15, 0),},
   ]
).set_index("t")

geo_df = GeoDataFrame(df)
traj = mpd.Trajectory(geo_df, 1)

trip = Trips(trip_id=1, trip=traj)
db.session.add(trip)
db.session.commit()

# Querying over this table would automatically map the resulting tgeompoint 
# column to movingpandas’ Trajectory class
result = db.session.query(TripsWithMovingPandas).filter(
   TripsWithMovingPandas.trip_id == 1
).first()

print(result.trip.__class__)
# <class 'movingpandas.trajectory.Trajectory'>

Bonus: trajectory data serialization

Along with mobilitydb-sqlalchemy, recently I have also released trajectory data serialization/compression libraries based on Google’s Encoded Polyline Format Algorithm, for python and javascript called trajectory and trajectory.js respectively. These libraries let you send trajectory data in a compressed format, resulting in smaller payloads if sending your data through human-readable serialization formats like JSON. In some of the internal APIs we use at Adonmo, we have seen this reduce our response sizes by more than half (>50%) sometimes upto 90%.

Want to learn more about mobilitydb-sqlalchemy? Check out the quick start & documentation.


This post is part of a series. Read more about movement data in GIS.

Quick webmaps with qgis2web

In Publishing interactive web maps using QGIS, I presented two plugins for exporting web maps from QGIS. Today, I want to add an new member to this family: the qgis2web plugin is the successor of qgis-ol3 and combines exports to both OpenLayers3 as well as Leaflet.

The plugin is under active development and currently not all features are supported for both OpenLayers3 and Leaflet, but it’s a very convenient way to kick-off a quick webmapping project.

Here’s an example of an OpenLayers3 preview with enabled popups:

OpenLayers3 preview

OpenLayers3 preview

And here is the same map in Leaflet with the added bonus of a nice address search bar which can be added automatically as well:

Leaflet preview

Leaflet preview

The workflow is really straight forward: select the desired layers and popup settings, pick some appearance extras, and then don’t forget to hit the Update preview button otherwise you might be wondering why nothing happens ;)

I’ll continue testing these plugins and am looking forward to seeing what features the future will bring.


Publishing interactive web maps using QGIS

We all know that QGIS is great for designing maps but did you know that QGIS is also great for interactive web maps? It is! Just check out qgis2leaf and qgis2threejs.

To give these two plugins a test run and learn some responsive web design, I developed a small concept page presenting cycle routes in 3D.

Screenshot 2015-01-31 22.20.15

Qgis2leaf makes it possible to generate Leaflet maps from QGIS layers. It provides access to different background maps and it’s easy to replace them in the final HTML file in case you need something more exotic. I also added another layer with custom popups with images but that was done manually.

Daten CC-BY-3.0: Land Kärnten - data.ktn.gv.at

The web maps use data CC-BY-3.0: Land Kärnten – data.ktn.gv.at

Qgis2threejs on the other hand creates 3D visualizations based on three.js which uses WebGL. (If you follow my blog you might remember a post a while back which showcased Qgis2threejs rendering OSM buildings.)

This is a great way to explore elevation data. I also think that the labeling capabilities add an interesting touch. Controlling the 3D environment takes some getting used to, but if you can handle Google Earth in your browser, this is no different.

Image of Heiligenblut by Angie (Self-photographed) (GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)), via Wikimedia Commons

Image of Heiligenblut by Angie (Self-photographed) (GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)), via Wikimedia Commons


Leaflet, Stamen Toner and QGIS Server – An Intro

Today’s post is a note-to-self on how to set up a really quick little Leaflet web map with Stamen’s “Toner” background and a WMS overlay served by QGIS Server.

Note the use of the map parameter when creating the QGIS Server WMS layer.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>

<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<meta http-equiv="content-language" content="en" />

<title>Leaflet, Stamen Toner and QGIS Server</title>

<link rel="stylesheet" href="http://leaflet.cloudmade.com/dist/leaflet.css" />
<link rel="stylesheet" href="my.css" type="text/css" />
 <!--[if lte IE 8]><link rel="stylesheet" href="http://leaflet.cloudmade.com/dist/leaflet.ie.css" /><![endif]-->

<script type="text/javascript" src="http://leaflet.cloudmade.com/dist/leaflet.js"></script>
<script type="text/javascript" src="http://maps.stamen.com/js/tile.stamen.js?v1.1.3"></script>
<script type="text/javascript">
function initialize() {
	var stamen = new L.StamenTileLayer("toner-lite");
	
	var myLayer = new L.TileLayer.WMS("http://10.101.21.28/cgi-bin/qgis_mapserv.fcgi", {
		map: "/usr/lib/cgi-bin/test/test.qgs",
		layers: 'mylayer',
		format: 'image/png',
		transparent: 'TRUE',
	});
	
	var map = new L.Map("map", {
		center: new L.LatLng(48.2,16.4),
		zoom: 13,
 		minZoom: 10,
 		maxZoom: 18,
		layers: [stamen,myLayer],
	});
}
</script>

</head>

<body onload="initialize()">
    <div id="map" class="map"></div>
</body>

</html>

With this my.css, the map will be displayed in “full screen”.

  div.map{
    display:block;
    position:absolute;
    top:0;
    left:0;
    width:100%;
    height:100%;
  }

Update

You can test an extended live example at http://anitagraser.github.com/Webmapping-Sandbox/leaflet.html


Back to Top

Sustaining Members