Related Plugins and Tags

QGIS Planet

Store and visualize your raster in the Cloud with COG and QGIS

September 14, 2020 QGIS Oslandia

We have recently been working for the French Space Agency ( CNES ) who needed to store and visualize satellite rasters in a cloud platform. They want to access the image raw data, with no transformation, in order to fullfill deep analysis like instrument calibration. Using classic cartographic server standard like WMS or TMS is not an option because those services transform datasets in already rendered tiles.

We chose to use a quite recent format managed by GDAL, the COG (Cloud Optimize Geotiff) and target OVH cloud platform for it provides OpenStack, a open source cloud computing platform.

How it works

A COG file is a GEOTiff file which inner structure is tiled, meaning that the whole picture is divided in fixed size tile (256 x 256 pixels for instance) so you can efficiently retrieve parts of the raster. In addition to the HTTP/1.1 standard feature range request, it is possible to get specific tiles of an image through the network without downloading the entire raster.

We used a service provided by OpenStack, called Object Storage to serve the COG imagery. Object storage allows to store and retrieve file as objects using HTTP GET/POST requests.

Why not WCS ?

Web Coverage Service standard could have been an option. A WCS server can serve raw data according to a given geographic extent. It’s completely possible to deploy a container or a VPS (Virtual Private Server) running a WCS Server in a cloud plateform. The main advantages of the COG solution over WCS Server is that you don’t have to deal with the burden of deploying a server, like giving it ressources, configuring load balancing, handle updates, etc…

The beauty of COG solution is its simplicity. It is only HTTP requests, and everything else (rendering for instance) is done on the client side.

Step by step

Here are the different steps you’d have to go through if you’re willing to navigate in a big raster image directly from the cloud.

First, let’s generate a COG file

gdal_translate inputfile.tif cogfile.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATE

Install your openstack-client, it can be achieved easily with Python pip install command line

$ pip install python-openstackclient

Next, configure your openstack client in order to generate an athentification token. To do so you need to download your project specific openrc file to setup your environment)

$ source myproject-openrc.sh
Please enter your OpenStack Password for project myproject as user myuser:
**********
$ openstack token issue                                 
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                                                                   |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| expires    | 2020-07-21T08:15:12+0000                                                                                                                                                                |
| id         | xxxx_my_token_xxxx
| project_id | 97e2e750f1904b41b76f80a50dabde0a                                                                                                                                                        |
| user_id    | 18f7ccaf1a2d4344a4e35f0d84eb065e                                                                                                                                                        |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

You are now good to push you COG file to the cloud instance

openstack object create MyContainer cogfile.tif --name cogfile.tif

Before starting QGIS, 2 environment variables need to be set.  (replace xxxx_my_token_xxxx with the token you’d just come to generate)

$ export SWIFT_AUTH_TOKEN=xxxx_my_token_xxxx
$ export SWIFT_STORAGE_URL=https://storage.sbg.cloud.ovh.net/v1/AUTH_$OS_PROJECT_ID

It can also be done directly from the QGIS Python console by setting those variable using the os.environ.

Finally, add a cloud raster data source in in QGIS

You can now navigating into your image directly reading it from the cloud

© CNES 2018, Distribution Airbus DS

Performances

While panning in the map, QGIS will download only few tiles from the image in order to cover the view extent. The display latency that you could see in the video depends essentially on:

  • The number of band of your image
  • The pixel size
  • Your internet connection (mine, the one use for the video, is not an awesome one)

Note that the white flickering that you could see when you move in the map and the raster is refreshed should be removed in next version of QGIS according to this QEP.

What’s next ?

Thanks so much to the GDAL and QGIS contributors for adding such a nice feature ! It brings lots of possibilities for organizations that have to deal with great number of big raster and just want to explore part of it.

We are already thinking about further improvments (ease authentification, better integration with processing…), so if you’re willing to fund them or just want to know more about QGIS, feel free to contact us at [email protected]. And please have a look at our support offering for QGIS.

by Julien Cabieces at 12:20 PM under cloud , gdal , gis , non classé , qgis , sig

Share and manage your Data with QGIS Cloud and WFS-T

September 2, 2014 Sourcepole

A lot of people are using QGIS Cloud as a service with ready to use QGIS webclient. It’s very easy to publish data and share maps in this way. But QGIS Cloud has more power under the hood. A not so obvious feature of QGIS Cloud is the option to share your data via Web Feature Service (WFS) and manage them via Web Feature Service Transactional (WFS-T). “The basic Web Feature Service allows querying and retrieval of features. A transactional Web Feature Service (WFS-T) allows creation, deletion, and updating of features” (Wikipedia). With WFS-T you have full access to your vector data for editing over the web. Since QGIS Server includes WFS-T functionality, you can manage and edit your data served by QGIS Cloud from every client supporting WFS-T. In addition, with QGIS Cloud Pro you have the option to control access to your published WFS.

How to setup a QGIS Cloud WFS-T in few steps:

  1. Setup a QGIS Project containing the data you like to pubish as WFS-T

  2. Load local vector data of your choice to your project.

  3. Define vector layers you wish to publish and set the appropriate settings for them in the following way:
    • open the Project Properties -> OWS Server tab.
    • scroll to the WFS-Capabilities section and setup the appropriate settings. Tick Published, Update, Insert and Delete for every layer you want to publish.

  • additionally you can set the published fields of every layer in the Layer Properties -> Fields tab.

  • Publish the project on QGIS Cloud.
    • save the project. (If you don’t have installed the QGIS Cloud plugin, than install it from the official QGIS Plugin Repository)
    • open the QGIS Cloud plugin and log in your QGIS Cloud account. (If you don’t have a QGIS Cloud account, sign up a new account).
    • upload the local data to your QGIS Cloud database (if you don’t have a QGIS Cloud database, create one from the QGIS Cloud plugin).
    • publish the project via QGIS Cloud plugin.
    • that’s it!

Have a look at the Services tab of the QGIS Cloud Plugin. There you will find the URL for Public WMS. Your just created WFS has the same URL. Now you can start working with WFS and WFS-T.

Working with WFS-T in QGIS Desktop

You can access your WFS-T with QGIS or any other client which supports WFS and WFS-T. As an example here we show how to access WFS with QGIS Desktop:

  1. Open the QGIS WFS Server connections dialog (Layer -> Add WFS Layer … ).
  2. Add a new connection
  3. Give the connection a name of your choice and add the above created URL
  4. Click connect and you will see the just published WFS layers
  5. Add one or more of them to your project

Thus you have set the Update, Insert and Delete options for the WFS, these layers can be edited in QGIS like any other editable layer.

All the services published under QGIS Cloud Free are public and accessible by everyone. If you need resctricted access , you can order the QGIS Cloud Pro plan.

Follow @QGISCloud on Twitter for QGISCloud related news and infos.

by hdus at 4:51 PM under cloud , news , ogc , qgis , wfs
  • Page 1 of 1 ( 2 posts )
  • cloud

Back to Top

Sustaining Members