Store and visualize your raster in the Cloud with COG and QGIS
We have recently been working for the French Space Agency ( CNES ) who needed to store and visualize satellite rasters in a cloud platform. They want to access the image raw data, with no transformation, in order to fullfill deep analysis like instrument calibration. Using classic cartographic server standard like WMS or TMS is not an option because those services transform datasets in already rendered tiles.
We chose to use a quite recent format managed by GDAL, the COG (Cloud Optimize Geotiff) and target OVH cloud platform for it provides OpenStack, a open source cloud computing platform.
How it works
A COG file is a GEOTiff file which inner structure is tiled, meaning that the whole picture is divided in fixed size tile (256 x 256 pixels for instance) so you can efficiently retrieve parts of the raster. In addition to the HTTP/1.1 standard feature range request, it is possible to get specific tiles of an image through the network without downloading the entire raster.
We used a service provided by OpenStack, called Object Storage to serve the COG imagery. Object storage allows to store and retrieve file as objects using HTTP GET/POST requests.
Why not WCS ?
Web Coverage Service standard could have been an option. A WCS server can serve raw data according to a given geographic extent. It’s completely possible to deploy a container or a VPS (Virtual Private Server) running a WCS Server in a cloud plateform. The main advantages of the COG solution over WCS Server is that you don’t have to deal with the burden of deploying a server, like giving it ressources, configuring load balancing, handle updates, etc…
The beauty of COG solution is its simplicity. It is only HTTP requests, and everything else (rendering for instance) is done on the client side.
Step by step
Here are the different steps you’d have to go through if you’re willing to navigate in a big raster image directly from the cloud.
First, let’s generate a COG file
Install your openstack-client, it can be achieved easily with Python pip install command line
Next, configure your openstack client in order to generate an athentification token. To do so you need to download your project specific openrc file to setup your environment)
You are now good to push you COG file to the cloud instance
Before starting QGIS, 2 environment variables need to be set. (replace xxxx_my_token_xxxx with the token you’d just come to generate)
It can also be done directly from the QGIS Python console by setting those variable using the os.environ.
Finally, add a cloud raster data source in in QGIS
You can now navigating into your image directly reading it from the cloud
© CNES 2018, Distribution Airbus DS
Performances
While panning in the map, QGIS will download only few tiles from the image in order to cover the view extent. The display latency that you could see in the video depends essentially on:
- The number of band of your image
- The pixel size
- Your internet connection (mine, the one use for the video, is not an awesome one)
Note that the white flickering that you could see when you move in the map and the raster is refreshed should be removed in next version of QGIS according to this QEP.
What’s next ?
Thanks so much to the GDAL and QGIS contributors for adding such a nice feature ! It brings lots of possibilities for organizations that have to deal with great number of big raster and just want to explore part of it.
We are already thinking about further improvments (ease authentification, better integration with processing…), so if you’re willing to fund them or just want to know more about QGIS, feel free to contact us at [email protected]. And please have a look at our support offering for QGIS.
You will probably interact first with our pre-sales engineer Bertrand Parpoil. He leads Geographical Information System projects for 15 years for large corporations, public administrations or hi-tech SME. Bertrand will listen to your needs and explore your use cases, to offer you the best set of services. 
Régis Haubourg also takes part in the first steps of projects to analyze your usages and improve them. GIS Expert, he knows QGIS by heart and will make the most of its capabilities. As QGIS Community Manager at Oslandia, he is very active in the QGIS community of developers and contributors. He is president of the Francophone OSGeo local chapter ( OSGeo-fr ), QGIS voting member, organizes the French QGIS day conference in Montpellier, and participates to QGIS community meetings. Before joining Oslandia, he led the migration to QGIS and PostGIS at the Adour-Garonne Water Agency, and now guides our clients with their GIS migrations to OpenSource solutions. Régis is also a great asset when working on water, hydrology and other specific thematic subjects.
Loïc Bartoletti develops QGIS, specializing in features corresponding to his fields of interests : network management, topography, urbanism, architecture… We find him contributing to advanced vector editing in QGIS, writing Python plugins, namely for DICT management. Pushing CAD and migrations from CAD tools to GIS and QGIS is one of his major goals. He will develop your custom applications, combining technical expertise and functional competences. When bored, Loïc packages software on FreeBSD.
Vincent Mora is senior developer in Python and C++, as well as PostGIS expert. He has a strong experience in numerical simulation. He likes coupling GIS (PostGIS, QGIS) with 3D numerical computing for risk management or production optimization. Vincent is an official QGIS committer and can directly integrate your needs into the core of the software. He is also GDAL committer and optimizes low-level layers of your applications. Among numerous activities, Vincent serves as lead developer of the development team for 
Hugo Mercier is an officiel QGIS committer too for several years. He regularly talks in international conferences on PostGIS, QGIS and other OpenSource GIS softwares. He will implement your needs with new QGIS features, develop innovative plugins ( like 
Paul Blottière completes our QGIS committers : very active on core development, Paul has refactored the QGIS server component to bring it to an industry-grade quality level. He also designed and implemented 
Julien Cabièces recently joined Oslandia, and quickly dived into QGIS : he contributes to the core of this Desktop GIS, on the server component, as well as applications linked to numerical simulation. Coming from a satellite imagery company with industrial applications, he uses his flexibility to answer all your needs. He brings quality and professionalism to your projects, minimizing risks for large production deployments.
