What are we looking for?

We would like to use WMS offline on QField. For that, we need to figure out what is the best way to get a raster from a WMS and which format is the most efficient (size and performance).

In this post we’ll show you is how to generate the ideal raster file from a WMS and the results of our efficiency tests for the the different raster formats.

WMS to GPKG

The simple way

If there is no limitation on the WMS or you need only a small region, here is the easiest process.

  1. Request the WMS and store a description file in XML:
gdal_translate "WMS:url" file.xml -of WMS
  1. Create a Geopackage from the information in the description file.
gdal_translate -of GPKG file.xml file.gpkg -co TILE_FORMAT=JPEG

That was quite simple, right?

The larger datasets way

If the command takes too much time, it means that it is trying to download too much data and could be caused by downloading higher resolution data than required.
The command might even completely fail if it contains a request for bigger data blocks thant the server allows.

Here is the process to get larger datasets in a simple way. Let’s use a real example:

  1. Use gdal_translate "WMS:https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv?request=getmap&service=wms&crs=EPSG:4326&format=image/jpeg&layers=gebco_latest&version=1.1.0" test.xml -of WMS
  2. Open the test.xml file for editing, here you’ll find the parameters of the WMS. We change the “SizeX” to 3600 and “SizeY” to 1800. By changing these parameters we lower the resolution. It is important to keep proportionality.
  3. Another thing we need to change are “BlockSizeX” and “BlockSizeY” that define the size of the tiles. We change both to 2048.
  4. Finally, use gdal_translate -of GPKG test.xml test.gpkg -co TILE_FORMAT=JPEG
  5. To make a Geopackage pyramid use gdaladdo GPKG:test.gpkg:gebco_latest. It will replace the Geopackage, if you want to keep the original one, you need to copy it first.

Now you have a raster Geopackage that you can use in QField.

Testing raster formats

Preparing the files

As first step we exported our test orthophoto WMS to a plain GeoTIFF using QGIS’ default behaviour.

Default parameters used to create the initial tiff
Formatgdal_translategdaladdo
gpkg JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEG
gpkg PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNG
gpkg PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEG
gpkg PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8
gpkg WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBP
gpkg pyramid_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEGgdaladdo GPKG:C:\test\test_JPEG.gpkg:test_gpkg_JPEG
gpkg pyramid_PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgdaladdo GPKG:C:\test\test_PNG.gpkg:test_gpkg_PNG
gpkg pyramid_PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgdaladdo GPKG:C:\test\test_PNG_JPEG.gpkg:test_gpkg_PNG_JPEG
gpkg pyramid_PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gdaladdo GPKG:C:\test\test_PNG8.gpkg:test_gpkg_PNG8
gpkg pyramid_WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgdaladdo GPKG:C:\test\test_WEBP.gpkg:test_gpkg_WEBP
JPEG2000gdal_translate -of JP2OpenJPEG “C:\test\ortho_test.tif” “C:\test\test_jpeg_2000.jpg”
COG DEFLATEgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATE
COG_JPEGgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog_JPEG.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=JPEG
tifIn QGIS right click on the layer > export > save as > (see the details in the picture under the table)
MBTgdal_translate -of MBTILES “C:\test\ortho_test.tif” “C:\test\test_mbt.mbtiles”
Creation commands for all the tested formats

Rendering test results

We have tested many formats, here is a table with the results of the size and rendering speed in QGIS and QField.
To analyze the speed we used qgis_bench.exe -i 10 -p "C:\test\test.qgs" >> "C:\test\test.log.
Qgis_bench is a tool that renders a QGIS project a number of times to get performance measurements. The parameter -i is to define the iterations and -p is the project used which contains only the generated raster.

FormatExtent [m]File size [GB]Total_avgTotal_maxdevTotal_minTotal_stdev
gpkg JPEG52’880/29’2300.4250.242255.7815.539244.984
gpkg PNG52’880/29’2302.9412.002490.328152.142259.859
gpkg PNG_JPEG52’880/29’2300.4250.125256.8756.750245.172
gpkg PNG852’880/29’2301.4283.875296.40612.625271.250
gpkg WEBP52’880/29’2300.3330.238348.10973.534256.703
gpkg pyramid_JPEG52’880/29’2300.51.0093.4062.3970.688
gpkg pyramid_PNG52’880/29’2303.01.2083.2812.0730.688
gpkg pyramid_PNG_JPEG52’880/29’2300.61.4914.3442.8531.016
gpkg pyramid_PNG852’880/29’2301.61.5084.3752.8670.969
gpkg pyramid_WEBP52’880/29’2300.41.3334.9063.5730.766
JPEG200052’880/29’2301.113.888136.109122.2220.219
COG DEFLATE52’880/29’2303.6264.427273.09425.411239.016
COG_JPEG52’880/29’2301.014.778131.172116.3941.734
tif52’880/29’2306.42.3676.7344.3671.672
MBT52’880/29’2304.40.4694.6414.1710
Comparison of file size and rendering speed of different raster formats. “Total” columns are rendering times in [s]. Lower file size is more storage friendly, lower Total_avg is more performant.

Analysis

File size

The Geopackage WEBP (with and without pyramid) has the best result for file size, but it is not yet supported by QField (from 1.6) and is only slightly smaller than the JPEG variant.

Plain GeoTiff, MBTiles, Cloud Optimized GeoTIFF (COG – DEFLATE mode) and Geopackages with PNG generate by far the largest file sizes (up to 20x larger) and are thus not recommended.

Rendering speed

MBTiles are on average double as fast as JPEG Geopackages with pyramids which in turn are more than double as fast as GeoTIFF and 15x faster than COG.
Geopackages without pyramids are 200 to 400 times slower.

Conclusion

Even though MBTiles render faster than the Geopackage pyramid JPEG, they come with an almost 10x bigger storage requirement which makes us say that the best offline raster format supported by QField is Geopackage pyramid JPEG or if you need transparency and slightly smaller files Geopackage pyramid WebP.

If you need transparency before QField 1.6, the best results are achieved with Geopackage pyramid PNG_JPEG.

by Lucie at 2:15 PM under processing , qfield , qgis , qgis.org , scripts (Comments)