Recently I was invited by the colleagues from OpenGIS.ch to lend a hand in a training session about QGIS server.

This was a good opportunity to update my presentation for QGIS3, to fix a few bugs and to explore the powerful capabilities of QGIS server and Python.

As a result, I published the full recipe of a Vagrant VM on github: https://github.com/elpaso/qgis3-server-vagrant

The presentation is online here: http://www.itopen.it/bulk/qgis3-server/

What’s worth mentioning is the sample plugins (I’ll eventually package and upload them to the official plugin site):

• XYZ: add simple XYZ tile server, ready to use within QGIS XYZ connections: https://github.com/elpaso/qgis3-server-vagrant/tree/master/resources/web/plugins/xyz
• custom service, does nothing but shows how to create a Python custom service by exploiting the new service capabilities available in QGIS3: https://github.com/elpaso/qgis3-server-vagrant/tree/master/resources/web/plugins/customservice
• HTTP Basic auth (how to add your auth scheme to QGIS server): https://github.com/elpaso/qgis3-server-vagrant/tree/master/resources/web/plugins/httpbasic
• more …

 

The VM uses 4 different (although similar) deployment strategies:

• good old Apache + mod_fcgi and plain CGI
• Nginx + Fast CGI
• Nginx + standalone HTTP Python wrapped server
• Nginx + standalone WSGI Python wrapped server

Have fun with QGIS server: it was completely refactored in QGIS 3 and it’s now better than ever!

 

The post QGIS 3 Server deployment showcase with Python superpowers first appeared on Open Web Solutions, GIS & Python Development.

Here is a little first experiment for a wsgi wrapper to QGIS 3 Server, not much tested, but basically working:  

#!/usr/bin/env python

# Simple QGIS 3 Server wsgi test

import signal
import sys
from cgi import escape, parse_qs
from urllib.parse import quote
# Python's bundled WSGI server
from wsgiref.simple_server import make_server

from qgis.core import QgsApplication
from qgis.server import *

# Init QGIS
qgs_app = QgsApplication([], False)
# Init server
qgs_server = QgsServer()


def reconstruct_url(environ):
    """Standard algorithm to retrieve the full URL from wsgi request
    From: https://www.python.org/dev/peps/pep-0333/#url-reconstruction
    """
    url = environ['wsgi.url_scheme']+'://'

    if environ.get('HTTP_HOST'):
        url += environ['HTTP_HOST']
    else:
        url += environ['SERVER_NAME']

        if environ['wsgi.url_scheme'] == 'https':
            if environ['SERVER_PORT'] != '443':
                url += ':' + environ['SERVER_PORT']
        else:
            if environ['SERVER_PORT'] != '80':
                url += ':' + environ['SERVER_PORT']

    url += quote(environ.get('SCRIPT_NAME', ''))
    url += quote(environ.get('PATH_INFO', ''))
    if environ.get('QUERY_STRING'):
        url += '?' + environ['QUERY_STRING']
    return url

def application (environ, start_response):

    headers = {} # Parse headers from environ here if needed

    # the environment variable CONTENT_LENGTH may be empty or missing
    # When the method is POST the variable will be sent
    # in the HTTP request body which is passed by the WSGI server
    # in the file like wsgi.input environment variable.
    try:
        request_body_size = int(environ.get('CONTENT_LENGTH', 0))
        request_body = environ['wsgi.input'].read(request_body_size)
    except (ValueError):
        request_body_size = 0
        request_body = None

    request = QgsBufferServerRequest(reconstruct_url(environ), (QgsServerRequest.PostMethod 
        if environ['REQUEST_METHOD'] == 'POST' else QgsServerRequest.GetMethod), {}, request_body)
    response = QgsBufferServerResponse()
    qgs_server.handleRequest(request, response)
    headers_dict = response.headers()
    try:
        status = headers_dict['Status']
    except KeyError:
        status = '200 OK'
    start_response(status, [(k, v) for k, v in headers_dict.items()])
    return [bytes(response.body())]

# Instantiate the server
httpd = make_server (
    'localhost', # The host name
    8051, # A port number where to wait for the request
    application # The application object name, in this case a function
)

print("Listening to http://localhost:8051 press CTRL+C to quit")

def signal_handler(signal, frame):
    """Exit QGIS cleanly"""
    global qgs_app
    print("\nExiting QGIS...")
    qgs_app.exitQgis()
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)

httpd.serve_forever()

 

The post A little QGIS3 Server wsgi experiment first appeared on Open Web Solutions, GIS & Python Development.

Another great QGIS hackfest is gone, and it’s time for a quick report. The location was the Linux Hotel, one of the best places where open source developers could meet, friendly, geek-oriented and when the weather is good, like this time, villa Vogelsang is a wonderful place to have a beer in the garden while talking about software development or life in general. This is a short list of what kept me busy during the hackfest:

• fixed some bugs and feature requests on the official QGIS plugin repo that I’m maintaining since the very beginning
• make the QGIS official plugin repository website mobile-friendly
• QGIS Server Python Plugin API refactoring, I’ve completed the work on the new API, thanks to the ongoing server refactoring it’s now much cleaner than it was in the first version
• attribute table bugs: I started to address some nasty bugs in the attribute table, some of those were fixed during the week right after the hackfest
• unified add layer button, we had a productive meeting where we decided the path forward to implement this feature, thanks to Boundless that is funding the development, this feature is what’s I’m currently working on these days

Thanks to all QGIS donors and funders that made yet another great hackfest possible and in particular to Boundless Spatial Inc. for funding my personal expenses.    

The post Essen 2017 QGIS Hackfest first appeared on Open Web Solutions, GIS & Python Development.

QGIS Developer Sprint in Lyon   QGIS Server 3.0 is going to be better than ever! Last week I attended to the mini code-sprint organized by the french QGIS developers in Lyon.   The code sprint was focused on QGIS Server refactoring to reach the following goals:

• increase maintainability through modularity and clean code responsibilities
• increase performances
• better multi-project handling and caching
• scalability
• multi threaded rendering

By working for different companies on such a big Open Source project like QGIS, coordination between developers is fundamentally achieved through those kind of events. We were a small group of engaged QGIS Server developers and I think that the alternance between brainstorming and coding has proven to be very productive: after two days we were able to set common milestones and commitments that will ensure a bright future to QGIS Server. A huge and warm thank to the french QGIS developers that organized this meeting!   Photo: courtesy of Règis Haubourg    

The post QGIS Developer Sprint in Lyon first appeared on Open Web Solutions, GIS & Python Development.

Today I’m releasing the first version of QGIS Server Simple Browser Plugin, a simple Server plugin that generates a browsable table of contents of the project’s layers and a link to an OpenLayers map.  

How it works

The plugin adds an XSL stylesheet to GetProjectsettings XML response, the generated HTML looks like this:  

The openlayers format

The map preview is generated by adding a new application/openlayers FORMAT option to GetMap requests, the generated map automatically fits to the layer’s extent and has basic GetFeatureInfo capabilities.

Limitations

The current version only supports EPSG:4326 that must be available (enabled) on the server.

Source code and download

The plugin is available on the official repository: ServerSimpleBrowser The code is on GitHub.

Sometimes is hard to debug segfaults appearing in QGIS Server when running in CGI mode. The classic approach is attaching a gdb to the running process. The problem is that there is not enough time to do it! A simple plugin filter, can provide you the time you need to attach the debugger:

from qgis.server import *
from qgis.core import *
import os
     
class DelayFilter(QgsServerFilter):
     
    def __init__(self, serverIface):
        super(DelayFilter, self).__init__(serverIface)    
     
    def responseComplete(self):        
        request = self.serverInterface().requestHandler()
        params = request.parameterMap()
        if params.get('DELAY', ''):
            QgsMessageLog.logMessage("PID: %s" % os.getpid())     
            import time
            time.sleep(30)

Calling the server with DELAY=1 will wait for 30 seconds and print the current PID in the server logs. This will give you enough time to fire gdb and attach it to the process.

With QGIS 2.12 the new Python bindings for QGIS server are now available and the server can be invoked directly from a python scripts with just a few lines of code:

 

from qgis.server import QgsServer
my_query_string = "map=/projects/my_project.qgs&SERVICE=WMS&request=GetCapabilities"
headers, body =  QgsServer().handleRequest(my_query_string)

 

Embedding QGIS in a Python web application

Embedding QGIS Server in a web application is now not only possible but really very easy, for example, a Django view:

# QGIS server view

from django.http import HttpResponse
from django.views.generic import View
from qgis.server import *


class OGC(View):
    """Pass a GET request to QGIS Server and return the response"""

    def __init__(self):
        self.server = QgsServer()        

    def get(self, request, *args, **kwargs):
        """Pass a GET request to QGIS Server and return the response"""
        headers, body = self.server.handleRequest(request.GET.urlencode())
        response = HttpResponse(body)
        # Parse headers
        for header in headers.split('\n'):
            if header:
                k, v = header.split(': ', 1)
                response[k] = v
        return response

 

Using server plugins

 

Of course Python server plugins can be plugged in easily, see the example below:

 

# QGIS server view

from django.http import HttpResponse
from django.views.generic import View
from qgis.server import *
from qgis.core import *



class OGC(View):
    """Pass a GET request to QGIS Server and return the response"""

    def __init__(self):
        self.server = QgsServer()
        # Call init to create serverInterface
        self.server.init()
        serverIface = self.server.serverInterface()

        class Filter1(QgsServerFilter):
            def responseComplete(self):
                QgsMessageLog.logMessage("Filter1.responseComplete", "Server", QgsMessageLog.INFO )
                request = self.serverInterface().requestHandler()
                if request.parameter('REQUEST') == 'HELLO':
                    request.clearHeaders()
                    request.setHeader('Content-type', 'text/plain')
                    request.clearBody()
                    request.appendBody('Hello from SimpleServer!')

            def requestReady(self):
                QgsMessageLog.logMessage("Filter1.requestReady")

        self.filter = Filter1(serverIface)
        serverIface.registerFilter(self.filter)


    def get(self, request, *args, **kwargs):
        """Pass a GET request to QGIS Server and return the response"""
        headers, body = self.server.handleRequest(request.GET.urlencode())
        response = HttpResponse(body)
        # Parse headers
        for header in headers.split('\n'):
            if header:
                k, v = header.split(': ', 1)
                response[k] = v
        return response

 

Enjoy QGIS Server with Python!

A few days ago the new WebGIS of the City of Asti, a 76000 inhabitants city in Piedmont, was launched.  The new WebGIS uses QGIS Server and QGIS Web Client to serve maps and provide street and cadastrial search and location services.

The new WebGIS was developed by ItOpen and is online at: http://sit.comune.asti.it/site/?map=PRGAsti

During the hackfest I’ve been working on the refactoring of the server component, aimed to wrap the server into a class and create python bindings for the new classes. This work is now in the PR queue and brings a first working python test for the server itself.

The server can now be invoked directly from python, like in the example below:

 

#!/usr/bin/env python
"""
Super simple QgsServer.
"""

from qgis.server import *
from BaseHTTPServer import *

class handler (BaseHTTPRequestHandler):

    server = QgsServer()

    def _doHeaders(self, response):
        l = response.pop(0)
        while l:
            h = l.split(':')
            self.send_header(h[0], ':'.join(h[1:]))
            self.log_message( "send_header %s - %s" % (h[0], ':'.join(h[1:])))
            l = response.pop(0)
        self.end_headers()

    def do_HEAD(self):
        self.send_response(200)
        response = str(handler.server.handleRequestGetHeaders(self.path[2:])).split('\n')
        self._doHeaders(response)

    def do_GET(self):
        response = str(handler.server.handleRequest(self.path[2:])).split('\n')
        i = 0
        self.send_response(200)
        self._doHeaders(response)
        self.wfile.write(('\n'.join(response[i:])).strip())

    def do_OPTIONS(s):
        handler.do_GET(s)

httpd = HTTPServer( ('', 8000), handler)

while True:
    httpd.handle_request()

The python bindings capture the server output instead of printing it on FCGI stdout and allow to pass the request parameters QUERY_STRING directly to the request handler as a string, this makes writing python tests very easy.

There have been some requests in the past about custom CSS for html GetFeatureInfo responses from QGIS Server.

Currently, the HTML response template is hardcoded and there is no way to customize it, the Python plugin support introduced with the latest version of QGIS Server provides an easy way to add some custom CSS rules or even provide custom templates.

To get you started, I’ve added a new filter to my example  HelloServer plugin:

import os

from qgis.server import *
from qgis.core import *
from PyQt4.QtCore import *
from PyQt4.QtGui import *


class GetFeatureInfoCSSFilter(QgsServerFilter):

    def __init__(self, serverIface):
        super(GetFeatureInfoCSSFilter, self).__init__(serverIface)

    def requestReady(self):
        """Nothing to do here, but it would be the ideal point
        to alter request **before** it gets processed, for example
        you could set INFO_FORMAT to text/xml to get XML instead of
        HTML in responseComplete"""
        pass

    def responseComplete(self):
        request = self.serverInterface().requestHandler()
        params = request.parameterMap( )
        if (params.get('SERVICE').upper() == 'WMS' \
                and params.get('REQUEST', '').upper() == 'GETFEATUREINFO' \
                and params.get('INFO_FORMAT', '').upper() == 'TEXT/HTML' \
                and not request.exceptionRaised() ):
            body = request.body()
            body.replace('<BODY>', """<BODY><STYLE type="text/css">* {font-family: arial, sans-serif; color: blue;}</STYLE>""")
            # Set the body
            request.clearBody()
            request.appendBody(body)

This filter is pretty simple, if the request is a WMS GetFeatureInfo with HTML format, it injects a STYLE tag into the HTML HEAD.

Here is the output with blue color and arial fonts applied:

As an exercise left to the reader, you can also intercept the call in requestReady(self), change the INFO_FORMAT to text/xml and then do some real templating, for example by using XSLT or by parsing the XML and injecting the values into a custom template.

 

Today it’s a great day for QGIS Server: Python plugins, the project that took me busy during the past two months, has been merged to master and will be available starting with the next QGIS release.

The project has been discussed and approved in Essen during the last QGIS HF (see my presentation about server plugins), thanks to the input and suggestions coming from Marco Hugentobler and Martin Dobias it is now implemented in the more complete and flexible way.

In this article I will introduce the core concepts and the main features of python plugins for QGIS server.

QGIS server plugins architecture

QGIS server provides some core services: WFS, WMS, WCS. What we wanted to achieve was a system to easily add new services and modify existing services through python plugins.

Mi first experiments were limited to a 404 handler that intercepts unhandled requests and hooks into python plugins capturing every stdout output, this was indeed not enough flexible for a full fledged plugins implementation.

The main loop

QGIS server is not different from most web services implementations: it listens for incoming requests, parses the URL query string parameters and returns its output accordingly to the incoming request.

The standard loop before introducing python plugins looked like the following:

• Get the request
  • create GET/POST/SOAP request handler
  • if SERVICE is WMS/WFS/WCS
    • create WMS/WFS/WCS server passing in request handler
      • call server’s executeRequest()
        • call request handler output method
  • else Exception

Plugins come into play

Server python plugins are loaded once when the FCGI application starts and they should register one or more QgsServerFilter (from this point, you might find useful a quick look to the server plugins API docs). Each filter should implement at least one of three callbacks (aka: hooks):

1. requestReady
2. sendResponse
3. responseComplete

All filters have access to the request/response object (QgsRequestHandler) and can manipulate all its properties (input/output) and can raise exceptions (while in a quite particular way as we’ll see below).

Here is a pseudo code showing how and when the filter’s callbacks are called:

• Get the request

  • create GET/POST/SOAP request handler
  • pass request to serverIface
  • call plugins requestReady filters
  • if there is not a response
    • if SERVICE is WMS/WFS/WCS
      • create WMS/WFS/WCS server
        • call server’s executeRequest and possibily call sendResponse plugin filters when streaming output or store the byte stream output and content type in the request handler
    • call plugins responseComplete filters

  • call plugins sendResponse filters

  • request handler output the response

requestReady

This is called when the request is ready: incoming URL and data have been parsed and before entering the core services (WMS, WFS etc.) switch, this is the point where you can manipulate the input and perform actions like:

• authentication/authorization
• redirects
• add/remove certain parameters (typenames for example)
• raise exceptions

You could even substitute a core service completely by changing SERVICE parameter and hence bypassing the core service completely (not that this make much sense though).

 

sendResponse

This is called whenever output is sent to FCGI stdout (and from there, to the client), this is normally done after core services have finished their process and after responseComplete hook was called, but in a few cases XML can become so huge that a streaming XML implementation was needed (WFS GetFeature is one of them), in this case, sendResponse is called multiple times before the response is complete (and before responseComplete is called). The obvious consequence is that sendResponse is normally called once but might be exceptionally called multiple times and in that case (and only in that case) it is also called before responseComplete.

SendResponse is the best place for direct manipulation of core service’s output and while responseComplete is typically also an option, sendResponse is the only viable option  in case of streaming services.

responseComplete

This is called once when core services (if hit) finish their process and the request is ready to be sent to the client. As discussed above, this is  normally called before sendResponse except for streaming services (or other plugin filters) that might have called sendResponse earlier.

responseComplete is the ideal place to provide new services implementation (WPS or custom services) and to perform direct manipulation of the output coming from core services (for example to add a watermark upon a WMS image).

Raising exception from a plugin

Some work has still to be done on this topic: the current implementation can distinguish between handled and unhandled exceptions by setting a QgsRequestHandler property to an instance of QgsMapServiceException, this way the main C++ code can catch handled python exceptions and ignore unhandled exceptions (or better: log them).

This approach basically works but it does not satisfy my pythonic way of handle exceptions: I would rather prefer to raise exceptions from python code to see them bubbling up into C++ loop for being handled there.

Conclusions

The new plugin system is very flexible and allows for basic input/output (i.e. request/response) manipulation and for new services implementation while it remains unobtrusive and has negligible impact on performances, in the next article I will discuss server plugin implementation in depth.

 

See also the second part of this article.

See all QGIS Server related posts

This is the second article about python plugins for QGIS server, see also the introductory article posted a few days ago.

In this post I will introduce the helloServer example plugin that shows some common implementation patterns exploiting the new QGIS Server Python Bindings API.

Server plugins and desktop interfaces

Server plugins can optionally have a desktop interface exactly like all standard QGIS plugins.

A typical use case for a server plugin that also has a desktop interface is to allow the users to configure the server-side of the plugin from QGIS desktop, this is the same principle of configuring WMS/WFS services of QGIS server from the project properties.

The only important difference it that while the WMS/WFS services configuration is stored in the project file itself, the plugins can store and access project data but not to the user’s settings (because the server process normally runs with a different user). For this reason, if you want to share configuration settings between the server and the desktop, provided that you normally run the server with a different user, paths and permissions have to be carefully configured to grant both users access to the shared data.

 

Server configuration

This is an example configuration for Apache, it covers both FCGI and CGI:

  ServerAdmin webmaster@localhost
  # Add an entry to your /etc/hosts file for xxx localhost e.g.
  # 127.0.0.1 xxx
  ServerName xxx
    # Longer timeout for WPS... default = 40
    FcgidIOTimeout 120 
    FcgidInitialEnv LC_ALL "en_US.UTF-8"
    FcgidInitialEnv PYTHONIOENCODING UTF-8
    FcgidInitialEnv LANG "en_US.UTF-8"
    FcgidInitialEnv QGIS_DEBUG 1
    FcgidInitialEnv QGIS_CUSTOM_CONFIG_PATH "/home/xxx/.qgis2/"
    FcgidInitialEnv QGIS_SERVER_LOG_FILE /tmp/qgis.log
    FcgidInitialEnv QGIS_SERVER_LOG_LEVEL 0
    FcgidInitialEnv QGIS_OPTIONS_PATH "/home/xxx/public_html/cgi-bin/"
    FcgidInitialEnv QGIS_PLUGINPATH "/home/xxx/.qgis2/python/plugins"
    FcgidInitialEnv LD_LIBRARY_PATH "/home/xxx/apps/lib"

    # For simple CGI: ignored by fcgid
    SetEnv QGIS_DEBUG 1
    SetEnv QGIS_CUSTOM_CONFIG_PATH "/home/xxx/.qgis2/"
    SetEnv QGIS_SERVER_LOG_FILE /tmp/qgis.log 
    SetEnv QGIS_SERVER_LOG_LEVEL 0
    SetEnv QGIS_OPTIONS_PATH "/home/xxx/public_html/cgi-bin/"
    SetEnv QGIS_PLUGINPATH "/home/xxx/.qgis2/python/plugins"
    SetEnv LD_LIBRARY_PATH "/home/xxx/apps/lib"

    RewriteEngine On
    
        RewriteCond %{HTTP:Authorization} .
        RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
    

  ScriptAlias /cgi-bin/ /home/xxx/apps/bin/
  <Directory "/home/xxx/apps/bin/">
    AllowOverride All
    Options +ExecCGI -MultiViews +FollowSymLinks
    Require all granted  

  ErrorLog ${APACHE_LOG_DIR}/xxx-error.log
  CustomLog ${APACHE_LOG_DIR}/xxx-access.log combined

In this particular example, I’m using a QGIS server built from sources and installed in /home/xxx/apps/bin the libraries are in /home/xxx/apps/lib and LD_LIBRARY_PATH poins to this location.
QGIS_CUSTOM_CONFIG_PATH tells the server where to search for QGIS configuration (for example qgis.db).
QGIS_PLUGINPATH is searched for plugins as start, your server plugins must sit in this directory, while developing you can choose to use the same directory of your QGIS desktop installation.
QGIS_DEBUG set to 1 to enable debug and logging.

Anatomy of a server plugin

For a plugin to be seen as a server plugin, it must provide correct metadata informations and a factory method:

Plugin metadata

A server enabled plugins must advertise itself as a server plugin by adding the line

server=True

in its metadata.txt file.

The serverClassFactory method

A server enabled plugins is basically just a standard QGIS Python plugins that provides a serverClassFactory(serverIface) function in its __init__.py. This function is invoked once when the server starts to generate the plugin instance (it’s called on each request if running in CGI mode: not recommended) and returns a plugin instance:

def serverClassFactory(serverIface):
    from HelloServer import HelloServerServer
    return HelloServerServer(serverIface)

You’ll notice that this is the same pattern we have in “traditional” QGIS plugins.

Server Filters

A server plugin typically consists in one or more callbacks packed into objects called QgsServerFilter.

Each QgsServerFilter implements one or all of the following callbacks:

• requestReady ()
• responseComplete ()
• sendResponse ()

The following example implements a minimal filter which prints HelloServer! in case the SERVICE parameter equals to “HELLO”.

from qgis.server import *
from qgis.core import *

class HelloFilter(QgsServerFilter):

    def __init__(self, serverIface):
        super(HelloFilter, self).__init__(serverIface)    

    def responseComplete(self):        
        request = self.serverInterface().requestHandler()
        params = request.parameterMap()
        if params.get('SERVICE', '').upper() == 'HELLO':
            request.clearHeaders()
            request.setHeader('Content-type', 'text/plain')
            request.clearBody()
            request.appendBody('HelloServer!')

The filters must be registered into the serverIface as in the following example:

class HelloServerServer:
    def __init__(self, serverIface):
        # Save reference to the QGIS server interface
        self.serverIface = serverIface
        serverIface.registerFilter( HelloFilter, 100 )          

The second parameter of registerFilter allows to set a priority which defines the order for the callbacks with the same name (the lower priority is invoked first).

Full control over the flow

By using the three callbacks, plugins can manipulate the input and/or the output of the server in many different ways. In every moment, the plugin instance has access to the QgsRequestHandler through the QgsServerInterface, the QgsRequestHandler has plenty of methods that can be used to alter the input parameters before entering the core processing of the server (by using requestReady) or after the request has been processed by the core services (by using sendResponse).

The following examples cover some common use cases:

Modifying the input

The example plugin contains a test example that changes input parameters coming from the query string, in this example a new parameter is injected into the (already parsed) parameterMap, this parameter is then visible by core services (WMS etc.), at the end of core services processing we check that the parameter is still there.

from qgis.server import *
from qgis.core import *

class ParamsFilter(QgsServerFilter):

    def __init__(self, serverIface):
        super(ParamsFilter, self).__init__(serverIface)

    def requestReady(self):
        request = self.serverInterface().requestHandler()
        params = request.parameterMap( )
        request.setParameter('TEST_NEW_PARAM', 'ParamsFilter')

    def responseComplete(self):
        request = self.serverInterface().requestHandler()
        params = request.parameterMap( )
        if params.get('TEST_NEW_PARAM') == 'ParamsFilter':
            QgsMessageLog.logMessage("SUCCESS - ParamsFilter.responseComplete", 'plugin', QgsMessageLog.INFO)
        else:
            QgsMessageLog.logMessage("FAIL    - ParamsFilter.responseComplete", 'plugin', QgsMessageLog.CRITICAL)

This is an extract of what you see in the log file:

src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] HelloServerServer - loading filter ParamsFilter
src/core/qgsmessagelog.cpp: 45: (logMessage) [1ms] 2014-12-12T12:39:29 Server[0] Server plugin HelloServer loaded!
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 Server[0] Server python plugins loaded
src/mapserver/qgsgetrequesthandler.cpp: 35: (parseInput) [0ms] query string is: SERVICE=HELLO&request=GetOutput
src/mapserver/qgshttprequesthandler.cpp: 547: (requestStringToParameterMap) [1ms] inserting pair SERVICE // HELLO into the parameter map
src/mapserver/qgshttprequesthandler.cpp: 547: (requestStringToParameterMap) [0ms] inserting pair REQUEST // GetOutput into the parameter map
src/mapserver/qgsserverfilter.cpp: 42: (requestReady) [0ms] QgsServerFilter plugin default requestReady called
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] HelloFilter.requestReady
src/mapserver/qgis_map_serv.cpp: 235: (configPath) [0ms] Using default configuration file path: /home/xxx/apps/bin/admin.sld
src/mapserver/qgshttprequesthandler.cpp: 49: (setHttpResponse) [0ms] Checking byte array is ok to set...
src/mapserver/qgshttprequesthandler.cpp: 59: (setHttpResponse) [0ms] Byte array looks good, setting response...
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] HelloFilter.responseComplete
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] SUCCESS - ParamsFilter.responseComplete
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] RemoteConsoleFilter.responseComplete
src/mapserver/qgshttprequesthandler.cpp: 158: (sendResponse) [0ms] Sending HTTP response
src/core/qgsmessagelog.cpp: 45: (logMessage) [0ms] 2014-12-12T12:39:29 plugin[0] HelloFilter.sendResponse

On line 13 the “SUCCESS” string indicates that the plugin passed the test.

The same technique can be exploited to use a custom service instead of a core one: you could for example skip a WFS SERVICE request or any other core request just by changing the SERVICE parameter to something different and the core service will be skipped, then you can inject your custom results into the output and send them to the client (this is explained here below).

Changing or replacing the output

The watermark filter example shows how to replace the WMS output with a new image obtained by adding a watermark image on the top of the WMS image generated by the WMS core service:

import os

from qgis.server import *
from qgis.core import *
from PyQt4.QtCore import *
from PyQt4.QtGui import *


class WatermarkFilter(QgsServerFilter):

    def __init__(self, serverIface):
        super(WatermarkFilter, self).__init__(serverIface)

    def responseComplete(self):
        request = self.serverInterface().requestHandler()
        params = request.parameterMap( )
        # Do some checks
        if (request.parameter('SERVICE').upper() == 'WMS' \
                and request.parameter('REQUEST').upper() == 'GETMAP' \
                and not request.exceptionRaised() ):
            QgsMessageLog.logMessage("WatermarkFilter.responseComplete: image ready %s" % request.infoFormat(), 'plugin', QgsMessageLog.INFO)
            # Get the image
            img = QImage()
            img.loadFromData(request.body())
            # Adds the watermark
            watermark = QImage(os.path.join(os.path.dirname(__file__), 'media/watermark.png'))
            p = QPainter(img)
            p.drawImage(QRect( 20, 20, 40, 40), watermark)
            p.end()
            ba = QByteArray()
            buffer = QBuffer(ba)
            buffer.open(QIODevice.WriteOnly)
            img.save(buffer, "PNG")
            # Set the body
            request.clearBody()
            request.appendBody(ba)

In this example the SERVICE parameter value is checked and if the incoming request is a WMS GETMAP and no exceptions have been set by a previously executed plugin or by the core service (WMS in this case), the WMS generated image is retrieved from the output buffer and the watermark image is added. The final step is to clear the output buffer and replace it with the newly generated image. Please note that in a real-world situation we should also check for the requested image type instead of returning PNG in any case.

The power of python

The examples above are just meant to explain how to interact with QGIS server python bindings but server plugins have full access to all QGIS python bindings and to thousands of python libraries, what you can do with python server plugins is just limited by your imagination!

 

See all QGIS Server related posts

Prerequisites

I assume that you are working on a fresh install with Apache and FCGI module installed with:

$ sudo apt-get install apache2 libapache2-mod-fcgid
$ # Enable FCGI daemon apache module
$ sudo a2enmod fcgid

Package installation

First step is to add debian gis repository, add the following repository:

$ cat /etc/apt/sources.list.d/debian-gis.list
deb http://qgis.org/debian trusty main
deb-src http://qgis.org/debian trusty main

$ # Add keys
$ sudo gpg --recv-key DD45F6C3
$ sudo gpg --export --armor DD45F6C3 | sudo apt-key add -

$ # Update package list
$ sudo apt-get update && sudo apt-get upgrade

Now install qgis server:

$ sudo apt-get install qgis-server python-qgis

Install the HelloWorld example plugin

This is an example plugin and should not be used in production!
Create a directory to hold server plugins, you can choose whatever path you want, it will be specified in the virtual host configuration and passed on to the server through an environment variable:

$ sudo mkdir -p /opt/qgis-server/plugins
$ cd /opt/qgis-server/plugins
$ sudo wget https://github.com/elpaso/qgis-helloserver/archive/master.zip
$ # In case unzip was not installed before:
$ sudo apt-get install unzip
$ sudo unzip master.zip 
$ sudo mv qgis-helloserver-master HelloServer

Apache virtual host configuration

We are installing the server in a separate virtual host listening on port 81.
Rewrite module can be optionally enabled to pass HTTP BASIC auth headers (only needed by the HelloServer example plugin).

$ sudo a2enmod rewrite

Let Apache listen to port 81:

$ cat /etc/apache2/conf-available/qgis-server-port.conf
Listen 81
$ sudo a2enconf qgis-server-port

The virtual host configuration, stored in /etc/apache2/sites-available/001-qgis-server.conf:

<VirtualHost *:81>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/qgis-server-error.log
    CustomLog ${APACHE_LOG_DIR}/qgis-server-access.log combined

    # Longer timeout for WPS... default = 40
    FcgidIOTimeout 120 
    FcgidInitialEnv LC_ALL "en_US.UTF-8"
    FcgidInitialEnv PYTHONIOENCODING UTF-8
    FcgidInitialEnv LANG "en_US.UTF-8"
    FcgidInitialEnv QGIS_DEBUG 1
    FcgidInitialEnv QGIS_SERVER_LOG_FILE /tmp/qgis-000.log
    FcgidInitialEnv QGIS_SERVER_LOG_LEVEL 0
    FcgidInitialEnv QGIS_PLUGINPATH "/opt/qgis-server/plugins"

    # ABP: needed for QGIS HelloServer plugin HTTP BASIC auth
    <IfModule mod_fcgid.c>
        RewriteEngine on
        RewriteCond %{HTTP:Authorization} .
        RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
    </IfModule>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
        AllowOverride All
        Options +ExecCGI -MultiViews +FollowSymLinks
        Require all granted
        #Allow from all
  </Directory>
</VirtualHost>

Enable the virtual host and restart Apache:

$ sudo a2ensite 001-qgis-server
$ sudo service apache2 restart

Test:

$ wget -q -O - "http://localhost:81/cgi-bin/qgis_mapserv.fcgi?SERVICE=HELLO"
HelloServer!

See all QGIS Server related posts