Related Plugins and Tags

QGIS Planet

qgis2img - A QGIS render benchmarking tool and image renderer

qgis2img is a new tool that I created, in a bit of friendly competition with the boss, which I lost but we will not speak of that again, for benchmarking QGIS layer rendering. The goal is simple. Take a project file(s), or QLR file(s), render the output, time the results, and dump a summary. Simples. The tool does 3 passes by default to get the average but can do more. It's nothing fancy. Written in Python so it can be evolved quickly.

qgis2img will render each image by itself to give single timings then it will render the whole project as you see in QGIS.

It uses QGIS 2.4 (qgis-dev) in order to use the new rendering methods. I don't have any plans to port it to work with QGIS 2.2, however feel free to send a pull request.

The usage is pretty simple:

usage: qgis2img [-h] [--size SIZE] [--passes PASSES] [--types TYPES] file

Benchmark QGIS project file and layer loading times

positional arguments:
  file             Project file to load into QGIS

optional arguments:
  -h, --help       show this help message and exit
  --size SIZE      Image output size
  --passes PASSES  Number of render passes per layer
  --types TYPES    What to render. Options are layer|project, layer, or project.
                   layer|project will render all layers as the if the project
                   is open in QGIS.

with the results:

$ python.exe qgis2img parcels.qgs --passes 5
Project Loaded with: [u'PARCEL_region - Shp', u'PARCEL_region - Spatialite']
Rendering images with 5 passes
Layer: PARCEL_region - Shp      4.907 sec
Layer: PARCEL_region - Spatialite       3.66 sec
Layer: Project     5.3378 sec

Easy.

It will generate an image for each layer and the project:

Alt Text

You can find the project at https://github.com/DMS-Aus/qgis2img

Pull requests and ideas welcome

qgisbench

There is a tool called qgisbench in the QGIS source tree that does this kind of thing too, however:

  • It's in C++
  • We don't ship it
  • It's in C++
  • <3 Python
  • These things are good examples for others
  • Using the Python API in this ways lets me see gaps

qgis2img - A QGIS render benchmarking tool and image renderer

qgis2img is a new tool that I created, in a bit of friendly competition with the boss, which I lost but we will not speak of that again, for benchmarking QGIS layer rendering. The goal is simple. Take a project file(s), or QLR file(s), render the output, time the results, and dump a summary. Simples. The tool does 3 passes by default to get the average but can do more. It's nothing fancy. Written in Python so it can be evolved quickly.

qgis2img will render each image by itself to give single timings then it will render the whole project as you see in QGIS.

It uses QGIS 2.4 (qgis-dev) in order to use the new rendering methods. I don't have any plans to port it to work with QGIS 2.2, however feel free to send a pull request.

The usage is pretty simple:

usage: qgis2img [-h] [--size SIZE] [--passes PASSES] [--types TYPES] file

Benchmark QGIS project file and layer loading times

positional arguments:
  file             Project file to load into QGIS

optional arguments:
  -h, --help       show this help message and exit
  --size SIZE      Image output size
  --passes PASSES  Number of render passes per layer
  --types TYPES    What to render. Options are layer|project, layer, or project.
                   layer|project will render all layers as the if the project
                   is open in QGIS.

with the results:

$ python.exe qgis2img parcels.qgs --passes 5
Project Loaded with: [u'PARCEL_region - Shp', u'PARCEL_region - Spatialite']
Rendering images with 5 passes
Layer: PARCEL_region - Shp      4.907 sec
Layer: PARCEL_region - Spatialite       3.66 sec
Layer: Project     5.3378 sec

Easy.

It will generate an image for each layer and the project:

qgis2img.png

You can find the project at https://github.com/DMS-Aus/qgis2img

Pull requests and ideas welcome

qgisbench

There is a tool called qgisbench in the QGIS source tree that does this kind of thing too, however:

  • It's in C++
  • We don't ship it
  • It's in C++
  • <3 Python
  • These things are good examples for others
  • Using the Python API in this ways lets me see gaps

The QGIS Field calculator is dead. Long live the Field calculator bar

Ahhh the good old field calculator, it's in a better place now. OK not really, it's still there if you need it, but we can do a little better in 2.4. Introducing the field calculator bar:

Alt Text

oooo fancy.

The field calculator has always bugged me, I think it was just the combination of a few things:

  • It's modal so it blocks you from doing anything else - this alone is motivation enough in my mind.
  • You have to do the Open - Run - Close - Open - Run - Close dance which isn't great - annoying to say the least.
  • Did I mention it's modal - AAAAAAAHHHHHHHHHHH
  • Defaults to creating a new field - which is the edge case
  • You only have All or Selection, which is a bit limiting

Anyway, enough with that. Last night I was having a chat to Nyall about something unrelated, and while looking at Excel I thought about that little bar at the top and how handy that is. You don't see a field calculator dialog in Excel - well there is one but not for the common case - you just wack in your expression and it does its thing. Why couldn't we have this for QGIS? I think I said to Nyall "you know this would be pretty cool, I might give it a go". Couple of hours later and this is it.

Alt Text

I have expressed in the past, and above, my hate for modal dialogs, so that was the main motivation and the results are much nicer then before.

What do we gain:

  • Not modal - WIN!
  • Don't have to close anything to see your results
  • See the results as soon as you run Update (All|Filtered)
  • Works on the features in table (All|Filtered)
  • Does one job

The other improvment to the old dialog is what features the bar works on. The bar will update what it sees in the dialog. If you need to update just the selection, simply select Show Selected and run the update. Need to search for something to update? Run a filter and press update. The method has changed from All and Selected to All and Filtered. Just remember if you see it in the attribute table it will be updated.

Alt Text

The last point is important too, if you need a new field you use the New Field button, then run the update, there is no need to mix the two function into one tool. SRP.

This feature will be in 2.4. If you find any bugs assign them to me at hub.qgis.org and I will try to address them before 2.4 is out.

RIP Field calculator dialog

The QGIS Field calculator is dead. Long live the Field calculator bar

Ahhh the good old field calculator, it's in a better place now. OK not really, it's still there if you need it, but we can do a little better in 2.4. Introducing the field calculator bar:

fieldcalc1.png

oooo fancy.

The field calculator has always bugged me, I think it was just the combination of a few things:

  • It's modal so it blocks you from doing anything else - this alone is motivation enough in my mind.
  • You have to do the Open - Run - Close - Open - Run - Close dance which isn't great - annoying to say the least.
  • Did I mention it's modal - AAAAAAAHHHHHHHHHHH
  • Defaults to creating a new field - which is the edge case
  • You only have All or Selection, which is a bit limiting

Anyway, enough with that. Last night I was having a chat to Nyall about something unrelated, and while looking at Excel I thought about that little bar at the top and how handy that is. You don't see a field calculator dialog in Excel - well there is one but not for the common case - you just wack in your expression and it does its thing. Why couldn't we have this for QGIS? I think I said to Nyall "you know this would be pretty cool, I might give it a go". Couple of hours later and this is it.

fieldcalc.png

I have expressed in the past, and above, my hate for modal dialogs, so that was the main motivation and the results are much nicer then before.

What do we gain:

  • Not modal - WIN!
  • Don't have to close anything to see your results
  • See the results as soon as you run Update (All|Filtered)
  • Works on the features in table (All|Filtered)
  • Does one job

The other improvment to the old dialog is what features the bar works on. The bar will update what it sees in the dialog. If you need to update just the selection, simply select Show Selected and run the update. Need to search for something to update? Run a filter and press update. The method has changed from All and Selected to All and Filtered. Just remember if you see it in the attribute table it will be updated.

fieldcalc-filter.png

The last point is important too, if you need a new field you use the New Field button, then run the update, there is no need to mix the two function into one tool. SRP.

This feature will be in 2.4. If you find any bugs assign them to me at hub.qgis.org and I will try to address them before 2.4 is out.

RIP Field calculator dialog

The QGIS Field calculator is dead. Long live the Field calculator bar

Ahhh the good old field calculator, it's in a better place now. OK not really, it's still there if you need it, but we can do a little better in 2.4. Introducing the field calculator bar:

Alt Text

oooo fancy.

The field calculator has always bugged me, I think it was just the combination of a few things:

  • It's modal so it blocks you from doing anything else - this alone is motivation enough in my mind.
  • You have to do the Open - Run - Close - Open - Run - Close dance which isn't great - annoying to say the least.
  • Did I mention it's modal - AAAAAAAHHHHHHHHHHH
  • Defaults to creating a new field - which is the edge case
  • You only have All or Selection, which is a bit limiting

Anyway, enough with that. Last night I was having a chat to Nyall about something unrelated, and while looking at Excel I thought about that little bar at the top and how handy that is. You don't see a field calculator dialog in Excel - well there is one but not for the common case - you just wack in your expression and it does its thing. Why couldn't we have this for QGIS? I think I said to Nyall "you know this would be pretty cool, I might give it a go". Couple of hours later and this is it.

Alt Text

I have expressed in the past, and above, my hate for modal dialogs, so that was the main motivation and the results are much nicer then before.

What do we gain:

  • Not modal - WIN!
  • Don't have to close anything to see your results
  • See the results as soon as you run Update (All|Filtered)
  • Works on the features in table (All|Filtered)
  • Does one job

The other improvment to the old dialog is what features the bar works on. The bar will update what it sees in the dialog. If you need to update just the selection, simply select Show Selected and run the update. Need to search for something to update? Run a filter and press update. The method has changed from All and Selected to All and Filtered. Just remember if you see it in the attribute table it will be updated.

Alt Text

The last point is important too, if you need a new field you use the New Field button, then run the update, there is no need to mix the two function into one tool. SRP.

This feature will be in 2.4. If you find any bugs assign them to me at hub.qgis.org and I will try to address them before 2.4 is out.

RIP Field calculator dialog

Setting up PyCharm for PyQGIS and Qt

I have been asked a few times how to setup PyCharm so you are able to do PyQGIS development, or even PyQt because that is just as great. Rather then tell each person one at a time I thought I would throw it out as a blog post so everyone gets the benift.

The first thing we need to do is create a batch file that will load PyCharm and setup all the paths correctly. We have to do this on Windows as Qt and QGIS are not on PATH. QGIS also ships with it's own version of copy of Python so we need to tell PyCharm about it.

The batch file is as simple as this:

SET OSGEO4W_ROOT=C:\OSGeo4W
SET QGISNAME=qgis
SET QGIS=%OSGEO4W_ROOT%\apps\%QGISNAME%
SET QGIS_PREFIX_PATH=%QGIS%
SET PYCHARM="C:\Program Files (x86)\JetBrains\PyCharm 3.0\bin\pycharm.exe"

CALL %OSGEO4W_ROOT%\bin\o4w_env.bat

SET PATH=%PATH%;%QGIS%\bin
SET PYTHONPATH=%QGIS%\python;%PYTHONPATH%

start "PyCharm aware of QGIS" /B %PYCHARM% %*

Save this somewhere called pycharm-pyqgis.bat and run it.

This is a pretty basic batch file. It just sets the variables that we need for the QGIS and Qt libs, and also sets the PYTHONHOME to the QGIS version. The magic sause here is the set PATH, set PYTHONHOM, and set PYTHONPATH variables. You can just update the OSGeo4W_ROOT, and PYCHARM variables for your setup.

After running the batch we need to setup a Python interpreter in PyCharm. Click Configure -> Settings on the load page (or settings in the File menu). Search for Python Interpreters, and press the green add button and select local. Here we need to add the Python interpreter that we setup in our batch file. In the one I posted above it will be found at C:\OSGeo4W\bin\python.exe. Press Ok and PyCharm will find all the python paths it needs for the setup.

Alt Text

Leave the settings dialog and create a new project.

Alt Text

Remember to select the interpreter that we setup for the installed version of QGIS.

That is pretty much it. We can now create a pyqgis and pyqt app in PyCharm.

Lets give it a go

from qgis.core import QgsApplication
from PyQt4.QtGui import QDialog

GUIEnabled=True
app = QgsApplication([], GUIEnabled)

dlg = QDialog()
dlg.exec_()

app.exit(app.exec_())

Run it in PyCharm with Alt+Shift+F10. Good to go!

Setting up PyCharm for PyQGIS and Qt

I have been asked a few times how to setup PyCharm so you are able to do PyQGIS development, or even PyQt because that is just as great. Rather then tell each person one at a time I thought I would throw it out as a blog post so everyone gets the benift.

The first thing we need to do is create a batch file that will load PyCharm and setup all the paths correctly. We have to do this on Windows as Qt and QGIS are not on PATH. QGIS also ships with it's own version of copy of Python so we need to tell PyCharm about it.

The batch file is as simple as this:

SET OSGEO4W_ROOT=C:\OSGeo4W
SET QGISNAME=qgis
SET QGIS=%OSGEO4W_ROOT%\apps\%QGISNAME%
SET QGIS_PREFIX_PATH=%QGIS%
SET PYCHARM="C:\Program Files (x86)\JetBrains\PyCharm 3.0\bin\pycharm.exe"

CALL %OSGEO4W_ROOT%\bin\o4w_env.bat

SET PATH=%PATH%;%QGIS%\bin
SET PYTHONPATH=%QGIS%\python;%PYTHONPATH%

start "PyCharm aware of QGIS" /B %PYCHARM% %*

Save this somewhere called pycharm-pyqgis.bat and run it.

This is a pretty basic batch file. It just sets the variables that we need for the QGIS and Qt libs, and also sets the PYTHONHOME to the QGIS version. The magic sause here is the set PATH, set PYTHONHOM, and set PYTHONPATH variables. You can just update the OSGeo4W_ROOT, and PYCHARM variables for your setup.

After running the batch we need to setup a Python interpreter in PyCharm. Click Configure -> Settings on the load page (or settings in the File menu). Search for Python Interpreters, and press the green add button and select local. Here we need to add the Python interpreter that we setup in our batch file. In the one I posted above it will be found at C:\OSGeo4W\bin\python.exe. Press Ok and PyCharm will find all the python paths it needs for the setup.

Alt Text

Leave the settings dialog and create a new project.

Alt Text

Remember to select the interpreter that we setup for the installed version of QGIS.

That is pretty much it. We can now create a pyqgis and pyqt app in PyCharm.

Lets give it a go

from qgis.core import QgsApplication
from PyQt4.QtGui import QDialog

GUIEnabled=True
app = QgsApplication([], GUIEnabled)

dlg = QDialog()
dlg.exec_()

app.exit(app.exec_())

Run it in PyCharm with Alt+Shift+F10. Good to go!

Setting up PyCharm for PyQGIS and Qt

I have been asked a few times how to setup PyCharm so you are able to do PyQGIS development, or even PyQt because that is just as great. Rather then tell each person one at a time I thought I would throw it out as a blog post so everyone gets the benift.

The first thing we need to do is create a batch file that will load PyCharm and setup all the paths correctly. We have to do this on Windows as Qt and QGIS are not on PATH. QGIS also ships with it's own version of copy of Python so we need to tell PyCharm about it.

The batch file is as simple as this:

SET OSGEO4W_ROOT=C:\OSGeo4W
SET QGISNAME=qgis
SET QGIS=%OSGEO4W_ROOT%\apps\%QGISNAME%
SET QGIS_PREFIX_PATH=%QGIS%
SET PYCHARM="C:\Program Files (x86)\JetBrains\PyCharm 3.0\bin\pycharm.exe"

CALL %OSGEO4W_ROOT%\bin\o4w_env.bat

SET PATH=%PATH%;%QGIS%\bin
SET PYTHONPATH=%QGIS%\python;%PYTHONPATH%

start "PyCharm aware of QGIS" /B %PYCHARM% %*

Save this somewhere called pycharm-pyqgis.bat and run it.

This is a pretty basic batch file. It just sets the variables that we need for the QGIS and Qt libs, and also sets the PYTHONHOME to the QGIS version. The magic sause here is the set PATH, set PYTHONHOM, and set PYTHONPATH variables. You can just update the OSGeo4W_ROOT, and PYCHARM variables for your setup.

After running the batch we need to setup a Python interpreter in PyCharm. Click Configure -> Settings on the load page (or settings in the File menu). Search for Python Interpreters, and press the green add button and select local. Here we need to add the Python interpreter that we setup in our batch file. In the one I posted above it will be found at C:\OSGeo4W\bin\python.exe. Press Ok and PyCharm will find all the python paths it needs for the setup.

pycharm_python.png

Leave the settings dialog and create a new project.

pycharm_newproject.png

Remember to select the interpreter that we setup for the installed version of QGIS.

That is pretty much it. We can now create a pyqgis and pyqt app in PyCharm.

Lets give it a go

from qgis.core import QgsApplication
from PyQt4.QtGui import QDialog

GUIEnabled=True
app = QgsApplication([], GUIEnabled)

dlg = QDialog()
dlg.exec_()

app.exit(app.exec_())

Run it in PyCharm with Alt+Shift+F10. Good to go!

What are all these QGIS file type. And why do I need them

After I added the new QGIS Layer Definition feature in QGIS I have noticed some indirect feedback regarding its use. I thought I would do this post in order to clear things up regarding the new feature so that we are all on the same page.

Some of this feedback, or I could say confusion, was mainly "Why not use the qml format?" or "How is it different from a project file". I can understand this confusion because they do kind of look and act the same however they are a little different.

I'm going to work my way up from the bottom up in terms of file levels and their use.

The QML file (.qml)

It contains: Style information

The QML (.qml) file in QGIS is a style file. It contains an export of the style information, including labels, from a layer. The .qml file however has no reference to the layer. This means I can share a .qml file with you and you can apply it to your own data without needing the data I exported it from. QML files are handy if you have one, or more, layers but have a collection of different style that you want to apply to them.

Style files can only be only be applied to a layer once it is opened. The recent Ordance Survey release of styles is a prefect QML use case.

The Layer Definition file (.qlr)

It contains: Layer source pointer + Style information

This newest feature, also in plugin form for 2.0 and 2.2, is a Layer Definition file. This file contains the reference - or pointer - to the data source plus any style information. This is like a the ArcGIS .lyr file, although maybe not as fully feature rich just yet. The use case for this file is simple: To have a single file to can open a data source bringing in all the related style information. These files also allow you to mask the underlying datasource in a easy to open file.

One of my use cases is to open MS SQL layers. Rather then having to go to the MS SQL connection dialog, connect, select and load, then style. I can simply add a .qlr file that points to the correct MS SQL layer pre styled.

In the future a .qlr file may hold a reference to more then one layer. The Ordance Survey or Natural Earth stuff could also be done with a QLR in order too allow just opening a single file.

The Project file (.qgs)

It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff

The Project file is the highest level file that QGIS has. This holds more then just a list of layers, it also holds order, groups, composers, marcos, etc. I don't really need to explain any of this because you already know.

Why not just extend QML (style file) or QGS (project file)?

In software development we have this thing called SRP (Single Responsibility Principle), and while it can be tricky to follow at times I think it can also apply in cases like this. Sometimes have a single file meaning many different things can be worse then having a many different files with one use case each.

Could I have extended QML to support datasource too? Sure. Easy. However I don't think it made any sense. The use case of the QML is to store only the style and to be applied after the layer is loaded. Who says I have the datasource you used in the first place? In this case the QML is the most portable format because it doesn't point to a data source. I could have just stored the layer reference and ignored it in the normal places we import QML, yes but now you start to muddy the water on what a QML is. If your answer to "What is a QML file?" is "Well sometimes it opens a layer, and sometimes it's just the style" that to me is a smell.

The project file stores a lot of different things and I could have used that as a base, but again I think it starts to muddy the waters. The project file also contains a lot of other things that we just don't need, composers, etc. I have started working on a Import from Project function, just like the plugin of the same function, however I don't think this process is a smooth as using something like a QLR file to bring in layers. Giving someone a full project just so then can import a single layer from it feels like giving someone a full toolbox when all they want is a screwdriver. Just give me the screwdriver, I don't care for the other stuff.

What are all these QGIS file types? Why do I need them

After I added the new QGIS Layer Definition feature in QGIS I have noticed some indirect feedback regarding its use. I thought I would do this post in order to clear things up regarding the new feature so that we are all on the same page.

Some of this feedback, or I could say confusion, was mainly "Why not use the qml format?" or "How is it different from a project file". I can understand this confusion because they do kind of look and act the same however they are a little different.

I'm going to work my way up from the bottom up in terms of file levels and their use.

The QML file (.qml)

It contains: Style information

The QML (.qml) file in QGIS is a style file. It contains an export of the style information, including labels, from a layer. The .qml file however has no reference to the layer. This means I can share a .qml file with you and you can apply it to your own data without needing the data I exported it from. QML files are handy if you have one, or more, layers but have a collection of different style that you want to apply to them.

Style files can only be only be applied to a layer once it is opened. The recent Ordance Survey release of styles is a prefect QML use case.

The Layer Definition file (.qlr)

It contains: Layer source pointer + Style information

This newest feature, also in plugin form for 2.0 and 2.2, is a Layer Definition file. This file contains the reference - or pointer - to the data source plus any style information. This is like a the ArcGIS .lyr file, although maybe not as fully feature rich just yet. The use case for this file is simple: To have a single file to can open a data source bringing in all the related style information. These files also allow you to mask the underlying datasource in a easy to open file.

One of my use cases is to open MS SQL layers. Rather then having to go to the MS SQL connection dialog, connect, select and load, then style. I can simply add a .qlr file that points to the correct MS SQL layer pre styled.

In the future a .qlr file may hold a reference to more then one layer. The Ordance Survey or Natural Earth stuff could also be done with a QLR in order too allow just opening a single file.

The Project file (.qgs)

It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff

The Project file is the highest level file that QGIS has. This holds more then just a list of layers, it also holds order, groups, composers, marcos, etc. I don't really need to explain any of this because you already know.

Why not just extend QML (style file) or QGS (project file)?

In software development we have this thing called SRP (Single Responsibility Principle), and while it can be tricky to follow at times I think it can also apply in cases like this. Sometimes have a single file meaning many different things can be worse then having a many different files with one use case each.

Could I have extended QML to support datasource too? Sure. Easy. However I don't think it made any sense. The use case of the QML is to store only the style and to be applied after the layer is loaded. Who says I have the datasource you used in the first place? In this case the QML is the most portable format because it doesn't point to a data source. I could have just stored the layer reference and ignored it in the normal places we import QML, yes but now you start to muddy the water on what a QML is. If your answer to "What is a QML file?" is "Well sometimes it opens a layer, and sometimes it's just the style" that to me is a smell.

The project file stores a lot of different things and I could have used that as a base, but again I think it starts to muddy the waters. The project file also contains a lot of other things that we just don't need, composers, etc. I have started working on a Import from Project function, just like the plugin of the same function, however I don't think this process is a smooth as using something like a QLR file to bring in layers. Giving someone a full project just so then can import a single layer from it feels like giving someone a full toolbox when all they want is a screwdriver. Just give me the screwdriver, I don't care for the other stuff.

What are all these QGIS file types? Why do I need them

After I added the new QGIS Layer Definition feature in QGIS I have noticed some indirect feedback regarding its use. I thought I would do this post in order to clear things up regarding the new feature so that we are all on the same page.

Some of this feedback, or I could say confusion, was mainly "Why not use the qml format?" or "How is it different from a project file". I can understand this confusion because they do kind of look and act the same however they are a little different.

I'm going to work my way up from the bottom up in terms of file levels and their use.

The QML file (.qml)

It contains: Style information

The QML (.qml) file in QGIS is a style file. It contains an export of the style information, including labels, from a layer. The .qml file however has no reference to the layer. This means I can share a .qml file with you and you can apply it to your own data without needing the data I exported it from. QML files are handy if you have one, or more, layers but have a collection of different style that you want to apply to them.

Style files can only be only be applied to a layer once it is opened. The recent Ordance Survey release of styles is a prefect QML use case.

The Layer Definition file (.qlr)

It contains: Layer source pointer + Style information

This newest feature, also in plugin form for 2.0 and 2.2, is a Layer Definition file. This file contains the reference - or pointer - to the data source plus any style information. This is like a the ArcGIS .lyr file, although maybe not as fully feature rich just yet. The use case for this file is simple: To have a single file to can open a data source bringing in all the related style information. These files also allow you to mask the underlying datasource in a easy to open file.

One of my use cases is to open MS SQL layers. Rather then having to go to the MS SQL connection dialog, connect, select and load, then style. I can simply add a .qlr file that points to the correct MS SQL layer pre styled.

In the future a .qlr file may hold a reference to more then one layer. The Ordance Survey or Natural Earth stuff could also be done with a QLR in order too allow just opening a single file.

The Project file (.qgs)

It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff

The Project file is the highest level file that QGIS has. This holds more then just a list of layers, it also holds order, groups, composers, marcos, etc. I don't really need to explain any of this because you already know.

Why not just extend QML (style file) or QGS (project file)?

In software development we have this thing called SRP (Single Responsibility Principle), and while it can be tricky to follow at times I think it can also apply in cases like this. Sometimes have a single file meaning many different things can be worse then having a many different files with one use case each.

Could I have extended QML to support datasource too? Sure. Easy. However I don't think it made any sense. The use case of the QML is to store only the style and to be applied after the layer is loaded. Who says I have the datasource you used in the first place? In this case the QML is the most portable format because it doesn't point to a data source. I could have just stored the layer reference and ignored it in the normal places we import QML, yes but now you start to muddy the water on what a QML is. If your answer to "What is a QML file?" is "Well sometimes it opens a layer, and sometimes it's just the style" that to me is a smell.

The project file stores a lot of different things and I could have used that as a base, but again I think it starts to muddy the waters. The project file also contains a lot of other things that we just don't need, composers, etc. I have started working on a Import from Project function, just like the plugin of the same function, however I don't think this process is a smooth as using something like a QLR file to bring in layers. Giving someone a full project just so then can import a single layer from it feels like giving someone a full toolbox when all they want is a screwdriver. Just give me the screwdriver, I don't care for the other stuff.

QGIS Layer Definitions (An ArcGIS like .lyr function for QGIS)

NOTE Just to clarify. This is not adding the ability to open ArcGIS .lyr files.

One of the reasons that I love working and being part of the QGIS project is how quickly you can take it from not having a needed feature to having said feature. A good example of this is a recent addition I added related to having a ArcGIS .lyr like feature. For those who don't use Arc, the .lyr file is a pretty much a file that points to the data, contains all the styling, and other information, you can then just add this file and it will do all the other magic for you.

A quote from the ArcGIS page on the topic

A layer can exist outside your map as a layer file (.lyr). This makes it easy for others to access the layers you've built. You can share layers over the network and by e-mail. When users add a layer file to their maps, it will draw exactly as it was saved as long as they can get access to the data referenced by the layer. A common way that users help support this is to use relative paths to each layer's data source.

A question came up on GIS.SE asking Does QGIS have the equivalent of ArcGIS's Layer (*.LYR) file?. To which the answer was. "Not really". Which of course was true. At the time.

That got me thinking. Why couldn't we have a feature like this? It would be handy to have as it would let you create layer shortcuts and the ability to a load layers quickly from a single place without worrying about were the data comes from. The XML that we need to recreate the layer on project load was already stored in the project file under the maplayer node. It already has everything we need we can just write that out to disk and create a new file type. After some digging into the API I found QgsMapLayer::writeLayerXML() and Bingo! .qlr was born.

Because I personally, and some others I talked to, didn't really like just "Save as Layer File.." as it sounds a bit vauge, we picked "QGIS Layer Definition file".

A QGIS Layer Deinition File is simply the XML maplayer node, with the id removed, saved to disk as a .qlr format.

Saving one is as simple as Right Click Layer -> Save As Layer Definition File..

Alt Text

and loading Layer -> Add from Layer Definition File...

Alt Text

Simple!

The layer file will store any of the styles, labels, edit widgets, etc, that you have defined for that layer so it's just a matter of adding the file and you are good to go.

Like I said at the start it's simply the XML maplayer node, so if you move the data or want to change the datasource just open the file in a text editor and change it.

<!DOCTYPE qgis-layer-definition>
<maplayer minimumScale="-4.65661e-10" maximumScale="1e+08" simplifyDrawingHints="1" minLabelScale="0" maxLabelScale="1e+08" simplifyDrawingTol="1" geometry="Polygon" simplifyMaxScale="1" type="vector" hasScaleBasedVisibilityFlag="0" simplifyLocal="1" scaleBasedLabelVisibilityFlag="0">
  <datasource>F:/gis_data/cadastre.shp</datasource>
  <title></title>
  <abstract></abstract>
  <keywordList>
    <value></value>
  </keywordList>
  <layername>cadastre</layername>
  ... {style, etc}
</maplayer

This will work with any layer supported in QGIS, expect special plugin layers.

I will add loading from the browser dock before the release aswell once I get some time.

Can we get it in an older QGIS version?

All the methods that I used for this feature are exposed to the Python API. Once I get some free time I plan on making a plugin that is aimed at QGIS <2.3 to give those users the same feature. Keep an eye out for it in the plugin installer.

QGIS Layer Definitions (An ArcGIS like .lyr function for QGIS)

NOTE Just to clarify. This is not adding the ability to open ArcGIS .lyr files.

One of the reasons that I love working and being part of the QGIS project is how quickly you can take it from not having a needed feature to having said feature. A good example of this is a recent addition I added related to having a ArcGIS .lyr like feature. For those who don't use Arc, the .lyr file is a pretty much a file that points to the data, contains all the styling, and other information, you can then just add this file and it will do all the other magic for you.

A quote from the ArcGIS page on the topic

A layer can exist outside your map as a layer file (.lyr). This makes it easy for others to access the layers you've built. You can share layers over the network and by e-mail. When users add a layer file to their maps, it will draw exactly as it was saved as long as they can get access to the data referenced by the layer. A common way that users help support this is to use relative paths to each layer's data source.

A question came up on GIS.SE asking Does QGIS have the equivalent of ArcGIS's Layer (*.LYR) file?. To which the answer was. "Not really". Which of course was true. At the time.

That got me thinking. Why couldn't we have a feature like this? It would be handy to have as it would let you create layer shortcuts and the ability to a load layers quickly from a single place without worrying about were the data comes from. The XML that we need to recreate the layer on project load was already stored in the project file under the maplayer node. It already has everything we need we can just write that out to disk and create a new file type. After some digging into the API I found QgsMapLayer::writeLayerXML() and Bingo! .qlr was born.

Because I personally, and some others I talked to, didn't really like just "Save as Layer File.." as it sounds a bit vauge, we picked "QGIS Layer Definition file".

A QGIS Layer Deinition File is simply the XML maplayer node, with the id removed, saved to disk as a .qlr format.

Saving one is as simple as Right Click Layer -> Save As Layer Definition File..

qlrsave.png

and loading Layer -> Add from Layer Definition File...

qlrload.png

Simple!

The layer file will store any of the styles, labels, edit widgets, etc, that you have defined for that layer so it's just a matter of adding the file and you are good to go.

Like I said at the start it's simply the XML maplayer node, so if you move the data or want to change the datasource just open the file in a text editor and change it.

<!DOCTYPE qgis-layer-definition>
<maplayer minimumScale="-4.65661e-10" maximumScale="1e+08" simplifyDrawingHints="1" minLabelScale="0" maxLabelScale="1e+08" simplifyDrawingTol="1" geometry="Polygon" simplifyMaxScale="1" type="vector" hasScaleBasedVisibilityFlag="0" simplifyLocal="1" scaleBasedLabelVisibilityFlag="0">
  <datasource>F:/gis_data/cadastre.shp</datasource>
  <title></title>
  <abstract></abstract>
  <keywordList>
    <value></value>
  </keywordList>
  <layername>cadastre</layername>
  ... {style, etc}
</maplayer

This will work with any layer supported in QGIS, expect special plugin layers.

I will add loading from the browser dock before the release aswell once I get some time.

Can we get it in an older QGIS version?

All the methods that I used for this feature are exposed to the Python API. Once I get some free time I plan on making a plugin that is aimed at QGIS <2.3 to give those users the same feature. Keep an eye out for it in the plugin installer.

QGIS Layer Definitions (An ArcGIS like .lyr function for QGIS)

NOTE Just to clarify. This is not adding the ability to open ArcGIS .lyr files.

One of the reasons that I love working and being part of the QGIS project is how quickly you can take it from not having a needed feature to having said feature. A good example of this is a recent addition I added related to having a ArcGIS .lyr like feature. For those who don't use Arc, the .lyr file is a pretty much a file that points to the data, contains all the styling, and other information, you can then just add this file and it will do all the other magic for you.

A quote from the ArcGIS page on the topic

A layer can exist outside your map as a layer file (.lyr). This makes it easy for others to access the layers you've built. You can share layers over the network and by e-mail. When users add a layer file to their maps, it will draw exactly as it was saved as long as they can get access to the data referenced by the layer. A common way that users help support this is to use relative paths to each layer's data source.

A question came up on GIS.SE asking Does QGIS have the equivalent of ArcGIS's Layer (*.LYR) file?. To which the answer was. "Not really". Which of course was true. At the time.

That got me thinking. Why couldn't we have a feature like this? It would be handy to have as it would let you create layer shortcuts and the ability to a load layers quickly from a single place without worrying about were the data comes from. The XML that we need to recreate the layer on project load was already stored in the project file under the maplayer node. It already has everything we need we can just write that out to disk and create a new file type. After some digging into the API I found QgsMapLayer::writeLayerXML() and Bingo! .qlr was born.

Because I personally, and some others I talked to, didn't really like just "Save as Layer File.." as it sounds a bit vauge, we picked "QGIS Layer Definition file".

A QGIS Layer Deinition File is simply the XML maplayer node, with the id removed, saved to disk as a .qlr format.

Saving one is as simple as Right Click Layer -> Save As Layer Definition File..

Alt Text

and loading Layer -> Add from Layer Definition File...

Alt Text

Simple!

The layer file will store any of the styles, labels, edit widgets, etc, that you have defined for that layer so it's just a matter of adding the file and you are good to go.

Like I said at the start it's simply the XML maplayer node, so if you move the data or want to change the datasource just open the file in a text editor and change it.

<!DOCTYPE qgis-layer-definition>
<maplayer minimumScale="-4.65661e-10" maximumScale="1e+08" simplifyDrawingHints="1" minLabelScale="0" maxLabelScale="1e+08" simplifyDrawingTol="1" geometry="Polygon" simplifyMaxScale="1" type="vector" hasScaleBasedVisibilityFlag="0" simplifyLocal="1" scaleBasedLabelVisibilityFlag="0">
  <datasource>F:/gis_data/cadastre.shp</datasource>
  <title></title>
  <abstract></abstract>
  <keywordList>
    <value></value>
  </keywordList>
  <layername>cadastre</layername>
  ... {style, etc}
</maplayer

This will work with any layer supported in QGIS, expect special plugin layers.

I will add loading from the browser dock before the release aswell once I get some time.

Can we get it in an older QGIS version?

All the methods that I used for this feature are exposed to the Python API. Once I get some free time I plan on making a plugin that is aimed at QGIS <2.3 to give those users the same feature. Keep an eye out for it in the plugin installer.

IntraMaps Roam - A Python QGIS data collection app

For the last couple of months I, though Digital Mapping Solutions, have been working on a tablet friendly data collection application that has been built on Python and QGIS. For those of you who have seen my QMap project I started a year or so ago you can consider this a reincarnation of that project, and that project retired. Most of the code has been reworked and using the QGIS libs gave me full flexibility in layout and workflow.

IntraMaps Roam (or Roam for short) is a standalone, fully bundled, Python application that was created to do data collection with a QGIS backend. The primary use of Roam is in a disconnected setup were one might not have internet connection, however Roam is using QGIS so will support any data format QGIS does. You can can use Roam in a connected environment, if your internet premits, by having WFS and WMS layers, or direct database connections; it's up to you. Roam forms also allow for custom logic to be added to each form using Python so you can add your own workflow if needed.

Roam Map Window Roam Data Capture

The binary package comes with a config manager application that can be used to create and manage Roam projects

Config Manager Config Manager

The release page contains links to the 2.0 installers. The wiki contains all the information to get started. You can also take a look at the FAQ for the common questions.

Roam has been a great exercise in using and bundling QGIS libs with a Python application, which I have never done before but turned out to be pretty easy. Being a fully bundled application means you don't need to install QGIS, or Python, on the client in order to run the application. Everything is in a nice bundled exe.

As Roam is based on PyQt and QGIS it is under the GPL2 license. Pull requests are welcome.

Currently Roam is only being packaged for Windows, because that was our first priority, however there isn't a lot of Windows only stuff in the code itself so creating a version for OS X and Linux shouldn't be too hard for someone with the know how.

Links

Happy mapping!?

IntraMaps Roam - A Python QGIS data collection app

For the last couple of months I, though Digital Mapping Solutions, have been working on a tablet friendly data collection application that has been built on Python and QGIS. For those of you who have seen my QMap project I started a year or so ago you can consider this a reincarnation of that project, and that project retired. Most of the code has been reworked and using the QGIS libs gave me full flexibility in layout and workflow.

IntraMaps Roam (or Roam for short) is a standalone, fully bundled, Python application that was created to do data collection with a QGIS backend. The primary use of Roam is in a disconnected setup were one might not have internet connection, however Roam is using QGIS so will support any data format QGIS does. You can can use Roam in a connected environment, if your internet premits, by having WFS and WMS layers, or direct database connections; it's up to you. Roam forms also allow for custom logic to be added to each form using Python so you can add your own workflow if needed.

Roam Map Window Roam Data Capture

The binary package comes with a config manager application that can be used to create and manage Roam projects

Config Manager Config Manager

The release page contains links to the 2.0 installers. The wiki contains all the information to get started. You can also take a look at the FAQ for the common questions.

Roam has been a great exercise in using and bundling QGIS libs with a Python application, which I have never done before but turned out to be pretty easy. Being a fully bundled application means you don't need to install QGIS, or Python, on the client in order to run the application. Everything is in a nice bundled exe.

As Roam is based on PyQt and QGIS it is under the GPL2 license. Pull requests are welcome.

Currently Roam is only being packaged for Windows, because that was our first priority, however there isn't a lot of Windows only stuff in the code itself so creating a version for OS X and Linux shouldn't be too hard for someone with the know how.

Links

Happy mapping!?

IntraMaps Roam - A Python QGIS data collection app

For the last couple of months I, though Digital Mapping Solutions, have been working on a tablet friendly data collection application that has been built on Python and QGIS. For those of you who have seen my QMap project I started a year or so ago you can consider this a reincarnation of that project, and that project retired. Most of the code has been reworked and using the QGIS libs gave me full flexibility in layout and workflow.

IntraMaps Roam (or Roam for short) is a standalone, fully bundled, Python application that was created to do data collection with a QGIS backend. The primary use of Roam is in a disconnected setup were one might not have internet connection, however Roam is using QGIS so will support any data format QGIS does. You can can use Roam in a connected environment, if your internet premits, by having WFS and WMS layers, or direct database connections; it's up to you. Roam forms also allow for custom logic to be added to each form using Python so you can add your own workflow if needed.

Roam Map Window Roam Data Capture

The binary package comes with a config manager application that can be used to create and manage Roam projects

Config Manager Config Manager

The release page contains links to the 2.0 installers. The wiki contains all the information to get started. You can also take a look at the FAQ for the common questions.

Roam has been a great exercise in using and bundling QGIS libs with a Python application, which I have never done before but turned out to be pretty easy. Being a fully bundled application means you don't need to install QGIS, or Python, on the client in order to run the application. Everything is in a nice bundled exe.

As Roam is based on PyQt and QGIS it is under the GPL2 license. Pull requests are welcome.

Currently Roam is only being packaged for Windows, because that was our first priority, however there isn't a lot of Windows only stuff in the code itself so creating a version for OS X and Linux shouldn't be too hard for someone with the know how.

Links

Happy mapping!?

Keeping QGIS settings in sync on different machines

Here is a quick tip from a GIS.SE post that I answered the other day.

The topic was keeping the WMS settings in sync over different operating systems and machines. Normally QGIS will store it settings in the registry on Windows and in different locations on Linux and OS X. So then comes the question of how do you keep them in sync if you are using different machines.

Well the answer is simple. QGIS provides --optionspath and --configpath command line options in order to move the .qgis2 and settings files. Using these two options, or just the one depending on what you need, will allow you to store the QGIS settings in a different location. Rather then storing the settings in the registry, or .config and .plist files, it will create a .ini file and save everything there.

All in all this means you can redirect your QGIS settings to a folder on dropbox and tell your QGIS installs to load the settings from a single place keeping everything in sync. When you change a setting it will sync with Dropbox and onto your other machines.

The simple way on Windows to add the --optionspath and --configpath options is to copy the shortcut to QGIS and append it to the end of the Target.

--optionspath "F:\mydropbox\qgis" --configpath "F:\mydropbox\qgis"

QGIS IN THE CLOUD!!11! ok not really but still cool

Keeping QGIS settings in sync on different machines

Here is a quick tip from a GIS.SE post that I answered the other day.

The topic was keeping the WMS settings in sync over different operating systems and machines. Normally QGIS will store it settings in the registry on Windows and in different locations on Linux and OS X. So then comes the question of how do you keep them in sync if you are using different machines.

Well the answer is simple. QGIS provides --optionspath and --configpath command line options in order to move the .qgis2 and settings files. Using these two options, or just the one depending on what you need, will allow you to store the QGIS settings in a different location. Rather then storing the settings in the registry, or .config and .plist files, it will create a .ini file and save everything there.

All in all this means you can redirect your QGIS settings to a folder on dropbox and tell your QGIS installs to load the settings from a single place keeping everything in sync. When you change a setting it will sync with Dropbox and onto your other machines.

The simple way on Windows to add the --optionspath and --configpath options is to copy the shortcut to QGIS and append it to the end of the Target.

--optionspath "F:\mydropbox\qgis" --configpath "F:\mydropbox\qgis"

QGIS IN THE CLOUD!!11! ok not really but still cool

Keeping QGIS settings in sync on different machines

Here is a quick tip from a GIS.SE post that I answered the other day.

The topic was keeping the WMS settings in sync over different operating systems and machines. Normally QGIS will store it settings in the registry on Windows and in different locations on Linux and OS X. So then comes the question of how do you keep them in sync if you are using different machines.

Well the answer is simple. QGIS provides --optionspath and --configpath command line options in order to move the .qgis2 and settings files. Using these two options, or just the one depending on what you need, will allow you to store the QGIS settings in a different location. Rather then storing the settings in the registry, or .config and .plist files, it will create a .ini file and save everything there.

All in all this means you can redirect your QGIS settings to a folder on dropbox and tell your QGIS installs to load the settings from a single place keeping everything in sync. When you change a setting it will sync with Dropbox and onto your other machines.

The simple way on Windows to add the --optionspath and --configpath options is to copy the shortcut to QGIS and append it to the end of the Target.

--optionspath "F:\mydropbox\qgis" --configpath "F:\mydropbox\qgis"

QGIS IN THE CLOUD!!11! ok not really but still cool

Back to Top

Sustaining Members