The QGIS 3.28 release is an extremely exciting release for all users who work in mixed software workplaces, or who need to work alongside users of ESRI software. In this post we’ll be giving an overview of all the new tools and features introduced in 3.28 which together result in a dramatic improvement in the workflows and capabilities in working with ESRI based formats and services. Read on for the full details…!

Before we begin, we’d like to credit the following organisations for helping fund these developments in QGIS 3.28:

• Naturstyrelsen, Denmark
• Provincie Gelderland, Netherlands
• Uppsala Universitet, Department of Archaeology and Ancient History
• Gemeente Amsterdam
• Provincie Zuid-Holland, Netherlands

FileGeodatabase (GDB) related improvements

The headline item here is that QGIS 3.28 introduces support for editing, managing and creating ESRI FileGeodatabases out of the box! While older QGIS releases offered some limited support for editing FileGeodatabase layers, this required the manual installation of a closed source ESRI SDK driver… which unfortunately resulted in other regressions in working with FileGeodatabases (such as poor layer loading speed and random crashes). Now, thanks to an incredible reverse engineering effort by the GDAL team, the open-source driver for FileGeodatabases offers full support for editing these datasets! This means all QGIS users have out-of-the-box access to a fully functional, high-performance read AND write GDB driver, no further action or trade-offs required.

Operations supported by the GDAL open source driver include:

• Editing existing features, with full support for editing attributes and curved, 3D and measure-value geometries
• Creating new features
• Deleting features
• Creating, adding and modifying attributes in an existing layer
• Full support for reading and updating spatial indexes
• Creating new indexes on attributes
• “Repacking” layers, to reduce their size and improve performance
• Creating new layers in an existing FileGeodatabase
• Removing layers from FileGeodatabases
• Creating completely new, empty FileGeodatabases
• Creating and managing field domains

On the QGIS side, the improvements to the GDAL driver meant that we could easily expose feature editing support for FileGeodatabase layers for all QGIS users. While this is a huge step forward, especially for users in mixed software workplaces, we weren’t happy to rest there when we  had the opportunity to further improve GDB support within QGIS!

So in QGIS 3.28 we also introduced the following new functionality when working with FileGeodatabases:

FileGeodatabase management tools

QGIS 3.28 introduces a whole range of GUI based tools for managing FileGeodatabases. To create a brand new FileGeodatabase, you can now right click on a directory from the QGIS Browser panel and select New – ESRI FileGeodatabase:

After creating your new database, a right click on its entry will show a bunch of available options for managing the database. These include options for creating new tables, running arbitrary SQL commands, and database-level operations such as compacting the database:

You’re also able to directly import existing data into a FileGeodatabase by simply dragging and dropping layers onto the database!

Expanding out the GDB item will show a list of layers present in the database, and present options for managing the fields in those layers. Alongside field creation, you can also remove and rename existing fields.

Field domain handling

QGIS 3.28 also introduces a range of GUI tools for working with field domains inside FileGeodatabases. (GeoPackage users also share in the love here — these same tools are all available for working with field domains inside this standard format too!) Just right click on an existing FileGeodatabase (or GeoPackage) and select the “New Field Domain” option. Depending on the database format, you’ll be presented with a list of matching field domain types:

Once again, you’ll be guided through a user-friendly dialog allowing you to create your desired field domain!

After field domains have been created, they can be assigned to fields in the database by right-clicking on the field name and selecting “Set Field Domain”:

Field domains can also be viewed and managed by expanding out the “Field domains” option for each database.

Relationship discovery

Another exciting addition in QGIS 3.28 (and the underlying GDAL 3.6 release) is support for discovering database relationships in FileGeodatabases! (Once again, GeoPackage users also benefit from this, as we’ve implemented full support for GeoPackage relationships via the “Related Tables Extension“).

Expanding out a database containing any relationships will show a list of all discovered relationships:

(You can view the full description and details for any of these relationships by opening the QGIS Browser “Properties” panel).

Whenever QGIS 3.28 discovers relationships in the database, these related tables will automatically be added to your project whenever any of the layers which participate in the relationship are opened. This means that users get the full experience as designed for these databases without any manual configuration, and the relationships will “just work”!

Dataset Grouping

Lastly, we’ve improved the way layers from FileGeodatabases are shown in QGIS, so that layers are now grouped according to their original dataset groupings from the database structure:

Edit ArcGIS Online / Feature Service layers

While QGIS has had read-only support for viewing and working with the data in ArcGIS Online (AGOL) vector layers and ArcGIS Server “feature service” layers for many years, we’ve added support for editing these layers in QGIS 3.28. This allows you to take advantage of all of QGIS’ easy to use, powerful editing tools and directly edit the content in these layers from within your QGIS projects! You can freely create new features, delete features, and modify the shape and attributes of existing features (assuming that your user account on the ArcGIS service has these edit permissions granted, of course). This is an exciting addition for anyone who has to work often with content in ArcGIS services, and would prefer to directly manipulate these layers from within QGIS instead of the limited editing tools available on the AGOL/Portal platforms themselves.

This new functionality will be available immediately to users upon upgrading to QGIS 3.28 — any users who have been granted edit capabilities for the layers will see that the QGIS edit tools are all enabled and ready for use without any further configuration on the QGIS client side.

Filtering Feature Service layers

We’ve also had the opportunity to introduce filter/query support for Feature Service layers in QGIS 3.28. This is a huge performance improvement for users who need to work with a subset of a features from a large Feature Service layer. Unfortunately, due to the nature of the Feature Service protocol, these layers can often be slow to load and navigate on a client side. By setting a SQL filter to limit the features retrieved from the service the performance can be dramatically increased, as only matching features will ever be requested from the backend server. You can use any SQL query which conforms to the subset of SQL understood by ArcGIS servers (see the Feature Service documentation for examples of supported SQL queries).

What’s next?

While QGIS 3.28 is an extremely exciting release for any users who need to work alongside ESRI software, we aren’t content to rest here! The exciting news is that in QGIS 3.30 we’ll be introducing a GUI driven approach allowing users to create new relationships in their FileGeodatabase (and GeoPackage!) databases.

At North Road we’re always continuing to improve the cross-vendor experience for both ESRI and open-source users through our continued work on the QGIS desktop application and our SLYR conversion suite. If you’d like to chat to us about how we can help your workplace transition from a fully ESRI stack to a mixed or fully open-source stack, just contact us to discuss your needs.

Our SLYR tool is the complete solution for full compatibility between ArcMap, ArcGIS Pro and QGIS. It offers a powerful suite of conversion tools for opening ESRI projects, styles and other documents directly within QGIS, and for exporting QGIS documents for use in ESRI software.

A lot has changed since our last SLYR product update post, and we’ve tons of very exciting improvements and news to share with you all! In this update we’ll explore some of the new tools we’ve added to SLYR, and discuss how these tools have drastically improved the capacity for users to migrate projects from the ESRI world to the open-source world (and vice versa).

ArcGIS Pro support

The headline item here is that SLYR now offers a powerful set of tools for working with the newer ArcGIS Pro document formats. Previously, SLYR offered support for the older ArcMap document types only (such as MXD, MXT, LYR, and PMF formats). Current SLYR versions now include tools for:

Directly opening ArcGIS Pro .lyrx files within QGIS

LYRX files can be dragged and dropped directly onto a QGIS window to add the layer to the current project. All the layer’s original styling and other properties will be automatically converted across, so the resultant layer will be an extremely close match to the original ArcGIS Pro layer! SLYR supports vector layers, raster layers, TIN layers, point cloud layers and vector tile layers. We take great pride in just how close the conversion results are to how these layers appear in ArcGIS Pro… in most cases you’ll find the results are nearly pixel perfect!

In addition to drag-and-drop import support, SLYR also adds support for showing .lyrx files directly in the integrated file browser, and also adds tools to the QGIS Processing Toolbox so that users can execute bulk conversion operations, or include document conversion in their models or custom scripts.

ArcGIS Pro map (mapx) and project (aprx) conversion

Alongside the LYRX support, we’ve also added support for the ArcGIS Pro .mapx and .aprx formats. Just like our existing .mxd conversion, you can now easily convert entire ArcGIS Pro maps for direct use within QGIS! SLYR supports both the older ArcGIS Pro 2.x project format and the newer 3.x formats.

Export from QGIS to ArcGIS Pro!

Yes, you read that correctly… SLYR now allows you to export QGIS documents into ArcGIS Pro formats! This is an extremely exciting development… for the first time ever QGIS users now have the capacity to export their work into formats which can be supplied directly to ESRI users. Current SLYR versions support conversion of map layers to .lyrx format, and exporting entire projects to the .mapx format. (We’ll be introducing support for direct QGIS to .aprx exports later this year.)

We’re so happy to finally provide an option for QGIS users to work alongside ArcGIS Pro users. This has long been a pain point for many organisations, and has even caused organisations to be ineligible to tender for jobs which they are otherwise fully qualified to do (when tenders require provision of data and maps in ArcGIS compatible formats).

ArcGIS Pro .stylx support

Alongside the other ArcGIS Pro documents, SLYR now has comprehensive support for reading and writing ArcGIS Pro .stylx databases. We’ve dedicated a ton of resources in ensuring that the conversion results (both from ArcGIS Pro to QGIS and from QGIS to ArcGIS Pro) are top-notch, and we even handle advanced ArcGIS Pro symbology options like symbol effects!

Take a look below how even very advanced ArcGIS Pro style libraries convert beautifully to QGIS symbol libraries:

ArcMap Improvements

While we’ve been focusing heavily on the newer ArcGIS Pro formats, we’ve also improved our support for the older ArcMap documents. In particular, SLYR now offers more options for converting ArcMap annotation layers and annotation classes to QGIS supported formats. Users can now convert Annotation layers and classes directly over to QGIS annotation layer or alternatively annotation classes can be converted over to the OGC standard GeoPackage format. When exporting annotation classes to GeoPackage the output database is automatically setup with default styling rules, so that the result can be opened directly in QGIS and will be immediately visualised to match the original annotation class.

Coming soon…

While all the above improvements are already available for all SLYR license holders, we’ve got many further improvements heading your way soon! For example, before the end of 2022 we’ll be releasing another large SLYR update which will introduce support for exporting QGIS projects directly to ArcGIS Pro .aprx documents. We’ve also got many enhancements planned which will further improve the quality of the converted documents. Keep an eye on this blog and our social media channels for more details as they are available…

You can read more about our SLYR tool at the product page, or contact us today to discuss licensing options for your organisation.

North Road are proud to announce the official release of SLYR Community Edition, a new open-source version of our powerful SLYR ESRI to Open Source compatibility suite. The Community Edition is available for download from the official QGIS plugin repository today, for QGIS versions 3.4 and above. It supports automated conversion of ESRI .style symbol databases, including conversion of markers, fills, line styles and color ramps to their closest QGIS symbology equivalent, allowing users to instantly transition their style libraries into QGIS!

If you’ve followed our work in the past, it will come as no surprise to hear that North Road are passionate about open source geospatial, and for reducing the barriers which users encounter when moving to open-source software. We see our SLYR tool as an integral part of this process, and the licensed version of the plugin currently supports automated conversion of MXD, LYR, PMF, and other ESRI-specific formats to QGIS documents.

Our intention all along has been to make this tool freely available for all users of open-source geospatial software, and to release our work under a permissive, open-source license so that other projects can take advantage of our reverse engineering efforts. That’s why we made an “open source pledge” a fundamental part of our SLYR tool development! By the terms of this pledge, exactly six months after we hit staged preset funding levels we will open-source more components of the code and update the community version of the plugin accordingly. (This approach gives motivated organisations instant access to the full functionality of the SLYR tool via a license purchase, or free access to a subset of this functionality via the community edition of the plugin. It allows us to heavily invest in further reverse-engineering efforts and improvements to the plugin, to QGIS, and to the wider open-source geospatial community.)

If you’re keen to explore transitioning your workplace from ESRI to open-source, send us an email to discuss what we can offer! North Road staff have years of experience in implementing open-source geospatial solutions within commercial workplaces, and for setting up dual commercial-and-open-source friendly environments.

Recently, staff at North Road have been hard at work on our SLYR “ESRI to QGIS compatiblity suite“, and we thought it’s time to share some of the latest exciting updates with you.

While SLYR begun life as a simple “LYR to QGIS conversion tool”, it quickly matured into a full ArcGIS compatibility suite for QGIS. Aside from its original task of converting ESRI LYR files, SLYR now extends the QGIS interface and adds seamless support for working with all kinds of ArcGIS projects and data files. It’s rapidly becoming a must-have tool for any organisation which uses a mix of ESRI and open source tools, or for any organisation exploring a transition away from ArcGIS to QGIS.

Accordingly, we thought it’s well past time we posted an update detailing the latest functionality and support we’ve added to SLYR over the past couple of months! Let’s dive in…

• Full support for raster LYR file conversion, including unique value renderers, color map renderers, classified renderers, RGB renderers and stretched color ramp renderers:

  

  

• Support for conversion of fill symbol outlines with complex offsets, decorations and dashed line templates
• Conversion of 3D marker and simple 3D lines to their 2d equivalent, matching ArcMap’s 2D rendering of these symbol types
• Beta support for converting map annotations and drawings, including custom text labels and reference scale support
  
• Label and annotation callout support*
• Support for converting bookmarks stored in MXD documents*
• Support for converting ESRI bookmark “.dat” files via drag and drop to QGIS*
  
• Correct conversion of OpenStreetMap and bing maps basemap layers
• SLYR now presents users with a friendly summary of warnings generated during the LYR or MXD conversion process (e.g. due to settings which can’t be matched in QGIS)
  
• Added support for MXD documents generated in very early ArcMap versions
• We’ve added QGIS Processing algorithms allowing for bulk LYR to QLR and MXD to QGS conversion. Now you can run a batch conversion process of ALL MXD/LYR files held at your organisation in one go!
  
• Greatly improved matching of converted symbols to their original ArcGIS appearance, including more support for undocumented ArcGIS symbol rendering behavior
• Support for conversion of text symbols and label settings stored in .style databases*
  
• Directly drag and drop layers and layer groups from ArcMap to QGIS to add them to the current QGIS project (maintaining their ArcGIS symbology and layer settings!)*
• Directly drag and drop layers from ArcCatalog to QGIS windows to open in QGIS*
• Support for ESRI MapServer layers

(*requires QGIS 3.10 or later)

Over the remainder of 2019, we’ll be hard at work further improving SLYR’s support for MXD document conversion, and adding support for automatic conversion of ArcMap print layouts to QGIS print layouts.

While SLYR is not currently an open-source tool, we believe strongly in the power of open source software, and accordingly we’ve been using a significant portion of the funds generated from SLYR sales to extend the core QGIS application itself. This has directly resulted in many exciting improvements to QGIS, which will become widely available in the upcoming QGIS 3.10 release. Some of the features directly funded by SLYR sales include:

• A “Segment Center” placement mode for marker line symbols
• Reworked bookmark handling in QGIS, with a greatly enhanced workflow and usability, and a stable API for 3rd party plugins and scripts to hook into
• Improved handling of layer symbology for layers with broken paths
• Auto repair of all other broken layers with a matching data source whenever a single layer path is fixed in a project
• Support for managing text formats and label settings in QGIS style libraries, allowing storage and management of label and text format presets
• A new Processing algorithm “Combine Style Databases“, allowing multiple QGIS style databases to be merged to one
• Adding a “Save layer styles into GeoPackage” option for the “Package Layers” algorithm
• New expression functions which return file info, such as file paths and base file names
• Adding new options to autofill the batch Processing dialog, including adding input files using recursive filter based file searches
• Coming in QGIS 3.12: A new option to set the color to use when rendering nodata pixels in raster layers
• Coming in QGIS 3.12: A new “random marker fill” symbol layer type, which fills polygons by placing point markers in random locations

You can read more about our SLYR ESRI to QGIS compatibility tool here, or email [email protected] to discuss licensing arrangements for your organisation! Alternatively, send us an email if you’d like to discuss your organisations approach to open-source GIS and for assistance in making this transition as painless as possible.

Thanks to an anonymous corporate sponsor, we’ve recently had the opportunity to add a new Hashed Line symbol type for QGIS 3.8. This allows for a repeating line segment to be drawn over the length of a feature, with a line-sub symbol used to render each individual segment.

There’s tons of options available for customising the appearance and placement of line hashes. We based the feature heavily off QGIS’ existing “Marker Line” support, so you can create hashed lines placed at set intervals, on line vertices, or at the start/end/middle of lines. There’s options to offset the lines, and tweak the rotation angle of individual hashes too. Added to QGIS’ rich support for “data defined” symbol properties, this allows for a huge range of new symbol effects.

E.g. using a data defined hash length which increases over the length of a feature gives us this effect:

Or, when using a complex line sub-symbol for rendering each hash, we can get something like this:

Or even go completely “meta” and use a hashed line sub symbol for the hash line itself!

With the right combination of symbol settings and QGIS draw effects you can even emulate a calligraphic pen effect:

Or a chunky green highlighter!

This same corporate sponsor also funded a change which results in a huge improvement to the appearance of both rotated hashed lines and marker lines. Previously, when marker or hash lines were rendered, the symbol angles were determined by taking the exact line orientation at the position of the symbol. This often leads to undesirable rendering effects, where little “bumps” or corners in lines which occur at the position of the symbol cause the marker or hash line to be oriented at a very different angle to what the eye expects to see.

With this improvement, the angle is instead calculated by averaging the line over a specified distance either side of the symbol. E.g. averaging the line angle over 4mm means we take the points along the line 2mm from either side of the symbol placement, and use these instead to calculate the line angle for that symbol. This has the effect of smoothing (or removing) any tiny local deviations from the overall line direction, resulting in much nicer visual orientation of marker or hash lines.

It’s easiest to show the difference visually. Here’s a before image, showing arrow markers following a line feature. Pay specific attention to the 3rd and last arrow, which seem completely random oriented due to the little shifts in line direction:

With new smoothing logic this is much improved:

The difference is even more noticeable for hashed lines. Here’s the before:

…and the after:

Suffice to say, cartographers will definitely appreciate the result!

Lastly, we’ve taken this new hash line feature as an opportunity to implement automatic conversion of ESRI hash line symbols within our SLYR ESRI to QGIS conversion tool. Read more about SLYR here, and how you can purchase this tool for .style, .lyr and .mxd document conversion.

One product which North Road had the chance to develop last year, and which we are super-proud of, is our SLYR ESRI style to QGIS conversion tool. If you haven’t heard of it before, this tool allows automatic conversion of ESRI .style database files to their equivalent QGIS symbology equivalent. It works well for the most part, and now we’re keen to take this to the next stage.

The good news is that North Road have been conducting extensive research and development over the past 12 months, and we’re pleased to announce our plans for extending SLYR to support ESRI LYR and MXD documents. The LYR and MXD formats are proprietary ESRI-only formats, with no public specifications allowing their use. This is a huge issue for organisations who want to move from an ESRI environment to the open geospatial world, yet are held back by hundreds (or thousands!) of existing ESRI MXD map documents and layer styles which they currently cannot utilise outside of the ESRI software ecosystem. Furthermore, many providers of spatial data only include ESRI specific layer formatting files with their data supplies. This leaves users with no means of utilising these official, pre-defined styles in non-ESRI tools.

In order for us to continue development of the SLYR tool and unlock use of LYR and MXD formats outside of ESRI tools, we are conducting a funding campaign. Sponsors of the campaign will receive access to the tools as they are developed and gain access to official support channels covering their use. At the conclusion of this drive we’ll be releasing all the tools and specifications under a free, open-source license.

You can read the full details of the campaign here, including pricing to become a project sponsor and gain access to the tools as they develop. As a campaign launch promo, we’re offering the first 10 sponsors a super-special discounted rate (as a reward for jumping on the development early).

The mockup below shows what the end goal is: seamless, fully integrated, automatic conversion of MXD and LYR files directly within the QGIS desktop application!

ESRI’s ArcGIS Online World Imagery is a high resolution satellite and aerial imagery base map for use in Google Earth, ArcMap and ArcGIS Explorer. The same excellent imagery is used by the Bing Maps Aerial layer. Somewhat surprisingly, World Imagery can also be accessed by QGIS, as it supports ESRI’s map servers that use Representational State Transfer (REST) and Simple Object Assess Protocol (SOAP) standards.

Simply copy and past the following code into the Python Console in QGIS and press return (Plugins – Console):

qgis.utils.iface.addRasterLayer("http://server.arcgisonline.com/arcgis/rest/services/ESRI_Imagery_World_2D/MapServer?f=json&pretty=true","raster")

The code adds an ESRI Online World Imagery base map to QGIS. It has a number of advantages over the popular OpenLayers Plugin that adds various Google, Bing and OpenStreetMap image layers to QGIS. Unlike images downloaded by the OpenLayers plugin the ESRI World Imagery base map is a true Raster who’s attributes are fully editable e.g. brightness, blending mode and transparency can be adjusted. World Imagery can also be printed at a very high resolution with other QGIS layers on a map and without it shifting relative to other layers; a conspicuous problem with OpenLayers that does not use “On the Fly” re-projection and only prints Google, Bing layers at a low resolution. It is an ideal aerial base map.

References:

QGIS: Adding An ArcServer Rest Service

Connecting to ArcGIS “mapserver” layers

Edit: Updated to correct URL

Note: This method has been superseded by a plug-in that adds ESRI imagery and other REST layers via a GUI

Note:The following post requires gdal 1.8

A lot of times we need to send/use 3D dxf files, we used to use FME however FME is not free or cheap. So I went looking for a free solution.

If you have gdal 1.8 it’s just one simple command line run using, what is becoming my favorite GIS tool, ogr2ogr.

All you have to do is run:

ogr2ogr -f "DXF" {outFile} {inFile} -zfield {ColumnWithZValue}

So in my case I ran:

ogr2ogr -f "DXF" C:\Temp\contourswarwick.dxf C:\Temp\Contours.TAB -zfield Height

I haven’t fully tested it but {outfile} can be any file ogr supports.

and the output:

Top view of contour layer

The side view of the above image.

In the words of, the not so great, Charlie Sheen. WINNING!

Happy mapping :D

At the ESRI Developer Summit there was news of the File Geodatabase (FGDB) API. Based on the tweets from the summit it appears:

• The API will be C++ only
• API works on Windows and Unix/Linux (specifically RedHat, Solaris, SuSE) operating systems
• Rudimentary support only—features such as annotation, relationships, topologies, etc. are lacking

Since ESRI is releasing a targeted API and a not a specification, support for Mac OS X is out of the question.

How well open source GIS applications will be able to support the FGDB (if at all) remains to be seen.

The other question is: Will the API be made freely available or will it require an ArcGIS license?

Will the release of the FGDB API be the “Shapefile-killer”? I have my doubts…