Related Plugins and Tags

QGIS Planet

Software quality in QGIS

According to the definition of software quality given by french Wikipedia

An overall assessment of quality takes into account external factors, directly observable by the user, as well as internal factors, observable by engineers during code reviews or maintenance work.

I have chosen in this article to only talk about the latter. The quality of software and more precisely QGIS is therefore not limited to what is described here. There is still much to say about:

  • Taking user feedback into account,
  • the documentation writing process,
  • translation management,
  • interoperability through the implementation of standards,
  • the extensibility using API,
  • the reversibility and resilience of the open source model…

These are subjects that we care a lot and deserve their own article.

I will focus here on the following issue: QGIS is free software and allows anyone with the necessary skills to modify the software. But how can we ensure that the multiple proposals for modifications to the software contribute to its improvement and do not harm its future maintenance?

Self-discipline

All developers contributing to QGIS code doesn’t belong to the same organization. They don’t all live in the same country, don’t necessarily have the same culture and don’t necessarily share the same interests or ambitions for the software. However, they share the awareness of modifying a common good and the desire to take care of it.

This awareness transcends professional awareness, the developer not only has a responsibility towards his employer, but also towards the entire community of users and contributors to the software.

This self-discipline is the foundation of the quality of the contributions of software like QGIS.

However, to err is human and it is essential to carry out checks for each modification proposal.

Automatic checks

With each modification proposal (called Pull Request or Merge Request), the QGIS GitHub platform automatically launches a set of automatic checks.

Example of proposed modification

Result of automatic checks on a modification proposal

The first of these checks is to build QGIS on the different systems on which it is distributed (Linux, Windows, MacOS) by integrating the proposed modification. It is inconceivable to integrate a modification that would prevent the application from being built on one of these systems.

The tests

The first problem posed by a proposed modification is the following “How can we be sure that what is going to be introduced does not break what already exists?”

To validate this assertion, we rely on automatic tests. This is a set of micro-programs called tests, which only purpose is to validate that part of the application behaves as expected. For example, there is a test which validates that when the user adds an entry in a data layer, then this entry is then present in the data layer. If a modification were to break this behavior, then the test would fail and the proposal would be rejected (or more likely corrected).

This makes it possible in particular to avoid regressions (they are very often called non-regression tests) and also to qualify the expected behavior.

There are approximately 1.3 Million lines of code for the QGIS application and 420K lines of test code, a ratio of 1 to 3. The presence of tests is mandatory for adding functionality, therefore the quantity of test code increases with the quantity of application code.

In blue the number of lines of code in QGIS, in red the number of lines of tests

There are currently over 900 groups of automatic tests in QGIS, most of which run in less than 2 seconds, for a total execution time of around 30 minutes.

We also see that certain parts of the QGIS code – the most recent – are better covered by the tests than other older ones. Developers are gradually working to improve this situation to reduce technical debt.

Code checks

Analogous to using a spell checker when writing a document, we carry out a set of quality checks on the source code. We check, for example, that the proposed modification does not contain misspelled words or “banned” words, that the API documentation has been correctly written or that the modified code respects certain formal rules of the programming language.

We recently had the opportunity to add a check based on the clang-tidy tool. The latter relies on the Clang compiler. It is capable of detecting programming errors by carrying out a static analysis of the code.

Clang-tidy is, for example, capable of detecting “narrowing conversions”.

Example of detecting “narrowing conversions”

In the example above, Clang-tidy detects that there has been a “narrowing conversion” and that the value of the port used in the network proxy configuration “may” be corrupted. In this case, this problem was reported on the QGIS issues platform and had to be corrected.

At that time, clang-tidy was not in place. Its use would have made it possible to avoid this anomaly and all the steps which led to its correction (exhaustive description of the issue, multiple exchanges to be able to reproduce it, investigation, correction, review of the modification), meaning a significant amount of human time which could thus have been avoided.

Peer review

A proposed modification that would validate all of the automatic checks described above would not necessarily be integrated into the QGIS code automatically. In fact, its code may be poorly designed or the modification poorly thought out. The relevance of the functionality may be doubtful, or duplicated with another. The integration of the modification would therefore potentially cause a burden for the people in charge of the corrective or evolutionary maintenance of the software.

It is therefore essential to include a human review in the process of accepting a modification.

This is more of a rereading of the substance of the proposal than of the form. For the latter, we favor the automatic checks described above in order to simplify the review process.

Therefore, human proofreading takes time, and this effort is growing with the quantity of modifications proposed in the QGIS code. The question of its funding arises, and discussions are in progress. The QGIS.org association notably dedicates a significant part of its budget to fund code reviews.

More than 100 modification proposals were reviewed and integrated during the month of December 2023. More than 30 different people contributed. More than 2000 files have been modified.

Therefore the wait for a proofreading can sometimes be long. It is also often the moment when disagreements are expressed. It is therefore a phase which can prove frustrating for contributors, but it is an important and rich moment in the community life of a free project.

To be continued !

As a core QGIS developer, and as a pure player OpenSource company, we believe it is fundamental to be involved in each step of the contribution process.

We are investing in the review process, improving automatic checks, and in the QGIS quality process in general. And we will continue to invest in these topics in order to help make QGIS a long-lasting and stable software.

If you would like to contribute or simply learn more about QGIS, do not hesitate to contact us at [email protected] and consult our QGIS support proposal.

Translating Open Source Software with Weblate: A GRASS GIS Case Study

Open source software projects thrive on the contributions of the community, not only for the code, but also for making the software accessible to a global audience. One of the critical aspects of this accessibility is the localization or translation of the software’s messages and interfaces. In this context, Weblate (https://weblate.org/) has proven to be a powerful tool for managing these translations, especially for projects such as GRASS GIS, which is part of OSGeo (Open Source Geospatial Foundation).

Weblate software logoGRASS GIS logo

What is Weblate?

Weblate is an open source translation management system designed to simplify the translation process of software projects. It provides an intuitive web interface that allows translators to work without deep technical knowledge. This ease of use combined with robust integration capabilities makes Weblate a popular choice for open source projects.

GRASS GIS and Localization

GRASS GIS (https://grass.osgeo.org/), a software suite for managing and analyzing geospatial data, is used worldwide and therefore needs to be available in many languages. The project uses Weblate, hosted by OSGeo, to manage and facilitate its translation work (see OSGeo-Weblate portal).

Marking messages for translation

Before translation work can begin, the messages to be translated must be marked for translation in the GRASS GIS source code. This is done with the gettext macro _(“…”). GNU gettext is a GNU library for the internationalization of software. Here is a simplified overview of the process:

  1. Identify the strings to be translated: The developers identify the strings in the source code that need to be translated. These are usually user messages, while debug messages are not marked for translation.
  2. Use the gettext macro: The identified strings are packed into a gettext macro. For example, a string “Welcome to GRASS GIS” in the source code would be changed to _(“Welcome to GRASS GIS”). This change indicates that the string should be used for translation.
  3. Extraction and template generation: Tools such as xgettext are used to extract these marked strings from the source code and create a POT (Portable Object Template) file. This file is used as a template for all translations. In the GRASS GIS project the template language is English.

There are three template files in the GRASS GIS project: one with the graphical user interface (GUI) messages, one with the library functions (libs) and one with the modules (mods).

Connecting the software project to Weblate

While the POT files could be transferred to Weblate manually, we chose the automated option. The OSGeo Weblate instance is directly connected to the GRASS GIS project via git (GitHub) using the Weblate version control integration.

How it works in practice:

  1. Developer makes a commit to the GRASS GIS repo on GitHub
  2. A GitHub webhook makes a call to weblate.osgeo.org – note that it has it’s own local git repo for GRASS GIS, as it does for other OSGeo projects, with translations being managed in this Weblate instance. This local git repo is updated when the webhook is fired.
  3. As messages are translated in OSGeo-Weblate, they are eventually pushed to the Weblate Github fork of GRASS GIS (the push frequency is set to 24 hours by default, i.e., new translations are collected over a day), and Weblate then triggers a pull request to the main GRASS GIS repo on GitHub.

For technical background on the OSGeo Weblate installation, see the related OSGeo-SAC Weblate page.

Translation process in Weblate

Here is how the typical translation process looks like:

  • Translator registration: Registration (via OSGeo-ID) and login to the Weblate instance.
  • Language selection: Select the language to be translated. If a language does not exist yet, it can be added with the approval of the project managers.
  • Translation interface: Weblate provides an easy-to-use web interface where translators can view the original texts and enter their translations. If activated, machine translation can also be used here (DeepL, Google Translate, etc.). The Weblate translation memory helps to quickly translate identical and similar sentences.
GRASS GIS messages in Weblate

GRASS GIS messages in Weblate

  • Together we are better: translators can discuss translations, resolve conflicts and suggest improvements. Weblate also offers quality checks to ensure consistency and accuracy. Translations in different languages can be compared in tabular form.
Message translation comparison in Weblate (GRASS GIS project example)

Message translation comparison in Weblate (GRASS GIS project example)

  • Integration with source code: Once translations are completed and checked, they are written back into the GRASS GIS source code (see above). Weblate supports automatic synchronization with source code repositories.
  • Continuous updates: As the source code evolves, new strings can be marked for translation and Weblate is automatically updated to reflect these changes.
Pull request with new translations opened by Weblate in GRASS GIS Github repository

Pull request with new translations opened by Weblate in GRASS GIS Github repository

Benefits for the GRASS GIS project

By using Weblate, GRASS GIS benefits from the following advantages:

  • Streamlined translation workflow: The process from tagging strings to integrating translations is efficient and manageable.
  • Community engagement: Weblate’s ease of use encourages more community members to participate in the translation process.
  • Quality and Consistency: Weblate ensures high quality translations through integrated quality checks and collaboration tools.
  • Up-to-date localization: Continuous synchronization with the source code repository ensures that translations are always up-to-date.

Conclusion

The integration of Weblate into the GRASS GIS development workflow underlines the importance of localization in open source software. By using tools such as gettext for message tagging and Weblate for translation management, GRASS GIS ensures that it remains accessible and usable for a global community, embodying the true spirit of open source software.

Thanks

Thanks to Regina Obe from OSGeo-SAC for her support in setting up and maintaining the OSGeo-Weblate instance and for her explanations of how things work in terms of Weblate/GitHub server communication.

The post Translating Open Source Software with Weblate: A GRASS GIS Case Study appeared first on Markus Neteler Consulting.

GRASS GIS 8.3.1 released

What’s new in a nutshell

The GRASS GIS 8.3.1 maintenance release provides more than 60 changes compared to 8.3.0. This new patch release brings in important fixes and improvements in GRASS GIS modules and the graphical user interface (GUI) which stabilizes the new single window layout active by default.

Some of the most relevant changes include: fixes for r.watershed which got partially broken in the 8.3.0 release; and a fix for installing addons on MS Windows with g.extension.

Translations continue in Weblate, which automatically creates pull requests with the translated chunks. We’d like to thank the translators of all languages for their ongoing support!

GRASS GIS 8.3.1 graphical user interface

Full list of changes and contributors

For all 60+ changes, see our detailed announcement with the full list of features and bugs fixed at GitHub / Releases / 8.3.1.

Thanks to all contributors!

Software downloads

Binaries/Installers download

Further binary packages for other platforms and distributions will follow shortly, please check at software downloads.

Source code download

First time users may explore the first steps tutorial after installation.

About GRASS GIS

The Geographic Resources Analysis Support System (https://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities. It can be used either as a stand-alone application, as backend for other software packages such as QGIS and R, or in the cloud. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).

The GRASS Dev Team

The post GRASS GIS 8.3.1 released appeared first on Markus Neteler Consulting.

How Oslandia invests in OpenSource

You may be wondering where Oslandia’s name is coming from ? Or maybe you already know ? In this article we focus on the “OS” part of Oslandia : OpenSource !

Oslandia positions itself as IT expert in the field of OpenSource geographical information systems. QGIS is namely one of the proheminent opensource softwares for the geospatial industry. This position is a key element of our business model.

But do you know how we work behind the scene ? This article will give you an opportunity to discover some of our contributions to the OpenSource ecosystem.

Principles

Our general business model is based on projects we carry out for our clients. They fund us to design and implement solutions adapted to their needs and requirements. Part of these developments consist in core development of Opensource software. This allows us to contribute actively to FOSS4G components.

But this funding method makes it complicated to fund maintenance, or new exploratory developments, as well as communication, community management or other tasks necessary for healthy opensource projects.

As a consequence, we introduced at Oslandia a mechanism of internal OpenSource project grants.

These grants constitute self-investment from the company into the OpenSource ecosystem, and can be applied to new projects, research and development or existing projects.

This mechanism has multiple interests :

  • For opensource projects : maintenance and new contributions
  • For Oslandia : image and potential new business opportunities
  • For the team : work on projects that matter to them

These OpenSource grants consist in a large range of possible tasks, as we often say : “Opensource projects are not only code”. Instead of developers, we prefer the term contributors. Development, code review, maintenance, documentation, community management, communication, each collaborator can choose the type of task to focus on.

We differentiate software maintenance grants and opensource project grants. We call the latter “OpenSource mini-projects

Software maintenance consists in refactoring, bugfixing, packaging, release management… All these tasks need dedicated time which is difficult to fund directly on client’s project.

Opensource mini-projects grants are specific opensource proposal which can be submitted by any collaborator on any subject. We then vote on the best proposal and the team can start working on the subject within the allocated budget.

Some numbers

We allocate around 5% of the global production time to software maintenance grants. Our Opensource maintenance grant for 2022 is therefore approximately 190 days of work. It mainly focus on QGIS, PostGIS, QWC2, Giro3D and a few other components we actively maintain.

We also allocate 5% of the global production time to opensource mini-projects grants. It represents an additional 190 days of work for 2022.

Oslandia therefore invests almost 400 days of work into the OpenSource ecosystem, outside of direct contributions for client’s projects.

Opensource Mini-projects

OpenSource mini-projects grants are submitted by Oslandia’s collaborators and focus on various task and thematics : innovation, development, design, prototyping, communication or any other kind of Opensource contribution.

Proposals have to define goals, deliverables, planning, team and needed budget. Then we evaluate the proposals given the following criteria :

  • proposal coherency ( e.g. deliverables vs budget )
  • alignment with Oslandia’s strategy
  • innovation level
  • business opportunities
  • fun and motivation
  • impacts in terms of communication
  • links with other projects at Oslandia
  • possibility of extra R&D funding

We then vote on best proposal and manage these mini-projects just as a client project.

Examples

QGIS

The maintenance grant on QGIS allowed us to work on the following tasks :

  • Bugfixing
  • Code review for PRs submitted by other developers
  • Code refactoring
  • Documentation
  • Packaging pipeline
  • OSGeo4W improvement

OpenSource mini-projects grants

During the year of 2022, we worked on the following mini-projects :

In 2023 we will continue to work on these projects, and others ! for example pg_featureserv, py3dtiles, infoclimat website, MapProxypgRouting

Conclusion

This investment mechanism allows Oslandia to be an opensource “pure player” and contribute actively to these OpenSource projects and to the OpenSource ecosystem as a whole.

Should you be interested in our contribution model, or if you have any question regarding our internal OpenSource grant program, do not hesitate to contact us : [email protected] !

GRASS GIS 8.0.1 released

This is an update release of the GRASS GIS 8.0 series.

Download source code tarball at

For further release details (bug fixes and enhancements), see

The GRASS GIS 8.0.1 release provides more than 20 fixes and improvements with respect to the release 8.0.0.

What’s Changed

Important fixes

  • utils/mkhtml: fix print warning/fatal message during compilation to not break offline compilations (#2139) by @tmszi
  • r.quantile, r.stats.quantile, lib/stats: fix quantile algorithm (#2108) by @metzm

Module fixes

  • r.colors: fix order of rules (#2167) by @metzm
  • r.geomorphon: combine profile parser rules to allow usage as pygrass module (#2154) (#2226) by@ninsbl
  • TGIS: fix semantic label (#2215) by @metzm
  • t.rast.mapcalc: transfer semantic label (#2219) by @metzm
  • t.rast.neighbors: transfer semantic label (#2218) by @metzm

GUI

Portability

  • MS Windows packaging: add missing dlls from MSYS (#2204) by @ninsbl

Documentation

  • CONTRIBUTING.md: describe the fork process more in detail by @pesekon2
  • Fix typo in v.import documentation (#2142) by @agiudiceandrea
  • howto_release.md: cleanup of marketing section by @neteler
  • manual pages: remove unneeded SVN Date entry and selected cleanup (#2143) by @neteler

Singularity

Full Changelog: 8.0.0...8.0.1
Visit also the release page

Thanks to all contributors!

The post GRASS GIS 8.0.1 released appeared first on Markus Neteler Consulting.

GRASS GIS 8.0.0 released! Finally…

Overview of changes

After more than 3 year of development the first stable release GRASS GIS 8.0.0 is available. Efforts have concentrated on making the user experience even better, providing many new useful additional functionalities to modules and further improving the graphical user interface.

Breaking news: new graphical user interface with entirely rewritten startup sequence!

This re-establishes user experience compatibility with QGIS and other connected software packages.

The GRASS GIS 8.0.0 release provides more than 1,400 fixes and improvements with respect to the release 7.8.6.

With the introduction of the semantic label raster metadata class, the temporal database was modified to version 3. Hence, to be able to read and process GRASS 7.x space-time datasets, users will be prompted to run t.upgrade. If users want to read newly created space-time datasets back in GRASS 7.x, they can run t.downgrade.

Launching the software

The user experience of the graphical user interface has been completely rewritten: no more clumsy selection screens – just enter the menu system directly! And on command line, GRASS GIS now starts versionless, i.e. as grass. Enjoy!

Download and detailed list of changes

Thanks to all contributors!

GRASS GIS 8.0.0RC2 contributors

The post GRASS GIS 8.0.0 released! Finally… appeared first on Markus Neteler Consulting.

GRASS GIS 8.0.0RC2 released

Overview of changes

After more than 3 year of development the first stable release GRASS GIS 8.0.0 is available. Efforts have concentrated on making the user experience even better, providing many new useful additional functionalities to modules and further improving the graphical user interface.

Breaking news: new graphical user interface with entirely rewritten startup sequence!

This re-establishes user experience compatibility with QGIS and other connected software packages.

The GRASS GIS 8.0.0 release provides more than 1,300 fixes and improvements with respect to the release 7.8.6.

With the introduction of the semantic label raster metadata class, the temporal database was modified to version 3. Hence, to be able to read and process GRASS 7.x space-time datasets, users will be prompted to run t.upgrade. If users want to read newly created space-time datasets back in GRASS 7.x, they can run t.downgrade.

Launching the software

The user experience of the graphical user interface has been completely rewritten: no more clumsy selection screens – just enter the menu system directly!

And on command line, GRASS GIS now starts versionless, i.e. as grass.

Download and detailed list of changes

See https://github.com/OSGeo/grass/releases/tag/8.0.0RC2

Thanks to all contributors!

GRASS GIS 8.0.0RC2 contributors

The post GRASS GIS 8.0.0RC2 released appeared first on Markus Neteler Consulting.

(Fr) Oslandia recrute : Ingénieur(e) développement d’applications SIG ( Python / SQL / QGIS ) – OSL2110A

Sorry, this entry is only available in French.

GRASS GIS 7.8.5 released

What’s new in a nutshell Zanzibar Mapping Initiative data processed in OpenDroneMap and interpolated respecting building footprints using v.surf.icw

As a follow-up to the previous GRASS GIS 7.8.4 we have published the new release GRASS GIS 7.8.5 with more than 80 improvements. This minor release offers new wxGUI fixes across the tree. Also the addon extension manager received various stability fixes. VRT raster map with tiled raster maps can now be properly exported and imported in the native GRASS GIS raster format.

The overview of new features in the 7.8 release series is available at new features in GRASS GIS 7.8. See also our detailed announcement with the full list of changes and bugs fixed at https://trac.osgeo.org/grass/wiki/Release/7.8.5-News.

Binaries/Installer download:

Source code download:

First time users may explore the first steps tutorial after installation.

About GRASS GIS

The Geographic Resources Analysis Support System (https://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics or in the cloud. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).

The GRASS Development Team, Dec 2020

The post GRASS GIS 7.8.5 released appeared first on Markus Neteler Consulting.

OSGeo at FOSDEM 2021 (Online) – Call for Participation for the geospatial devroom

Call for Participation for FOSDEM 2021 geospatial devroom

Submission Deadline: 26.12.2020

FOSDEM is the largest free and open source developers meeting in Europe, with 8000+ participants. It normally takes place in Brussels (Belgium).

FOSDEM 2021 will be an online event. We will have a geospatial devroom again. The geospatial devroom will be on Saturday, 6 February 2021. In 2021 it will be our opportunity to bring our global community to FOSDEM.

The geospatial devroom is the place to talk about the state of the art of open, geo-related data, free and open source geospatial software and its ecosystem. This includes standards and tools, e.g. spatial databases, online mapping tools, geospatial services, used for collecting, storing, delivering, analysing, and visualizing geodata.

We are looking for you to be part of the geospatial devroom. The geospatial devroom aims to host talks about geospatial topics, GIS, and geodata projects, standards and open data.

Please submit your talk till 26. December 2020.

You find detailed information about the CfP at:
https://www.osgeo.org/foundation-news/osgeo-at-fosdem-2021-online-call-for-participation/

We hope you will submit a talk or come around at FOSDEM 2021.

See you at FOSDEM 2021

The post OSGeo at FOSDEM 2021 (Online) – Call for Participation for the geospatial devroom appeared first on Markus Neteler Consulting.

(Fr) Entretien avec Vincent Picavet

Sorry, this entry is only available in French.

Remarks on SVN-trac to GitHub migration

GRASS GIS is an open source geoinformation system which is developed by a globally distributed team of developers. Besides the source code developers also message translators, people who write documentation, those who report bugs and wishes and more are involved.

1. Early days… from pre-Internet to CVS and SVN

While GRASS GIS is under development since 1982 (no typo!) it has been put into a centralized source code management system in December 1999. Why so late? Because the World Wide Web (WWW) became available in the 1990s along with tools like browsers and such, followed by the development of distributed source code management tools. We moved on 29th Dec 1999 (think Y2K bug) the entire code into our instance of CVS (Concurrent Versioning System). With OSGeo being founded in 2006, we migrated the CVS repository to SVN (Subversion for the source code management) and trac (bug and wish tracker) on 8 Dec 2007. See here for historic details on our various bug trackers.

2. Time to move on: git

Now, after more than 10 years using SVN/trac time had come to move on and join the large group of projects managing their source code in git (see also our related Wiki page on migration). Git comes with numerous advantages, yet we needed to decide which hosting platform to use. Options where github.com, gitlab.com, gitlab or gitea on OSGeo infrastructure, or other platforms. Through a survey we found out that the preference among contributors is GitHub. While not being open source itself it offers several advantages: it is widely known (good to get new developers interested and involved), numerous OSGeo projects are hosted there under the GitHub “OSGeo organization“.

If all fails (say, one day GitHub no longer being a reasonable choice) the import of our project from GitHub to GitLab is always possible. Indeed, we meanwhile mirror our code on OSGeo’s gitea server.

Relevant script code and migration ticket:

Relevant steps:

  • migrated SVN trunk -> git master
  • migrated and tagged release branches (milestones)
  • deleted “develbranch6” (we compared it to “releasebranch_6_4” and didn’t discover relevant differences)
  • Fix commit messages (yes, we really wanted to be brave, updating decades of commit messages!):
    • references to old RT tracker tickets (used Dec 2000 – Dec 2006)
    • references to old GForge tracker tickets (used Jan 2007 – Dec 2008)
    • references to other trac tickets (#x -> https://trac.osgeo.org/…)

3. Source code migration: the new git repositories

  • github repository “grass” (repo)

    • Source code from 1999 to present day (SVN-trunk -> git-master)
    • all 7.x release branches
  • github repository “grass-legacy” (repo)

    • separate repository for older GRASS GIS releases (3.2, 4.x, 5.x, 6.x), hence source code now available in git since 1987!
  • github repository “grass-addons” (repo)

    • repository for addons
  • github repository “grass-promo” (repo)
    • repository for promotional material
  • github repository “grass-website” (repo)
    • repository for upcoming new Website

4. Remarks on the “grass-legacy” repository

What special about it:

  • the source code goes back to 1987!
  • file timestamps (which I tried to preserve for decades :-) have been used to reconstruct the source code history (e.g., releasebranch_3_2)
  • junk files removed (plenty of leftover old binary files, files consisting of a special char only etc)
  • having this grass-legacy repo available in parallel to the main grass repo which contains the  recent source code we have a continuous source code coverage from 1987 to today in git.
  • size is about 250MB

What’s missing

  • the 4.3 source code doesn’t have distinct timestamps. Someone must have once packaged without mtime preservation… a pity. Perhaps a volunteer may fix that by carrying over the timestamps from GRASS GIS 4.2 in case the md5sum of a file is identical (or so).

5. Trac issue migration

A series of links had to be updated. Martin Landa invested days and days on that (thanks!!). He used the related GDAL efforts as a basis (Even Rouault: thanks!). As the date for the trac migration we selected 2007-12-09 (r25479) as it was the first SVN commit (after the years in CVS). The migration of trac bugs to github (i.e. transfer of trac ticket content) required several steps:

Link updates in the ticket texts:

  • links to other tickets (now to be pointed to full trac URL). Note that there were many styles of referring in the commit log message which had to be parsed accordingly
  • links to trac wiki (now to be pointed to full trac URL)
  • links source code in SVN (now to be pointed to full trac URL)
  • images and attachments (now to be pointed to full trac URL)

Transferring:

  • “operating system” trac label into the github issue text itself (following the new issue reporting template)
  • converting milestones/tickets/comments/labels
  • converting trac usernames to Github usernames
  • setting assignees if possible, set new “grass-svn2git” an assignee otherwise
  • slowing down transfer to match the 60 requests per second API limit rate at github

6. Fun with user name mapping

Given GRASS GIS’ history of 35+ years we had to invest major effort in identifying and mapping user names throughout the decades (see also bug tracker history). The following circumstances could be identified:

  • user present in CVS but not in SVN
  • user present in SVN but not in CVS
  • user present in both with identical name
  • user present in both with different name (well, in our initial CVS days in 1999 we often naivly picked our surnames like “martin”, “helena”, “markus”, “michael” … cute yet no scaling very much over the years!) as some were changed in the CVS to SVN migration in 2007, leading to
    • colliding user names
  • some users already having a github account (with mostly different name again)

We came up with several lookup tables, aiming at catching all variants. Just a “few” hours to dig in old source code files and in emails for finding all the missing email addresses…

7. Labels for issues

We cleaned up the trac component of the bug reports, coming up with the following categories which have to be visually grouped by color since the label list is just sorted alphabetically in github/gitlab:

  • Issue category:
    • bug
    • enhancement
  • Issue solution (other than fixing and closing it normally):
    • duplicate
    • invalid
    • wontfix
    • worksforme
  • Priority:
    • blocker
    • critical
    • feedback needed
  • Components:
    • docs
    • GUI
    • libs
    • modules
    • packaging
    • python
    • translations
    • unittests
    • Windows specific

Note that the complete issue migration is still to be done (as of Nov. 2019). Hopefully addressed at the GRASS GIS Community Sprint Prague 2019.

8. Setting up the github repository

In order to avoid users being flooded by emails due to the parsing of user contributions which normally triggers an email from github) we reached out to GitHub support in order to temporarily disable these notifications until all source code and selected issues were migrated.

The issue conversion rate was 4 min per trac bug to be converted and uploaded to github. Fairly slow but likely due to the API rate limit imposed and the fact that the migration script above generates a lot of API requests rather than combined ones..
Note to future projects to be migrated: use the new gihub import API (unfortunately we got to know about its existence too late in our migration process).

Here out timings which occurred during the GRASS GIS project migration from SVN to github:

  • grass repo: XX hours (all GRASS GIS 7.x code)
  • grass-legacy repo: XX hours (all GRASS GIS 3.x-6.x code)
  • NNN issues: XX hours – forthcoming.

9. New issue reporting template

In order to guide the user when reporting new issues, we will develop a small template – forthcoming.

10. Email notifications: issues to grass-dev and commits to grass-commit

We changed the settings from SVN post-hook to Github commit notifications and they flow in smoothly into the grass-commit mailing list. Join it to follow the development.

Overall, after now several months of using our new workflow we can state that things work fine.

The post Remarks on SVN-trac to GitHub migration appeared first on GFOSS Blog | GRASS GIS and OSGeo News.

Funding for selective masking in QGIS is now complete!

Few months ago, Oslandia launched QGIS lab’s , a place to advertise our new ideas for QGIS, but also a place to help you find co funders to make dreams become reality.

The first initiative is about label selective masking, a feature that will allow us to achieve even more professional rendering for our maps.

Selective masking

 

Thanks to the commitment of the Swiss QGIS user group and local authorities, this work is now funded !

We now can start working hard to deliver you this great feature for QGIS 3.10

Thanks again to our funders

A last word, this is not a classical crowd funding initiative, but a classical contract for each funder.

No more reason not to contribute to free and open source software!

Happy birthday OSGeo!

On February 4, 2006 OSGeo held its first meeting in Chicago, with 25 participants representing 18 groups and over 20 different Open Source GIS projects, and 39 others participating via Internet Relay Chat. During the meeting, participants made important decisions in the formation and organization of the foundation, including the name, structure and purpose. The consensus reached in Chicago opened the way for the establishment of a productive and representative foundation.

Today we are happy to announce that the we have meanwhile over 32,800 unique subscribers in the huge list of over 290 OSGeo mailing lists!

And: check out the web site of the OSGeo foundation.

1. More to come this year!

… see here for the growing list of events

The post Happy birthday OSGeo! appeared first on GFOSS Blog | GRASS GIS and OSGeo News.

GRASS GIS 7.4.2 released

We are pleased to announce the GRASS GIS 7.4.2 release

What’s new in a nutshell

After a bit more than four months of development the new update release GRASS GIS 7.4.2 is available. It provides more than 50 stability fixes and improvements compared to the previous stable version 7.4.1. An overview of the new features in the 7.4 release series is available at New Features in GRASS GIS 7.4.

Efforts have concentrated on making the user experience even better, providing many small, but useful additional functionalities to modules and further improving the graphical user interface. Segmentation now support extremely large raster maps. Dockerfile and Windows support received updates. Also the manual was improved. For a detailed overview, see the list of new features. As a stable release series, 7.4.x enjoys long-term support.

Binaries/Installer download:

Source code download:

More details:

See also our detailed announcement:

About GRASS GIS

The Geographic Resources Analysis Support System (https://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities in a single integrated software suite. GRASS GIS includes tools for spatial modeling, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It also provides the capability to produce sophisticated presentation graphics and hardcopy maps. GRASS GIS has been translated into about twenty languages and supports a huge array of data formats. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).

The GRASS Development Team, October 2018

The post GRASS GIS 7.4.2 released appeared first on GFOSS Blog | GRASS GIS and OSGeo News.

GRASS GIS 7.2.0RC1 released

We are pleased to announce the first release candidate of GRASS GIS 7.2.0

What’s new in a nutshell

This is the first release candidate of the upcoming major release GRASS GIS 7.2.0.

The new GRASS GIS 7.2.0RC1 release provides more than 1900 stability fixes and manual improvements compared to the stable releases 7.0.x.

hexagons_python_editorAbout GRASS GIS 7: Its graphical user interface supports the user to make complex GIS operations as simple as possible. The updated Python interface to the C library permits users to create new GRASS GIS-Python modules in a simple way while yet obtaining powerful and fast modules. Furthermore, the libraries were significantly improved for speed and efficiency, along with support for huge files. A lot of effort has been invested to standardize parameter and flag names. Finally, GRASS GIS 7 comes with a series of new modules to analyse raster and vector data, along with a full temporal framework. For a detailed overview, see the list of new features. As a stable release series, 7.2.x enjoys long-term support.

Binaries/Installer download:

Source code download:

More details:

See also our detailed announcement:

http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures (overview of new 7 stable release series)

First time users may explore the first steps tutorial after installation.

About GRASS GIS

The Geographic Resources Analysis Support System (http://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities in a single integrated software suite. GRASS GIS includes tools for spatial modeling, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It also provides the capability to produce sophisticated presentation graphics and hardcopy maps. GRASS GIS has been translated into about twenty languages and supports a huge array of data formats. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).

The GRASS Development Team, October 2016

The post GRASS GIS 7.2.0RC1 released appeared first on GFOSS Blog | GRASS GIS Courses.

How to effectively get things changed in QGIS

I’ve been heavily involved in the open source QGIS mapping project for a number of years now. During this time I’ve kept a close watch on the various mailing lists, issue trackers, stackexchange, tweets and other various means users have to provide feedback to the project. Recently, I’ve started to come to the conclusion that there’s a lot of fundamental confusion about how the project works and how users can get changes made to the project. Read on for these insights, but keep in mind that these are just my thoughts and not reflective of the whole community’s views!..

Firstly – QGIS is a community driven project. Unlike some open source projects (and unlike the proprietary GIS offerings) there is no corporate backer or singular organisation directing the project. This means two things:

  1. The bad news: No-one will do your work for you. QGIS has been created through a mix of user-led contributions (ie, users who have a need to change something and dive in and do it themselves) and through commercially supported contributions (either organisations who offer commercial QGIS support pushing fixes because their customers are directly affected or because they’ve been contracted by someone to implement a particular change). There HAS been a number of volunteer contributions from developers who are just donating their time (for various reasons), but these contributions are very much the minority.
  2. The good news: YOU have the power to shape the project! (And whenever I say “you” – I’m referring directly to the person reading this, or the company you work for. Just pretend it’s in 24 point bold red blinking text.) Because QGIS is community driven (and not subject to the whims of any one particular enterprise) every user has the ability to implement changes and fixes in the program.

So how exactly can users get changes implemented in QGIS? Well, let’s take a look at all the possible different ways that changes get made and how effective each one is:

  1. YOU can make the changes yourself. This implies that you have the c++/Python skills required to make the changes, are able to find your way around the source code, and push through the initial hurdles of setting up a build environment and navigating git. This can be a significant time investment, but the ultimate result is that you can make whatever changes you want, and so long as your pull request is accepted you’ll get your changes directly into QGIS. You’ll find the QGIS team is very open to new contributors and will readily lend a hand if you need assistance navigating the source or for advise on the best way to make these changes. Just ask!
  2. YOU (or your employer) can pay (or “sponsor”) someone to make the changes on your behalf. Reinvesting some of those savings you’re making through using an open source program back into the program itself is a great idea, and everyone benefits. There’s numerous organisations who specialise in QGIS development (eg… my own consultancy, North Road). You can liaise with these organisations to get them to make the changes on your behalf. This is probably the most effective way of getting changes implemented. These organisations all have a history with QGIS development and this experience generally translates to much faster development then if you code it yourself. It’s also somewhat of a shortcut – if you hire a core QGIS developer to make your changes, then you can be confident that they are familiar with the coding style, policies, and long-term goals of the project and accordingly can get the changes accepted rapidly. The obvious down side of paying for changes is that, well, it costs money. Understandably, not everyone has the resources available to do this.
  3. Following on from option 2 – if you can’t directly sponsor changes yourself, you could help indirectly raise funds to pay for the changes. This is a great way to get changes implemented, because everyone has the power to do this. You could seek out similar organisations/users who have the same need and pool your resources, get involved with the local QGIS user group and raise funds together, organise a crowd-funding campaign, etc.
  4. Ask a developer to make the changes for you. This is not terribly effective – you’re basically asking someone to work for free, and take time away from their family/job/hobbies/social life to do work for you. That said, it does sometimes happen, and here’s a few reasons I can think of why:
    • You’ve build up enough “karma” within the project through other contributions. If someone has been heavily involved in the non-development side of the project (eg translations, documentation, helping users out on mailing lists/stackexchange, organising hackfests or user groups, etc) then developers are much more likely to want to help them out in turn.
    • You’ve got a fantastic idea which has just never occurred to anyone before. By bringing it to the attention of a developer you might trigger the “wow, I could really benefit from that too!” impulse which is hard-wired into some of us!
    • It’s a particularly interesting or challenging problem, and sometimes developers just like to extend themselves.
  5. (For bugs only) File a bug report, and hope it gets picked up in one of the pre-release bug fixing sprints. This is basically the same as option 2 – expect that in this case someone else (the QGIS steering committee) is paying for the development time. There’s no way of guaranteeing that your bug will get fixed during this time though, so it’s not a particularly reliable approach if the fix is critical for you.

Finally, there’s two more very ineffective approaches:

  1. File a bug report/feature request, and wait. This isn’t very effective, because what you’re doing is basically the same as 1-4 above, but just waiting for someone else to either do the work or sponsor the changes. This might happen in a week, or might take 10 years.
  2. Complain about something and hope for the best. This is… not very effective. No-one is particularly motivated to help out someone who is being a jerk.

That’s it. Those are the ONLY ways changes get made in QGIS. There’s no other magical short-cuts you can take. Some of these approaches are much more effective than others, and some require skills or resources which may not be available. If you want to see something change in QGIS, you need to take a look at these options and decide for yourself which best meets your needs. But please, just don’t choose option 7!

Update: a follow up to this article was published

New stable release of GRASS GIS 7.0.0!

The GRASS GIS Development team has announced the release of the new major version GRASS GIS 7.0.0. This version provides many new functionalities including spatio-temporal database support, image segmentation, estimation of evapotranspiration and emissivity from satellite imagery, automatic line vertex densification during reprojection, more LIDAR support and a strongly improved graphical user interface experience. GRASS GIS 7.0.0 also offers significantly improved performance for many raster and vector modules: “Many processes that would take hours now take less than a minute, even on my small laptop!” explains Markus Neteler, the coordinator of the development team composed of academics and GIS professionals from around the world. The software is available for Linux, MS-Windows, Mac OSX and other operating systems.

Detailed announcement and software download:
http://grass.osgeo.org/news/42/15/GRASS-GIS-7-0-0/

About GRASS GIS
The Geographic Resources Analysis Support System (http://grass.osgeo.org/), commonly referred to as GRASS GIS, is an open source Geographic Information System providing powerful raster, vector and geospatial processing capabilities in a single integrated software suite. GRASS GIS includes tools for spatial modeling, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It also provides the capability to produce sophisticated presentation graphics and hardcopy maps. GRASS GIS has been translated into about twenty languages and supports a huge array of data formats. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).

The post New stable release of GRASS GIS 7.0.0! appeared first on GFOSS Blog | GRASS GIS Courses.

QGIS 2.6 ‘Brighton’ released

In the new release of QGIS 2.6.0 a series of new features have been added concerning

  • General: new features and bugfixes,
  • DXF export (improvements),
  • Map Composer (enhancements),
  • Processing (including a new modeler implementation),
  • QGIS Server (improvements),
  • Symbology (including user interface improvements),
  • User Interface with improvements.

A visual changelog is available for more details with lots of screenshots.

Congratulations to all QGIS developers! Looking forward to see the Fedora RPM available…

You can download QGIS 2.6 at http://qgis.org/en/site/forusers/download.html

The post QGIS 2.6 ‘Brighton’ released appeared first on GFOSS Blog | GRASS GIS Courses.

Video: GRASS GIS development visualization from 1999 to 2013

Watch how the community based GRASS GIS 6.4 software development evolved! You can see how the individual contributors modify and expand the source code:

  • Dec 29, 1999: GRASS GIS 5.0 is being stored in an online source code repository in December 1999…
  • Dec 02, 2000: The developers work on all parts of the code…
  • Jan 15, 2002: Working on the future GRASS GIS 5.1 release
  • Nov 25, 2002: Starting GRASS 5.1 development with code restructuring
  • Jun 14, 2004: GRASS GIS 5.7 released in June 2004
  • Nov 09, 2004: Source code restructuring to get a better directory layout (all other developers waiting…)
  • Nov 09, 2004: … thousands of files are modified in this operation …
  • Nov 10, 2004: All developers resume their activities after the restructuring
  • Jan 10, 2005: Preparing the GRASS GIS 6.0.0 release…
  • Apr 09, 2005: GRASS GIS 6.0.0 published, release branch being split off from trunk for easier maintenance
  • Feb 22, 2006: Release of GRASS GIS 6.0.2 and new source code refactoring startedApr 05, 2006: Heavy development activity in trunk (development branch) …
  • Oct 25, 2006: GRASS GIS 6.2.0 released in October 2006
  • Apr 10, 2007: Preparing the GRASS GIS 6.2.2 release…
  • Jun 16, 2007: GRASS GIS 6.2.2 released in June 2007
  • Nov 01, 2007: Raster and vector modules being actively maintained…
  • Apr 02, 2007: New graphical user interface development speeding up (wxGUI)
  • Feb 20, 2008: Copyright statements prettified in many files
  • May 31, 2008: New GRASS 6 development branch being split off from trunk (which becomes GRASS 7)
  • Jun 10, 2008: Developers moving over to new branch
  • Feb 23, 2009: GRASS 6.4 release branch split off from GRASS 6 development branch
  • Apr 03, 2009: GRASS GIS 6.4 preparations starting…
  • Feb 24, 2010: Intense maintenance in GRASS 6.4 release branch
  • Sep 15, 2010: GRASS GIS 6.4.0 released in September 2010
  • Apr 12, 2011: GRASS GIS 6.4.1 released in April 2011
  • Jun 27, 2011: GRASS GIS 6.4.svn matures for the upcoming 6.4.2 release
  • Aug 16, 2011: Intense maintenance in GRASS 6.4 release branch (GRASS GIS 7 development not shown here)…
  • Feb 19, 2012: GRASS GIS 6.4.2 released in February 2012
  • Nov 13, 2012: Backporting graphical user interface bugfixes from GRASS GIS 7 to GRASS GIS 6.4
  • Apr 17, 2013: Further maintenance in GRASS 6.4 release branch
  • Jul 10, 2013: Fixing odds ‘n ends for the new stable release
  • Jul 27, 2013: GRASS GIS 6.4.3 released in July 2013

The corresponding timeline is also available at
http://grass.osgeo.org/home/history/releases/

THANKS TO ALL CONTRIBUTORS!
http://grass.osgeo.org/development/

Rendering: Markus Neteler
Audio track editing: Duccio Rocchini & Antonio Galea

Music:
Le bruit peut rendre sourd – Track 6/18 Album “Sensation electronique” by Saelynh (CC-BY-NC-ND) http://www.jamendo.com/en/track/1236/le-bruit-peut-rendre-sourd

Software used:
Gource software: http://code.google.com/p/gource/ (GPL)
OpenShot video editor: http://www.openshotvideo.com/ (GPL

The post Video: GRASS GIS development visualization from 1999 to 2013 appeared first on GFOSS Blog | GRASS GIS Courses.

  • Page 1 of 2 ( 23 posts )
  • >>
  • opensource

Back to Top

Sustaining Members