
Internationalisation Support
============================

This document is divided into three parts:

* Quick reference - use this is you just want to quickly remember how to mark
  a word or phrase for translation
* QGIS Plugin - this is in-depth information on how the translation framework
  is set up for Qt based code components
* Library - this is in-depth information on how the translation framework is
  set up for the pure python library implementation.

Quick Reference
---------------

The translation system works differently for different parts of the project:

* Strings that appear in the InaSAFE graphical user interface (gui module). These include buttons, help strings, menu items etc
* Strings that appear in the Python code which constitutes the rest of InaSAFE including the impact functions
* Strings that appear in data and keywords used by InaSAFE

Flagging strings for translation is done differently for these three cases
and are treated separately in the next three sections. However, making and
deploying the translations is unified and takes care of all three and
described in section "Make translations" below.


Translation of strings in the gui module
........................................

* Classes and anonymous functions that do not inherit from the class QObject
  should use the format
  :samp:`QCoreApplication.translate('Riab', 'Translations loaded')`
* Classes that inherit from QObject should use the form :samp:`self.tr('foo')`
* String replacement arguments should be provided using the QString
  :samp:`arg` method. Example: :samp:`self.tr('Error: %1').arg(message)`


Translation of strings *not* in the gui such as impact functions
................................................................

* Import the gettext helper e.g::

   from safe.common.utilities import ugettext as _

   (Note that you must have the sequence "import ugettext" in the statement. It will not work as part of a multiple import such as import x, y, ugettext as _)
* All strings should be wrapped using the _ helper e.g::

    tr('Are there enough shelters available for %i people?') % displaced

* The library will use at run-time the :samp:`LANG` environment variable which
  should be set to the iso code e.g. 'id' for 'Indonesia' of the Locale
  you wish to use. This is done automatically for you by the QGIS Plugin, but
  if you are using the riab library in another context, be sure to set it before
  using any libary functions if you want them to return translated strings e.g.::

      os.environ['LANG'] = 'id'
      # do stuff with riab lib


Translation of strings that appear at runtime
.............................................

This applies e.g. to titles of layers or attribute names in data.

The translation system works by scanning the Python code for strings marked as described above. However, it has know way of knowing about titles of layers or names that appear in datasets processed by InaSAFE. However, if such names are known a priori they can be made visible to the translation system as follows:

* Edit the file::

    common/dynamic_translations.py

  and add the name to the dictionary "names". E.g. :samp:`'college' = tr('college')`
* Update the translation strings as described in the section below
* Make impact functions refer to the dynamic translations e.g. as in this example::

      from common.dynamic_translations import names as internationalised_values

      if building_type in internationalised_values:
          building_type = internationalised_values[building_type]


Make translations
-----------------

When new strings have been added as described above the procedure to translate them is (example is given for LANG=id):

* run :samp:`make update-translation-strings` to collect all strings marked for translation
* Using either an editor or the tool Qt Linguist provide translations in the files

  * safe/i18n/id/LC_MESSAGES/inasafe.po
  * safe_qgis/i18n/inasafe_id.ts

* run :samp:`make compile-translation-strings` to make the translations available to InaSAFE


QGIS Plugin
-----------

The QGIS Plugin uses QtLinguist. this free, open source application can
be downloaded and used to translate the Qt translation files.


Preparing for a release
.......................

As developer, before a release you should do:

* run :samp:`make update-translation-strings` to update the translation files
* distribute the .ts files under :samp:`gui/i18n` to the translators
* instruct them to open the .ts file for their locale with QtLinguist
* commit the returned file from the translator when all strings have been
  translated
* run :samp:`make compile-translation-strings` to create binary loadable
  translations
* ensure the .qm files are distributed with the release (the .ts files do not
  need to be released)


.. note:: Translators should take heed - when refreshing the .ts file in
   QtLinguist, the file *must be closed* (:menuselection:`File --> Close`) and
   then reopened. Simply loading doing (:menuselection:`File --> Open`) and
   choosing the same file you already have in the workspace will not refresh the
   workspace with any new changes that appeared on disk.

.. note::
   *make update-translation-strings* is non destructive. That is,
   you can safely run it as many times as you like, new strings will be added
   to it, deprecated strings will be left in place and already translated
   strings will remain translated.


Adding a new language
.....................

To add a new language, edit the :samp:`gui/riab.pro` file and append the new
locale to the bottom of the file. For example, to add South African english
as a new locale, change this::

   TRANSLATIONS = i18n/riab_id.ts

to this::

    TRANSLATIONS = i18n/riab_id.ts\
                   i18n/riab_en_ZA.ts

Save and close the .pro file. Next run :samp:`make update-translation-strings`
to generate the new .ts file under gui/i18n. Don't forget to :samp:`git add`
the new file and place it under version control.

InaSAFE Library Translations
----------------------------

Low level gettext usage
.......................

Translation is done using gettext.

Create the initial .po file::

   xgettext -d id -o i18n/id/LC_MESSAGES/riab.po i18ntest.py

After you create the initial .pot, you need to specify the characterset and
encoding for that file (by editing it with a text editor). For example::

   "Content-Type: text/plain; charset=UTF-8\n"
   "Content-Transfer-Encoding: 8bit\n"


If you add strings to the file, update the .pot file by adding -j option::

   xgettext -j -d id -o i18n/id/LC_MESSAGES/riab.po i18ntest.py

Next, you can make the .po files available to translators. Recent versions of
QtLinguist support translations of .po files, so you can use a similar process
to that described in the gui section above.

When the .po file has been updated, it should be committed to the git
repository (e.g. via a pull request from the user's repository clone, or by
emailing the .po file to a developer). After receiving an updated .po file,
it should be compiled to a :samp:`.mo` file (which is a binary representation
of the strings)::

   msgfmt -o i18n/id/LC_MESSAGES/riab.mo i18n/id/LC_MESSAGES/riab.po

The :samp:`msgfmt` command accepts one or more input files which can be
merged into a single :samp:`.mo`.

.. note:: These functions are wrapped as make scripts so you should not need to
   use them on a day to day basis.

.. _library-release-label:

Preparing for a release
.......................

As developer, before a release you should do:

* run :samp:`make update-translation-strings` to update the translation files
* distribute the .po files under :samp:`i18n/<locale>/LC_MESSAGES/riab.po` to
  the translators
* instruct them to open the .po file for their locale with QtLinguist
* commit the returned file from the translator when all strings have been
  translated
* run :samp:`make compile-translation-strings` to create binary loadable
  translations (.mo files)
* ensure the .mo files are distributed with the release (the .po files do not
  need to be released)


.. note:: Translators should take heed - when refreshing the .po file in
   QtLinguist, the file *must be closed* (:menuselection:`File --> Close`) and
   then reopened. Simply loading doing (:menuselection:`File --< Open`) and
   choosing the same file you already have in the workspace will not refresh the
   workspace with any new changes that appeared on disk.

.. note:: *make update-translation-strings* is non destructive. That is,
   you can safely run it as many times as you like, new strings will be added
   to it, deprecated strings will be left in place and already translated
   strings will remain translated.



Adding a new language
.....................

To add a new language, edit the :samp:`Makefile` file and append the new
locale to the bottom of the file. For example, to add South African english
as a new locale, change this section::

   update-translation-strings: compile

copy one of the existing stanzas e.g.::

   xgettext -j -d id -o i18n/id/LC_MESSAGES/riab.po \
      storage/test_io.py \
      impact_functions/flood/flood_building_impact.py

Save and close the Makefile file. Next you need to create the initial translation
stringlist for that locale by creating a locale directory and running the
command above without the :samp:`-j` (j is for 'join' which merges old content
with new, avoiding destroying previous translated strings). So for example you
would run from the command line::

   mkdir -p i18n/en_ZA/LC_MESSAGES/
   xgettext -d id -o i18n/en_ZA/LC_MESSAGES/riab.po \
      storage/test_io.py \
      impact_functions/flood/flood_building_impact.py

The above adding a hypothetical new translation for South African English. After
the inital creation of your .po files using the above commands, you can update
them anytime the strings in the library have been changed by doing::

   make update-translation-strings`

to generate the updated .po file under i18n/en_ZA/LC_MESSAGES. Don't forget to
:samp:`git add` the new directory and file and place them under version control.

To convert the .po file to a binary .mo file (which is used at runtime for the
actual translation), follow the :ref:`library-release-label` section above.

Adding a new source file for translation
........................................

To add a new source file, edit the :samp:`Makefile` file and append the new
sourcefile to the bottom of the file list in the
:samp:`update-translation-strings` section. For example::

   xgettext -j -d id -o i18n/id/LC_MESSAGES/riab.po \
      storage/test_io.py \
      impact_functions/flood/flood_building_impact.py

Would become::

   xgettext -j -d id -o i18n/id/LC_MESSAGES/riab.po \
      storage/test_io.py \
      impact_functions/flood/flood_building_impact.py \
      impact_functions/flood/flood_population_fatality

The above adding the impact_function *flood_population_fatality* to the list of
translatable source files. Now run::

   make update-translation-strings`

to generate the updated .po file and make it available to translators. When the
translated file is returned, convert the .po file to a binary .mo file (which is
used at runtime for the actual translation), then follow the
:ref:`library-release-label` section above.

Sphinx Translation
------------------

Generic documentation on how to translate sphinx documentation is available
`here <http://sphinx.pocoo.org/latest/intl.html>`-.

Initial notes on the process::

   cd docs
   make gettext

Which will create pot files which can be found under build/locale. Note that
this make target needs to be tweaked so that it builds a unique directory
for each supported locale.
