FastGettext

Ruby GetText, but 12x faster + 530x less garbage + simple + clean namespace + threadsafe + extendable + multiple backends

Description

GetText but 3.5 x faster, 560 x less memory, simple, clean namespace (7 vs 34) and threadsafe!

It supports multiple backends (.mo, .po, .yml files, Database(ActiveRecord + any other), Chain, Loggers) and can easily be extended.

Code Quality Rank: L5

Monthly Downloads: 900,854

Programming language: Ruby

License: Ruby License

Tags: Internationalization     Time     Space     I18n    

Latest version: v2.0.3

README

FastGettext

GetText but 12 x faster, 530 x less garbage, clean namespace (8 vs 26), simple and threadsafe!

It supports multiple backends (.mo, .po, .yml files, Database(ActiveRecord + any other), Chain, Loggers) and can easily be extended.

Example Rails application

Comparison

Hash FastGettext GetText ActiveSupport I18n::Simple Speed* 0.08s 0.14s 1.75s 3.75s Objects* 11K 15K 8017K 7107K Included backends db, yml, mo, po, logger, chain mo yml (db/key-value/po/chain in other I18n backends) *500.000 translations with ruby 2.5.3 through bundle exec rake benchmark

Setup

1. Install

gem install fast_gettext

2. Add a translation repository

From mo files (traditional/default)

FastGettext.add_text_domain('my_app', path: 'locale')

Or po files (less maintenance than mo)

FastGettext.add_text_domain('my_app', path: 'locale', type: :po)
# ignore_fuzzy: true to not use fuzzy translations
# report_warning: false to hide warnings about obsolete/fuzzy translations

Or yaml files (use I18n syntax/indentation)

# A single locale can be segmented in multiple yaml files but they all should be
# named with a `qq.yml` suffix, where `qq` is the locale name.
FastGettext.add_text_domain('my_app', path: 'config/locales', type: :yaml)

Or database (scaleable, good for many locales/translators)

# db access is cached <-> only first lookup hits the db
require "fast_gettext/translation_repository/db"
FastGettext::TranslationRepository::Db.require_models # load and include default models
FastGettext.add_text_domain('my_app', type: :db, model: TranslationKey)

3. Choose text domain and locale for translation

Do this once in every Thread. (e.g. Rails -> ApplicationController)

FastGettext.text_domain = 'my_app'
FastGettext.available_locales = ['de', 'en', 'fr', 'en_US', 'en_UK'] # only allow these locales to be set (optional)
FastGettext.locale = 'de'

4. Start translating

FastGettext supports all the translation methods of ruby-gettext with added support for block defaults. (to get *gettext methods, use FastGettext::TranslationAliased)

_() or gettext(): basic translation

extend FastGettext::Translation
_('Car') == 'Auto'             # found translation for 'Car'
_('not-found') == 'not-found'  # The msgid is returned by default

n_() or ngettext(): pluralization

n_('Car', 'Cars', 1) == 'Auto'
n_('Car', 'Cars', 2) == 'Autos' # German plural of Cars

You'll often want to interpolate the results of n_() using ruby builtin % operator.

n_('Car', '%{n} Cars', 2) % { n: count } == '2 Autos'

p_() or pgettext(): translation with context

p_('File', 'Open') == _("File\004Open") == "öffnen"
p_('Context', 'not-found') == 'not-found'

s_() or sgettext(): translation with namespace

s_('File|Open') == _('File|Open') == "öffnen"
s_('Context|not-found') == 'not-found'

The difference between s_() and p_() is largely based on how the translations are stored. Your preference will be based on your workflow and translation editing tools.

pn_() or pngettext(): context-aware pluralized

pn_('Fruit', 'Apple', 'Apples', 3) == 'Äpfel'
pn_('Fruit', 'Apple', 'Apples', 1) == 'Apfel'

sn_() or sngettext(): without context pluralized

sn_('Fruit|Apple', 'Apples', 3) == 'Äpfel'
sn_('Fruit|Apple', 'Apples', 1) == 'Apfel'

N_() and Nn_(): make dynamic translations available to the parser.

In many instances, your strings will not be found by the ruby parsing. These methods allow for those strings to be discovered.

N_("active"); N_("inactive"); N_("paused") # possible value of status for parser to find.
Nn_("active", "inactive", "paused")        # alternative method
_("Your account is %{account_state}.") % { account_state: _(status) }

Managing translations

mo/po-files

Generate .po or .mo files using GetText parser (example tasks at gettext_i18n_rails)

Tell Gettext where your .mo or .po files lie, e.g. for locale/de/my_app.po and locale/de/LC_MESSAGES/my_app.mo

FastGettext.add_text_domain('my_app', path: 'locale')

Use the original GetText to create and manage po/mo-files. (Work on a po/mo parser & reader that is easier to use has started, contributions welcome @ get_pomo )

Database

Example migration for ActiveRecord The default plural separator is |||| but you may overwrite it (or suggest a better one...).

This is usable with any model DataMapper/Sequel or any other(non-database) backend, the only thing you need to do is respond to the self.translation(key, locale) call. If you want to use your own models, have a look at the default models to see what you want/need to implement.

To manage translations via a Web GUI, use a Rails application and the translation_db_engine

Rails

Try the gettext_i18n_rails plugin, it simplifies the setup. Try the translation_db_engine, to manage your translations in a db.

Setting available_locales,text_domain or locale will not work inside the environment.rb, since it runs in a different thread then e.g. controllers, so set them inside your application_controller.

# config/environment.rb after initializers
Object.send(:include, FastGettext::Translation)
FastGettext.add_text_domain('accounting', path: 'locale')
FastGettext.add_text_domain('frontend', path: 'locale')
...

# app/controllers/application_controller.rb
class ApplicationController ...
  include FastGettext::Translation
  before_filter :set_locale
  def set_locale
    FastGettext.available_locales = ['de', 'en', ...]
    FastGettext.text_domain = 'frontend'
    FastGettext.set_locale(params[:locale] || session[:locale] || request.env['HTTP_ACCEPT_LANGUAGE'])
    session[:locale] = I18n.locale = FastGettext.locale
  end

Advanced features

Abnormal pluralisation

Plurals are selected by index, think of it as ['car', 'cars'][index] A pluralisation rule decides which form to use e.g. in english its count == 1 ? 0 : 1. If you have any languages that do not fit this rule, you have to add a custom pluralisation rule.

Via Ruby:

FastGettext.pluralisation_rule = ->(count){ count > 5 ? 1 : (count > 2 ? 0 : 2)}

Via mo/pofile:

Plural-Forms: nplurals=2; plural=n==2?3:4;

Plural expressions for all languages.

default_text_domain

If you only use one text domain, setting FastGettext.default_text_domain = 'app' is sufficient and no more text_domain= is needed

default_locale

If the simple rule of "first available_locale or 'en'" is not sufficient for you, set FastGettext.default_locale = 'de'.

default_available_locales

Fallback when no available_locales are set

with_locale

If there is content from different locales that you wish to display, you should use the with_locale option as below:

FastGettext.with_locale 'gsw_CH' do
  FastGettext._('Car was successfully created.')
end
# => "Z auto isch erfolgriich gspeicharat worda."

Chains

You can use any number of repositories to find a translation. Simply add them to a chain and when the first cannot translate a given key, the next is asked and so forth.

repos = [
  FastGettext::TranslationRepository.build('new', path: '....'),
  FastGettext::TranslationRepository.build('old', path: '....')
]
FastGettext.add_text_domain 'combined', type: :chain, chain: repos

Merge

In some cases you can benefit from using merge repositories as an alternative to chains. They behave nearly the same. The difference is in the internal data structure. While chain repos iterate over the whole chain for each translation, merge repositories select and store the first translation at the time a subordinate repository is added. This puts the burden on the load phase and speeds up the translations.

repos = [
  FastGettext::TranslationRepository.build('new', path: '....'),
  FastGettext::TranslationRepository.build('old', path: '....')
]
domain = FastGettext.add_text_domain 'combined', type: :merge, chain: repos

Downside of this approach is that you have to reload the merge repo each time a language is changed.

FastGettext.locale = 'de'
domain.reload

Logger

When you want to know which keys could not be translated or were used, add a Logger to a Chain:

repos = [
  FastGettext::TranslationRepository.build('app', path: '....')
  FastGettext::TranslationRepository.build('logger', type: :logger, callback: ->(key_or_array_of_ids) { ... }),
}
FastGettext.add_text_domain 'combined', type: :chain, chain: repos

If the Logger is in position #1 it will see all translations, if it is in position #2 it will only see the unfound. Unfound may not always mean missing, if you choose not to translate a word because the key is a good translation, it will appear nevertheless. A lambda or anything that responds to call will do as callback. A good starting point may be examples/missing_translations_logger.rb.

Plugins

Want an xml version? Write your own TranslationRepository!

# fast_gettext/translation_repository/wtf.rb
module FastGettext
  module TranslationRepository
    class Wtf
      define initialize(name,options), [key], plural(*keys) and
      either inherit from TranslationRepository::Base or define available_locales and pluralisation_rule
    end
  end
end

Multi domain support

If you have more than one gettext domain, there are two sets of functions available:

extend FastGettext::TranslationMultidomain

d_("domainname", "string") # finds 'string' in domain domainname
dn_("domainname", "string", "strings", 1) # ditto
dp_("domainname", "context", "key")
ds_("domainname", "context|key")
dnp_("domainname", "context", "string", "strings")
dns_("domainname", "context|string", "strings")

These are helper methods so you don't need to write:

FastGettext.with_domain("domainname") { _("string") }

It is useful in Rails plugins in the views for example. The second set of functions are D functions which search for string in all domains. If there are multiple translations in different domains, it returns them in random order (depends on the Ruby hash implementation).

extend FastGettext::TranslationMultidomain

D_("string") # finds 'string' in any domain
Dn_("string", "strings", 1) # ditto
Dp_("context", "key")
Ds_("context|key")
Dnp_("context", "string", "strings")
Dns_("context|string", "strings")

Alternatively you can use merge repository to achieve the same behavior.

Block defaults

All the translation methods (including MultiDomain) support a block default, a feature not provided by ruby-gettext. When a translation is not found, if a block is provided the block is always returned. Otherwise, a key is returned. Methods doing pluralization will attempt a simple translation of alternate keys.

_('not-found'){ "alternative default" } == alternate default

This block default is useful when the default is a very long passage of text that wouldn't make a useful key. You can also instrument logging not found keys.

_('terms-and-conditions'){
  load_terms_and_conditions
  request_terms_and_conditions_translation_from_legal
}

# Override _ with logging
def _(key, &block)
  result = gettext(key){ nil } # nil returned when not found
  log_missing_translation_key(key) if result.nil?
  result || (block ? block.call : key)
end

FAQ

• Problems with ActiveRecord messages?
• Iconv require error in 1.9.2

Authors

Mo/Po-file parsing from Masao Mutoh, see vendor/README

Contributors

• geekq
• Matt Sanford
• Antonio Terceiro
• J. Pablo Fernández
• Rudolf Gavlas
• Ramón Cahenzli
• Rainux Luo
• Dmitry Borodaenko
• Kouhei Sutou
• Hoang Nghiem
• Costa Shapiro
• Jamie Dyer
• Stephan Kulow
• Fotos Georgiadis
• Lukáš Zapletal
• Dominic Cleal
• Tomas Strachota
• Martin Meier
• morcoteg
• Daniel Schepers
• Robert Graff

Michael Grosser [email protected] License: MIT, some vendor parts under the same license terms as Ruby (see headers)