Experimental Ruby I18n extensions: Pluralization, Fallbacks, Gettext, Cache and Chained backend

posted: July 19th, 2009 · by: Sven

in: Globalization, Programming · tagged as: , ·  24 comments »

Since we’ve come up with I18n, that single “Ruby gem to rule them all”, we’ve always said we wanted new functionality to be tried out in plugin-land, giving it a good round of battle-testing and discussion. We also said we’d plan for another “round” of evaluating these things and considering to include them to I18n itself.

The I18n gem has probably been in production for long enough that we can be sure the API works well enough. Also, in release 0.2.0 internals were cleaned up quite a bit and the API has further consolidated.

So, my feeling is that we can start moving things to I18n that have proven useful in plugin-land – reviewing and reassessing their implementation and sticking to a modular design.

Here’s an overview of a few recent changes to the I18n gem library following these considerations.

Backend::Simple < Backend::Base

In Ruby what is the most obvious, elegant and maintainable pattern to extend an existing class’ or object’s functionality? No, the answer to that is definitely not in using alias_method_chain. It’s simply including a module to that class. You probably knew that already ;)

We’ve done this with the I18n Simple backend before but one needs to extend the Simple backend first in order to then inject additional modules to the inheritance chain so that these modules’ methods would be able to call super and find the original implementation.

To make this a bit easier I’ve moved the original Simple backend implementation to a new Base backend class and simply extend the (otherwise empty) Simple backend class from it (see here and here). This way you now do not need to extend the Simple backend class yourself but you can directly include your modules into it:

module I18n::Backend::Transformers
  def translate(*args)
    transform(super)
  end

    def transform(entry)
    # your Transformers logic
    end
end

I18n::Backend::Simple.send(:include, I18n::Backend::Transformers)

I have no clue what your Transformers module could do exactly but that’s the point about extensible libraries, isn’t it? In any case this is simply the pattern that the new, experimental Pluralization, Fallbacks, Gettext and Cache modules use that I wanted to talk about :)

Pluralization

Out-of-the-box pluralization for locales other than :en has been recurring requests for the I18n gem. Even though it was easy to extend the Simple backend to plug in custom pluralization logic and there are working backends doing that (e.g. in Globalize2) there does not seem to be a point in still rejecting a basic feature like this from being included to I18n.

I’ve thus added a Pluralization module that was largely inspired by Yaroslav’s work. It can be included to the Simple backend (or other compatible backend implementations) and will do the following things:

  • overwrite the existing pluralize method
  • try to find a pluralizer shipped with your translation data and if so use it
  • call super otherwise and use the default behaviour

One can ship pluralizers (i.e. lambdas that implement locale specific pluralization algorithms) as part of any Ruby translation file anywhere in the I18n.load_path. The implementation expects to find them with the key :pluralize in a (newly invented) translation metadata namespace :i18n. E.g. you could store an Farsi (Persian) pluralizer like this:

# in locales/fa.rb
{ :fa  => { :i18n => { :pluralize => lambda { |n| :other } } } }

And include the Pluralization module like this:

require "i18n/backend/pluralization" 
I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)

We still have to figure out how to actually ship a bunch of default pluralizers best, but you can find a complete list of CLDR’s language plural rules compiled to Ruby here (part of ours test suite).

Locale Fallbacks

Another feature that was requested quite often, too, is Locale fallbacks. Simple backend just returns a “translation missing” error message or raises an exception if you tell it so. It won’t check any other locales if it can’t find a translation for the current or given locale though.

There were proposals for a minimal fallback functionality that just checks the default locale’s translations if a translation is not available for the current locale. Globalize2 on the other hand ships with a quite powerful Locale fallbacks implementation that also enforces RFC 4646/47 standard compliant locale (language) tags.

I’ve discussed this with Joshua and we’ve decided to extract a simplified version from Globalize2’s fallbacks that makes the RFC 4646 standard compliance an optinal feature but still allows enough flexibility to define arbitrary fallback rules if you need them. If you don’t define anything it will just use the default locale as a single fallback locale.

Again enabling Locale fallbacks is just a matter of including the module to any compatible backend:

require "i18n/backend/fallbacks" 
I18n::Backend::Simple.send(:include, I18n::Backend::Fallbacks)

This overwrites the Base backend’s translate method so that it will try each locale given by I18n.fallbacks for the given locale. E.g. for the locale :"de-DE" it will try the locales :"de-DE", :de and :en until it finds a result with the given options. If it does not find any result for any of the locales it will then raise a MissingTranslationData exception as usual.

The :default option takes precedence over fallback locales, i.e. it will first evaluate a given default option before falling back to another locale.

You can add custom fallback rules to the I18n.fallbacks instance like this:

# use Spanish translations if Catalan translations are missing:
I18n.fallbacks.map(:ca => :"es-ES")
I18n.fallbacks[:ca] # => [:ca, :"es-ES", :es, :en]

If you do not add any custom fallback rules it will just use the default locale and the default locales fallbacks:

# using :"en-US" as a default locale:
I18n.default_locale = :"en-US" 
I18n.fallbacks[:ca] # => [:ca, :"en-US", :en]

If you want RFC 4646 standard compliance to be enforced for your locales you can use the Rfc4646 Tag class:

I18n::Locale::Tag.implementation = I18n::Locale::Tag::Rfc4646

This will make a locale “de-Latn-DE-1996-a-ext-x-phonebk-i-klingon” fall back to the following locales in this order:

de-Latn-DE-1996-a-ext-x-phonebk-i-klingon
de-Latn-DE-1996-a-ext-x-phonebk
de-Latn-DE-1996-a-ext
de-Latn-DE-1996
de-Latn-DE
de-Latn
de

Most of the time you probably won’t need anything like this. Thus we’ve used the (much cheaper) I18n::Locale::Tag::Simple class as the default implementation. It simply splits locales at dashes and thus can do fallbacks like this:

de-Latn-DE-1996
de-Latn-DE
de-Latn
de

Should be good enough in most cases, right :)

Gettext

The difference between the Gettext support and all the other extensions discussed here is that I haven’t had a real use for it myself – so far, so please consider this stuff highly experimental. It shares the fact that people requested it in one way or the other though, so I’d appreciate feedback about it.

Gettext support comes with three parts, only two of them being relevant to the user:

To include the accessor helpers to your application you can simply include the module whereever you need them (e.g. in your views):

require "i18n/helpers/gettext" 
include I18n::Helpers::Gettext

The backend extension is, again, a matter of including the module to a compatible backend (e.g. Simple):

require "i18n/backend/gettext" 
I18n::Backend::Simple.send(:include, I18n::Backend::Gettext)

Now you should be able to include your Gettext translation (*.po) files to the I18n.load_path so they’re loaded to the backend and you can use them as usual:

I18n.load_path += Dir["path/to/locales/*.po"]

Please note that following the Gettext convention this implementation expects that your translation files are named by their locales. E.g. the file en.po would contain the translations for the English locale.

Translate Cache

Right from the beginning people benchmarked the I18n gem implementation and compared their numbers against native Hash lookups or other, less rich implementations of similar APIs. Also, thedarkone has experimented with a Fast backend that implements some significant optimizations.

For the I18n gem itself we’ve refrained from overly extensive “early optimizations” and only applied some tweeks that made the implementation cheaper in obvious ways. This rather conservative approach actually paid out: the clean and readable implementation made the Simple backend refactorings and extensions (like those discussed here) almost trivial.

On the other hand, even though calls to I18n don’t actually add that much load to an application sparing resources obviously is an important concern.

The probably both most effective and least intrusive way of doing that is simply caching all calls to the backend. Again you can include the Cache layer by simply including the module:

require "i18n/backend/cache" 
I18n::Backend::Simple.send(:include, I18n::Backend::Cache)

As we do not provide any particular cache implementation though you also have to set your cache to the I18n backend. The cache layer assumes that you use a cache implementation compatible to ActiveSupport::Cache. If you’re using this in the context of Rails this is a matter of one line:

I18n.cache_store = ActiveSupport::Cache.lookup_store(:memory_store)

Obviously this pluggable approach again allows you to pick whatever cache is most appropriate for your setup. For example ActiveSupport out of the box ships with compatible implementations for plain memory, drb demon and compressed, synchronized and plain memcached storage – wow. These options should be sufficient for 99% of all Rails apps.

Whatever cache implementation you use the I18n backend cache layer will simply cache results from calls to translate (and will do the right thing with MissingTranslationData exceptions raised in your backend).

For that it relies on the assumption that calls to the backend are idempotent: “A unary operation is called idempotent if, whenever it is applied twice to any value, it gives the same result as if it were applied once.”. I18n’s library design does not garantuee that by itself but in all practical cases will behave this way unless your doing really weird things with it. Basically, make sure you only pass objects to the translate method that respond to the hash method correctly. If you use custom lambda translation data make sure they always return the same values when passed the same arguments.

Chained backend

The Chain backend is another feature ported from Globalize2. It can be used to chain multiple backends together and will check each backend for a given translation key.

This is useful when you want to use standard translations with a Simple backend but store custom application translations in another backends. E.g. you might want to use the Simple backend for managing Rails’ internal translations (like ActiveRecord error messages) but use a database backend for your application’s translations.

To use the Chain backend you can instantiate it and set it to the I18n module. You can then add chained backends through the initializer or backends accessor:

# preserves an existing Simple backend set to I18n.backend
I18n.backend = I18n::Backend::Chain.new(I18n::Backend::ActiveRecord.new, I18n.backend)

The implementation assumes that all backends added to the Chain implement a lookup method with the same API as Simple backend does.

Cool, what’s next?

If you’re interested in any of these features, please try these things out and provide some feedback on the rails-i18n mailinglist. They might make it into the next I18n gem release or not depending on the amount of “real world feedback” we’ve gotten until then.

Also, by now there’s ton of good stuff that uses or extends I18n to do useful things with it on Github. I’ve collected a few things that I found particular suited for being evaluated, maybe merged and potentially included to I18n here: Interesting I18n repositories.

I’ve also done some work on my i18n-tools repository recently, so you hopefully you can expect some news from that front, too.

Shameless plug

In case you find this stuff useful I’m always happy to receive another recommendation on working-with-rails :)

Leave a comment

24 Comments

  1. erwann said July 27th, 2009 at 12:29 PM  

    Hi Sven,

    Good job! A thing I’d like to be able to do is a URL translation ie myapp.com/en/products/ would become myapp.com/es/productos/

    it would be nice for users and probably better for SEO

    Cheers! erwann

  2. Sven said July 27th, 2009 at 06:33 PM  

    Hey erwann,

    this is what we’re referring to as translated routes - it’s part of Rails’ scope, not I18n’s (which is supposed to work fine for any framework or app).

    People are asking a lot about this. Check out this repo: translated_routes I’ve never used it but I’ve been told it works.

  3. erwann said July 29th, 2009 at 02:49 PM  

    Thanks for the tip Sven!

  4. Home Design said May 5th, 2010 at 02:23 PM  

    Hi all this is what we’re referring to as translated routes - it’s part of Rails’ scope, not I18n’s (which is supposed to work fine for any framework* benz

  5. Cloud Hosting said August 11th, 2010 at 12:24 PM  

    Hi all this is what we’re referring to as translated routes - it’s part of Rails’ scope, not I18n’s (which is supposed to work fine for any framework* benz

  6. Exchange said September 14th, 2010 at 08:45 AM  

    Thanks for sharing this great article! That is very interesting Smile I love reading and I am always searching for informative information like this.

  7. leviafan said February 3rd, 2011 at 09:28 AM  

    Thanks for review and sapmles. Ruby are really super cool. best acai berry products

  8. QQQ said February 7th, 2011 at 06:39 PM  

    Finally we kissed and the passion scale went sky high and I knew I was onto a good thing - sex was a certainty free porn videos. She never hesitated when I began to fondle her breasts and she willingly exposed them for me mobile porn. They were firm and I suspected a breast enhancement but said nothing - they still felt good and I was enjoying them and gradually working my way further south free porn tube. She was a step ahead of me and before I could completely undress her she moved on me atk hairy and I was suddenly having my pants pulled down and I was enjoying one of he best cock sucking hairy pussy experiences I had ever had. ABB728019394

  9. caroline said March 2nd, 2011 at 06:19 AM  

    Yeah this was talked about on a article site and I came to a similar conclusion.

  10. Nuborn said March 13th, 2011 at 08:09 AM  

    Hi Sven,

    Ruby is very powerful as you have shown. This appears to be a translated route and within the scope. Regards, Nuborn is the owner of the blog the best flat iron and enjoys coding or trying to code.

  11. p said March 22nd, 2011 at 12:00 PM  

    cfcfcfcfcfcfcfcfcfcf

  12. Cams said March 30th, 2011 at 07:48 PM  

    :salutation => lambda { |gender| gender == “m” ? “Mr. {{name}}” : “Mrs. {{name}}” }

    This kept giving errors where it showed Mr. only on my output form. Thanks…

  13. imarion said March 31st, 2011 at 01:20 PM  

    A thing I’d like to be able to do is a URL translation ie myapp.com/en/products/ would become myapp.com/es/productos/

    it would be nice for users and probably better for SEO. download torrent

  14. ?????? ??? ???? said April 12th, 2011 at 09:51 AM  

    yes, the implementation assumes that all backends added to the Chain implement a lookup method with the same API as Simple backend does…

  15. Stefan said April 12th, 2011 at 07:05 PM  

    Schöne Sache, die Fallbacks auf meiner fähren urlaub Seite haben mir letztens ziemliche Probleme gemacht und ich habe einige Email von den Usern bekommen. Ich probiere es mal mit deinen Tipps, danke dir.

  16. side sleeper pillow said April 22nd, 2011 at 07:04 AM  

    Nicely written article, Knowledgeable and informative post. I’m really glad I came my way along your site. Keep posting, I really like the whole topic. Thanks for sharing.

  17. side sleeper pillow said April 22nd, 2011 at 07:06 AM  

    Nicely written article, Knowledgeable and informative post. I’m really glad I came my way along your site. Keep posting, I really like the whole topic. Thanks for sharing.

  18. vibrating alarm clock said May 2nd, 2011 at 02:26 PM  

    Great article, I have to say I thoroughly enjoyed it.

  19. cd duplication london said May 12th, 2011 at 01:27 PM  

    Informative post..thanks for sharing! I’ll try using the codes when needed.

  20. Okey oyunu said May 12th, 2011 at 03:24 PM  

    Tüm dünya art?k okey oyunu oynuyor. Y?llard?r bir çok oyun program? olmas?na ra?men, içlerinden en güzeli olarak nitelendirebilece?imiz tek bir site göze çarpmaktad?r. Di?er tüm okey oyunu programlar?n?n aksine ücretsiz olmas? ve 3 boyutlu olarak hizmet vermesi mükemmel bir geli?medir. Sizlerde www. okey-oyunu.com adresinden bu e?siz okey oyununu indirebilirsiniz. Kullan?m? çok basit ve Türkçe dil seçene?i ile kolayl?kla oyuna ba?layabilirsiniz. ?ster kendi ülkenizden, isterseniz dünyan?n tüm farkl? bölgelerinden diledi?iniz oyun odalar?n? seçerek, oyuna hemen ba?layabilirsiniz. Okey oyunu oynamak için art?k arkada? bile araman?za gerek kalmadan, bilgisayar?n?zdan 100 binlerce üye ile online olarak okey oyununu oynaman?n zevkine varabilirsiniz.

  21. dissertations said May 20th, 2011 at 02:15 PM  

    This post is useful for my research writing, thanks!

  22. p said May 21st, 2011 at 05:04 PM  







    chaussures nike lunarhaze+ white black red [cn3340] - €49.99 : nike tn,requin tn,tn nike,tn requin,nike requin,basket tn,chaussures tn
    nike tn,requin tn,tn nike,tn requin,nike bw,basket tn,chaussures tn : nike free 3.0 3 - Kvinner sko Menn sk
    Nous avons une satisfaction garantie à 100%. Si vous n’êtes pas satisfait avec votre produit dans les 365 jours suivant la réception vous pouvez l’envoyer de nouveau à nous pour un remboursement complet.

    Il est 24 heures période de traitement une fois le paiement re?u. Une fois que les chaussures sont expédiée, vous recevrez un email de notre part avec numéro de suivi afin que vous mai suivre votre commande. Vous aurez vos produits dans 3-5 jours d’affaires (sans compter les week-ends ou jours fériés). La plupart des commandes sont traitées et re?ues par les clients dans les 3 jours.



  23. porno said May 23rd, 2011 at 01:03 PM  

    good comment. thanks you friends.

    I’ve surfed the net more than three hours today, however, I haven’t found such useful information. Thanks a lot, it is really useful to me

  24. porno said May 23rd, 2011 at 01:03 PM  

    I do agree with all of the ideas you have presented in your post. They’re really convincing and will definitely work. Still, the posts are too short for newbies. Could you please extend them a bit from next time? Thanks for the post.

Sorry, comments are closed for this article.

artweb design
Sven Fuchs
Grünberger Str. 65
10245 Berlin, Germany


http://www.artweb-design.de

Fon +49 (30) 47 98 69 96
Fax +49 (30) 47 98 69 97