Some common questions on getting started - Get on Rails with Globalize! Part 2 of 8

posted: December 3rd, 2006 · by: Sven

in: Globalization, Programming · tagged as: , , , , , , ·  21 comments »

In Part 1 of this series you’ve seen how easy it is to get up and running with Globalize. And you’ve seen what the most basic concepts of this multilanguage plugin for Ruby on Rails are, namely:

  • .t for ViewTranslations (arbitrary, static text)
  • the translates directive for ModelTranslations and
  • .loc to localize your date, time and number formats.

In this article we’ll talk about some common questions on getting started like:

  • How to setup your application to use Unicode
  • How to select and keep the current user’s locale
  • How to translate entire templates (instead of individual strings)
  • How to translate Rails ActiveRecord messages

(Update 06/12/06.: mention ActiveRecord::Multibyte and explain MySQL/Unicode character-set)

Let’s talk Unicode

As you’ve enabled your application to virtually support even the strangest languages on earth you now should start using Unicode throughout your application so that non-latin characters will be stored and displayed correctly. If you don’t, you’d probably get something like this:

Globalize.Unicode.Broken

... instead of this:

Globalize.Unicode.Ok

Globalize already includes jcode.rb and sets $KCODE to UTF8 (see vendor/plugins/globalize/init.rb) which basically means that Ruby will treat all input and output as Unicode encoded. So that’s already been taken care of.

(Beware though that there are several String methods that won’t work as expected with Unicode yet like length, reverse or slice. This is being worked on and it is neither Globalize’s nor JCode’s fault but due to Ruby’s own lack of Unicode support which reportedly will be fixed in Ruby 1.9/2.0.

As this will take some time Rails will stand in with ActiveSupport::Multibyte – which will be included in the upcoming Rails 1.2 release (thanks, Josh!) and allow all the standard String methods to work in Unicode. Watch a screencast about ActiveSupport::Multibyte here.)

So, essentially, to get Unicode being used throughout your app you’ll primarily need to make sure that it’s being used by a) the database and b) the browser.

... to your database …

To tell Rails that you want to use Unicode with your database add the line in config/database.yml:

 encoding: utf8    # for MySQL
 encoding: unicode # for PostgreSQL

For MySQL that’s the eqivalent to: SET NAMES utf8. For PostgreSQL it does: SET CLIENT_ENCODING TO unicode. For SQLite you just need to compile Unicode support in.

Also, you additionally need to configure MySQL itself to use utf8 as character set. There are some ways to do this. According to the MySQL Manual (see sections 10.3.1 through 10.3.4) you can set the character set at server, database, table and even column level. More specific settings overwrite a more general setting. E.g. you could do:

CREATE TABLE articles (
  ... 
) CHARSET=utf8;

... which would work. As you’re going to use migrations though (you do, don’t you?) you need to ensure that MySQL uses utf8 as the default character set for new tables – and you can achieve that by setting the character set at the database or even server level. For the database that’s as simple as:

[CREATE|ALTER] DATABASE db_name 
    CHARACTER SET utf8

Also, you’ll probably want to pay attention to your collation. The database server’s collation defines the way your data is sorted – which may cover all kinds of wired conventions. (For example, see this section in Wikipedia for some wired variations of sorting conventions regarding German Umlauts). You can set the collation you want to use in one go with the character set. E.g.:

[CREATE|ALTER] DATABASE db_name 
    CHARACTER SET utf8 
    COLLATION utf8_spanish_ci

... and to your clients

Next make sure that your application’s clients (i.e. the user’s browser) will decode your output (like HTML …) correctly. The safest (and standard-compliant) way is to send an appropriate header:

class ApplicationController < ActionController::Base
  before_filter :set_charset
  def set_charset
      content_type = headers["Content-Type"] || "text/html" 
      if /^text\//.match(content_type)
        headers["Content-Type"] = "#{content_type}; charset=utf-8" 
      end
  end
end

This will add the Content-Type header to our http response and signalize that we’ll be sending Unicode to the client. This also ensures (in modern browsers, that is) that the browser will return stuff encoded as Unicode (e.g. data sent as forms).

Also, to be nice to Safari add a Content-Type meta tag to your layout templates (such as default.rhtml):

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 

How to select and keep the current user’s locale

Like you’ve seen in Part 1 you can set the current locale by using Locale#set (e.g. Locale.set("de-DE")). But how do you know which one to set? There’s no one-fits-all answer, it depends. But here are some options you’ll probably want to think about.

To determine the users locale you could:

  1. use a locale that’s been suggested by the browser (using the <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4">accept-language</a> header)
  2. use a locale the users ip address seems to belong to
  3. offer (or force) the user to choose a locale manually
  4. use a default locale otherwise.

Once the users current locale is set (be it that you’ve decided to present a default language or that the user already has chosen the preferred language) you need to persist it throughout her requests somehow. Basically your options are:

  • Keep the locale in the URL as a …
    • parameter (like in http://site.com/home?lang=pl)
    • part of the path (like in http://site.com/pl/home)
    • subdomain (like in http://pl.site.com/home) or domain (like in http://site.pl/home)
  • Store it in the session or a cookie.

Now, to set your users locale you’d most likely do something like this in your ApplicationController:

before_filter :set_locale 
def set_locale 
  # somehow determine and set the locale 
  true 
end 

For example the following has been suggested:

This code sets the Globalize locale based on the http accept-language header field. Other options are also considered, in order or precedence:

  • Explicetely locale set on URL
  • Previous language selection (stored on user session)
  • Accept-Language field contents
  • Default locale
before_filter :set_locale

def set_locale
  default_locale = 'en-US'
  request_language = request.env['HTTP_ACCEPT_LANGUAGE']
  request_language = request_language.nil? ? nil : 
    request_language[/[^,;]+/]

  @locale = params[:locale] || session[:locale] ||
            request_language || default_locale
  session[:locale] = @locale
  begin
    Locale.set @locale
  rescue
    Locale.set default_locale
  end
end

Also, you might want to consider to improve this solution by checking the locale against an array of locales that are available in your application, like so:

locale = LOCALES.join(' ').match params[:locale] || default_locale

Beware though that using the session for something like this might yield unexpected behaviour (even if it might seem invitingly simple). In usability tests we’ve often seen users getting confused by this type of solution when they are switching back and forth between multiple browser windows. When a user changes the locale in one window it won’t change in the other windows of course – until he does the next click (or refresh). This type of behaviour might (probably depending on the nature of your application) lead to confusion and disorientation of some degree.

So, all in all you might be better off to avoid usability gotchas like these altogether, resort to a more RESTful approach and encode your locale directly in your URLs. When you’re going this route you also might find it useful to redirect the user when no locale parameter is provided. E.g., Szymek suggests in his Globalize example application:

def set_locale
  if !params[:locale].nil? && LOCALES.keys.include?(params[:locale])
    Locale.set LOCALES[params[:locale]]
  else
    redirect_to params.merge('locale' => Locale.base_language.code)
  end
end

How to localize your templates

You’re able to translate strings in templates using .t. But how can you localize entire templates?

Why you’d want to do that? A common example for this requirement is the need for right-handed layouts for languages such as Arabic and Hebrew. It might be sufficient to check Locale.active.language.direction() to conditionally load different style sheets using a before_filter in your ApplicationController in some cases. In other cases this just won’t cut it though.

Globalize therefor looks for localized templates for the current language using the following naming scheme:

<templatename>.<lang>.rhtml

If it does not find a template named like this, the default <templatename>.rhtml file will be used.

Of course this approach is most oftenly less DRY than just using one template for the markup and control the layout of the page through CSS – but that’s not always possible.

How to translate Rails ActiveRecord messages

This is probably one of the most frequent asked questions.

Joshua Harvey writes: “[...] Globalize was designed to handle this. Globalize automatically tries to translate ActiveRecord validation messages, so if you trigger a validation error, that message should show up in the globalize_translations table and be available for translation.”

Sounds cool. Let’s have a look at this.

First let’s create a model, e.g.:

class Page < ActiveRecord::Base  
  translates :title, :text
  validates_presence_of :title
end    

Now switch over to your script/console terminal and force an error message:

>> Locale.set "de-DE" # without an active Locale nothing interesting would happen
>> p = Page.new
>> p.save!
ActiveRecord::RecordInvalid: Validation failed: Title can't be blank

Great, let’s check the database:

>> Translation.find :first, :order => "id DESC" 
=> #<Globalize::ViewTranslation:0x2657ba0 @attributes={..., 
   "tr_key"=>"Title can't be blank", ..., "language_id"=>"1556", ...}>

Bingo. Globalize has inserted an empty recordset for the key “Title can’t be blank” into the globalize_translations table. The language with the id 1556 is German.

Joshua writes: “Because Globalize’s string substitution isn’t quite as sophisticated as we want it to be, it can’t translate field names separately from the validation message. This means you have to translate all combinations of field name/validation type. In many apps this isn’t that big a problem, but if it is in your case, let us know.”

Of course there’s more …

In Part 3 of this series we’ll have a look at Globalize’s more advanced features like:

  • Abstracting ViewTranslations (sprintf-like usage)
  • Singular and (multiple) plural ViewTranslations
  • Globalize’s Currency class
  • Piggyback translated ActiveRecord associations

Leave a comment

21 Comments

  1. Josh H said December 3rd, 2006 at 10:03 PM  

    Another great installment, Sven! This is just about the best documentation there is for Globalize.

    I have a few comments, also. First, you might want to mention that the upcoming Rails 1.2 release will include ActiveSupport::Multibyte, which will allow all the standard methods such as length and reverse to work in Unicode. Also, Unicode support in MySql is a very prickly subject. I'd mention that it's important to make sure the database tables are set to the utf-8 character set as well, otherwise all kinds of weird things are bound to happen.

    Again, thanks so much for this top-notch documentation.

  2. Sven said December 4th, 2006 at 05:44 AM  

    Hey! Thanks for the kind words, Joshua :)

    Yes, I remember having read something about the Multibyte stuff recently, but apperently I've lost track of that. I'll add it.

    I've never had any issues with MySql/Unicode. But yes, a note about the table character set is missing. I'll add that, too.

    Thanks!

  3. mylar said April 7th, 2007 at 12:38 AM  

    About how to determine the users locale and maintaning it, there is a little plugin that does right that using RFC2616 rules, and have a helper to produce a flags menu:

    http://agilewebdevelopment.com/plugins/localist

  4. Javier said May 22nd, 2007 at 01:29 PM  

    this didn’t work for me (on mysql 5.0.22):

    ALTER DATABASE my_db_name CHARACTER SET UTF-8;

    …but this did:

    ALTER DATABASE my_db_name CHARACTER SET UTF8;

    cheers!

  5. dc said May 24th, 2007 at 12:39 PM  

    hi -

    I’m not having much luck with this:

    <templatename>.<lang>.rhtml

    could you possibly provide more detail where to look on this feature?

    I tried:

    <templatename>.Japanese.rhtml <templatename>.ja-JP.rhtml <templatename>.ja.rhtml

    Locale.active.inspect

    shows: {“dateformat”=>nil, “currencydecimalsep”=>”.”, “thousandssep”=>”,”, “code”=>”JP”, “numbergroupingscheme”=>”western”, “englishname”=>”Japan”, “decimalsep”=>”.”, “id”=>”109”, “currencycode”=>”JPY”, “currencyformat”=>”¥%n”}>, @currencydecimalsep=”.”, @code=”ja-JP”, @currencyformat=”¥%n”, @decimalsep=”.”, @dateformat=nil, @thousandssep=”,”, @language=Japanese, @numbergroupingscheme=:western>

  6. Sven said May 26th, 2007 at 09:01 PM  

    @ Javier: yes, that’s right. Thanks for spotting this! I’ve fixed that typo.

    @ dc: I suspect that this might be some kind of a stupid bug in the current trunk? The only advice I really can give you here is to register to the Globalize users mailinglist and restate your problem there. Probably it will help to include some more information about the version of Globalize and Rails you are using. I hope that helps.

  7. churi said June 20th, 2007 at 07:42 PM  

    When you say:

    locale = params[:locale].match(LOCALES) || default_locale

    (locale being a string, LOCALES an array of strings)

    where is that ‘match’ implemented? It doesn’t work here, String#match accepts only regexps. This works:

    LOCALES.join(’ ‘).match params[:locale] || default_locale

    or, to be pedantic:

    LOCALES.any?{|l|l.match(/^#{params[:locale]}/}) ? params[:locale] : default_locale

  8. Sven said June 26th, 2007 at 02:39 PM  

    Hey churi!

    Thanks for pointing that out!

    I really don’t have a clue where this one comes from. Usually I doublecheck every bit of code that goes into these articles … but this looks like something that couldn’t have worked without some serious hacking into String#match. I’m going to replace that line with your first suggestion - I guess that’s enough in most cases.

  9. Neelesh said September 22nd, 2007 at 06:03 PM  

    How does this work in a multi user scenario? Consider a site which supports English and French. User A clicks on the french version and user B non the English version. On WEBricks, the same runtime is used (unlike CGI where a new interpreter is launched per request). Since Locale.set uses a class variable to store the locale, the locale is the same “across” the application and not for a user session. This means the locales will switch arbitrarily based on who clicks on which language.

    Am I missing something? This will work fine for a CGI environment, as a new ruby interpreter is launched per request. In all other cases (fastCGI/mongrel/WEBricks), the same runtime is shared and this may not work.

  10. Sven said September 23rd, 2007 at 03:22 PM  

    Hey Neelesh!

    Yes, it will work. And no, there’s no problem like this. :-)

    Rails is not multi-threaded, each request is processed from beginning to end before another request is started. So, there’s no chance of one user overwriting another user’s class variables.

    Or, put another way: “Rails synchronizes access to the ‘request-response loop’ - from when a request comes in, to when the response is returned to the browser, only one of those request-response actions can happen concurrently. If you want to handle more traffic, add instances.” (from “Threading in Rails”)

  11. Artem Vasiliev said October 16th, 2007 at 10:24 AM  

    the suggested set_character code

    def set_charset
        content_type = headers["Content-Type"] || "text/html"
        if /^text\//.match(content_type)
          headers["Content-Type"] = "#{content_type}; charset=utf-8"
        end
    end
    

    successfully screws partial rendering with RJS - response comes with Content-Type: text/html; charset=utf-8 instead of Content-Type: text/javascript; charset=utf-8 - at least with Rails 1.2.4 and Globalize for 1.2. Solution is to throw away set_character filter at all - encoding is already set to UTF8.

  12. Zoga said October 29th, 2007 at 05:01 PM  

    I can see that there is a way to translate entire template. But is there a way to translate entire view? What i want is that url /en/info/service includes /info/service.en.rhtml page, and if it does not exists, then proceed to /info/service.rhtml. I’m not talking here about templates - but simple views. THANKS

  13. Sven said October 30th, 2007 at 06:47 PM  

    Zoga,

    I’m not sure if I understand what you mean by “views, not templates”. I think Globalize just does what you describe what you want. :)

  14. Zoga said October 31st, 2007 at 10:43 PM  

    Thanks Sven for quick response and for this GREAT getting started guide!!

    You are right, globalize does just that. But for base language it will include ‘/info/service.rhtml’ - not the ‘/info/service.en.rhtml’ page.

  15. QQQ said February 7th, 2011 at 06:33 PM  

    Finally we kissed and the passion scale went sky high and I knew I was onto a good thing - sex was a certainty free porn videos. She never hesitated when I began to fondle her breasts and she willingly exposed them for me mobile porn. They were firm and I suspected a breast enhancement but said nothing - they still felt good and I was enjoying them and gradually working my way further south free porn tube. She was a step ahead of me and before I could completely undress her she moved on me atk hairy and I was suddenly having my pants pulled down and I was enjoying one of he best cock sucking hairy pussy experiences I had ever had. ABB728019394

  16. chat said March 31st, 2011 at 08:07 PM  

    The following cleaned up the issue:

    Dependencies.loadoncepaths -= Dependencies.loadoncepaths.select{|path| \ path =~ %r(^#{File.dirname(FILE)}) }

  17. Okey oyunu said May 12th, 2011 at 04:14 PM  

    Thanks for this article. Tüm dünya artik okey oyunu oynuyor. Yillardir bir çok oyun programi olmasina ragmen, içlerinden en güzeli olarak nitelendirebilecegimiz tek bir site göze çarpmaktadir. Diger tüm okey oyunu programlarinin aksine ücretsiz olmasi ve 3 boyutlu olarak hizmet vermesi mükemmel bir gelismedir. Sizlerde www.okey-oyunu.com adresinden bu essiz okey oyununu indirebilirsiniz. Kullanimi çok basit ve Türkçe dil seçenegi ile kolaylikla oyuna baslayabilirsiniz. Ister kendi ülkenizden, isterseniz dünyanin tüm farkli bölgelerinden dilediginiz oyun odalarini seçerek, oyuna hemen baslayabilirsiniz. Okey oyunu oynamak için artik arkadas bile aramaniza gerek kalmadan, bilgisayarinizdan 100 binlerce üye ile online olarak okey oyununu oynamanin zevkine varabilirsiniz.

  18. porno said May 22nd, 2011 at 01:14 PM  

    I do agree with all of the ideas you have presented in your post. They’re really convincing and will definitely work. Still, the posts are too short for newbies. Could you please extend them a bit from next time? Thanks for the post.

  19. porno said May 22nd, 2011 at 01:17 PM  

    I do agree with all of the ideas you have presented in your post. They’re really convincing and will definitely work. Still, the posts are too short for newbies. Could you please extend them a bit from next time? Thanks for the post.

  20. porno said May 22nd, 2011 at 01:55 PM  

    good comment. thanks you friends.

    I’ve surfed the net more than three hours today, however, I haven’t found such useful information. Thanks a lot, it is really useful to me

  21. porno said May 22nd, 2011 at 01:58 PM  

    good comment. thanks you friends.

    I’ve surfed the net more than three hours today, however, I haven’t found such useful information. Thanks a lot, it is really useful to me

Sorry, comments are closed for this article.

artweb design
Sven Fuchs
Grünberger Str. 65
10245 Berlin, Germany


http://www.artweb-design.de

Fon +49 (30) 47 98 69 96
Fax +49 (30) 47 98 69 97