How does the web browser make a request for specific languages?

The browser writes a value for the Accept-Language request header field that it sends to the web server. You can set this value in Preferences (Netscape) or Internet Options (Internet Explorer). If you choose multiple languages, they are sent to the server as a comma-delimited list in your preferred order.

With the settings above, the request header field will be Accept-Language = de,en-us,it,fr,pt-br,es.

How does the web server determine which language to serve? Is there a naming convention for multi-lingual web pages?

Deciding which page to serve is called content negotiation. Tim Berners-Lee's discussion of generic web pages and their language variants describes two different naming conventions, index.fr.html and index.html.fr. The Apache Web Server 2.0 compiles in content negotiation (the mod_negotiation module) by default. It postpends the two-letter language code to the URL and looks for files to serve. For example, if you set your browser preferred language to French and browse to http://www.openinternetlexicon.com, an Apache server would look for the file www.openinternetlexicon.com/index.html.fr. Unfortunately, postfixing the language code is not as desirable as infixing it - index.fr.html - which allows the server's operating system to use a familiar extension. Microsoft Internet Information Server negotiates language with an optional ISAPI filter. There does not appear to be a standard ISAPI filter for language negotiation on the market.

Server-driven Negotiation ( W3C HTTP/1.1 spec RFC2616 Section 12.1) If the selection of the best representation for a response is made by an algorithm located at the server, it is called server-driven negotiation. Selection is based on the available representations of the response (the dimensions over which it can vary; e.g. language, content-coding, etc.) and the contents of particular header fields in the request message or on other information pertaining to the request (such as the network address of the client). Server-driven negotiation is advantageous when the algorithm for selecting from among the available representations is difficult to describe to the user agent, or when the server desires to send its "best guess" to the client along with the first response (hoping to avoid the round-trip delay of a subsequent request if the "best guess" is good enough for the user). In order to improve the server's guess, the user agent MAY include request header fields (Accept, Accept-Language, Accept-Encoding, etc.) which describe its preferences for such a response.

Do web servers handle multi-lingual requests automatically?

Not normally. The web server must have multiple language versions of a web page in order to serve them. It needs to know how the language-variant web pages are named. Besides naming them with a URI (index.fr.html), it may be possible to transmit the language variance as metadata in the HTTP header. This is the direction of the WebDAV protocol being developed by the IETF.

PlanetarySales.com offers an ASP or Java component ($499) that allows a web server to respond with a page in the language that the browser requests. But this can be done on the server with Active Server Pages (ASP), or with JSP, PHP, or Perl, as this site shows.

What are some example sites which serve multi-lingual pages? Do they recognize the browser language request?

There are many multi-lingual sites, but most ask you to choose your language from their home page. Few respond to your browser settings for a preferred language. One who does (besides us) is Alis Technologies (though only in French and English). Another is Planetary Sales (in English, German, and Japanese).
An Apache 2.0 web server default installation supports several languages. Your browser must be set to a non-English language to see these work.

Do you know of some exemplary multilingual sites that we should list? Send us their URLs.

Are there any documents about the web that are translated into several languages as a guide for someone doing localization of web pages?

The Lernout & Hauspie and Berlitz GlobalNET web pages are available with brief amounts of aligned text in many languages. Logos has about 10 languages. None of these respond to the browser language request.

Bill Dunlap's Global Reach site is localized in nine languages.

Arlene H. Rinaldi's Netiquette User Guidelines is available in ten languages.

Kevin Werbach's Bare Bones Guide to HTML has been translated into more than twenty languages. Multilingual names for all the HTML elements are here.

Other HTML manuals in various languages (by different authors) contain lots of web terminology.

HTML Manual in French
HTML Manual in German
HTML Manual in Spanish
HTML Manual in Portuguese

How many people are online today in different languages/locales?

Bill Dunlap's Global Internet Statistics page reports regularly on the number of web users worldwide browsing in different languages. Also available are details by country and the growth of language populations on the web since 1996 with future projections.

Send us your questions , or add a comment (below) to this page.