Facebook Translator Runs on Neural Networks
Why Facebook is the most dangerous competitor for every translation agency
Communicating in your native language is always easier than in a foreign language.
Facebook understands this very clearly and is constantly working on improving their translator. For this company, this function is particularly important, since, perhaps, its main mission is to connect people from all over the world.
More recently, the translation was carried out by “pressing a button”, now users of Facebook can read translated posts of their foreign friends at once. For those who are fluent in several languages and prefer to read the original text, there is a special “show original” button. Each user can evaluate the translation, so analysts will be able to continue collecting feedback and improve the algorithm.
Since the beginning of this decade first experiments of implementing their own machine translator into the social network were tested in the company. Users were allowed to offer their own translation option, which was evaluated by the rating system. In a case that their version gained the highest score, it could become the main one.
In comparison with classic translators, translation was available butnot for the entire page, only for a specific post. This function can be extremely convenient if you need to translate only a specific part of the text, rather than read everything completely in a broken language, for example, it occurs in Google translator. When reading a volume amount of information, switching between languages takes a lot of time, it can also sometimes lose the point which you stopped at as a result of updating the page, and also it’s tedious. In that time the availability of the translation was only in five languages: Chinese, Japanese, Korean, Russian and Taiwanese. Now this list has grown to 90 languages.
Eventually, the company introduced its own application for the translation of posts. It can be used absolutely free by any social network user. To activate the function, you need to select an additional language in the settings. For example, if you have a lot of Hispanic friends, then in the settings select the Spanish language. When you publish a post in English, for them it will be visible in Spanish. First, this technology used machine translation models based on simple phrases. Now this is a new turn of developing artificial intelligence – neural networks.
Translation using neural networks
The company has focused on this architecture of artificial intelligence (AI), because it is important for them to make the world closer and for this it is necessary to provide a translation of high quality. Neural networks allow the algorithm to process all information in general, considering different aspects: context, slang, abbreviations, accumulated glossary of specialized vocabulary and even common typos. Thereby the accuracy of the translation significantly improves. The image below shows the translation before and after using of neural networks.
The source states that the BLEU indicator (bilingual evaluation understudy – used to estimate machine translation) has grown by an average of 11% for all languages supported by the system. According to the words of company itself, this is an important milestone on the way to providing service in any language. According to the statistics, 4.5 billion translations are processed daily in the social network.
Specialists of the development department David Grangier and Michael Auli reveal the secrets of the success using artificial intelligence. Modern AI systems are built on recurrent neural networks (RNN), while the Facebook system uses convolutional neural networks (CNN). RNN is characterized by a strict sequence of data analysis: the system translates the entire sentence word by word. In the case of CNN, various aspects of the data are considered simultaneously. This style of calculations is classed as parallel and much better suited for graphics accelerators, the most widely used for learning the most advanced neural networks. Thus, CNN provides a more thorough solution to the task, allowing to analyze the structure of sentences at a higher level.
Put the pieces together
Machine translation is a two-stage process. For people it is absolutely natural to immediately understand a foreign sentence in our language, but the computer needs first to translate it into its own language and only then to the required one. A person subconsciously estimates a lot of probabilities. For example, the word “fire” can be both a verb in various roles (to set fire, to shoot, to light), and a noun in different roles (fire like flame, fire like shooting). For a correct identification of the role of this word in a specific sentence, we apply to the context.
To simulate these processes, the technology of multi-level attention uses the simultaneous nature of convolutional networks, allowing AI to access different parts of the text to understand their meaning during translation, which is not possible for a regular machine translation. Therefore, in recent years convolutional neural networks have become popular in the use of text translation.
Facebook employees believe that their models can be designed for more than just machine translation. According to their words, this method allows to build a logical structure at the top of the text. Their network can be used in any scenario where a computer needs to understand the text and express something: for example, to retell the text.
In previous studies of CNN applied to the translation it did not exceed RNN. Nevertheless, due to the architectural potential of CNN, the research team of the company – Facebook Artificial Intelligence Research (FAIR) began the research that led to the creation of a translation model demonstrating CNN’s high efficiency for translation purposes. The large calculative efficiency of CNN has the potential to scale translation and reach more than 6,500 languages worldwide.
Facebook, as always, sticks to its principles and therefore the source code of the modeling of the FAIR’s sequence is available under license so that other researchers can create their own models for translation. In 2016, Facebook reported that they developed software that could collect a glossary of slang terms. The new software is configured in such a way in order to identify and analyze the slang used in the published messages, and then try to predict one that will become popular only after a while. Initially, it was assumed that users will have the opportunity to add and edit words in the glossary, but so far such a function has not been presented.
It is worth notice that Google is also actively implementing this technology in its translator. They began using deep neural networks, after what the quality of translations provided by the service improved significantly. Now they continue to support translation service with AI sequentially to all languages.
As a result, this method is an alternative machine translation architecture, which opens up new opportunities for other text processing tasks. New discoveries occur from year to year. Now there are ways of multipurpose translation “on-the-fly” for conferences and personal use. In dialog systems, for example, it allows neural networks to focus on individual parts of the conversation, such as two separate facts, and link them together to better respond to the difficult questions.
