Are there translation probabilities for major language pairs?

Question

Are there reasonable alignment probability files for major language pairs?

Specifically, given a target language and a target word/phrase, what are the translation equivalents in other major languages and what are the associated distributional probabilities of that word. For example, given language English and phrase "to be" the results for Spanish might be: ser (0.55) and estar (0.45).

Most likely from GIZA++ train ([prefix].actual.ti.final), but others would be interesting too.

Ideally news or user-generated data, but any reasonably large domain or a mix of domains is acceptable.

Answer 1

There are probabilities for English with all EU languages (except Irish and Croatian) available here.

Inside the .zip for each language pair English-X you will find files lex.e2x and lex.x2e.

They contain strictly unigram-to-unigram probabilities, which maintains simplicity but is an inherent limitation.

The files are the result of a 2014 paper by Aker, Lestari Paramita, Pinnis & Gaizauskas.

I also found a repo on GitHub which includes English-Arabic.

Answer 2

The best place to find translation equivalents is from Project Panlex -- According to their website, their mission is:

"To make panlingual lexical translation possible, the PanLex project is constructing a database designed to include translations among all words (and word-like phrases) of the languages of the world. We are gathering translations from thousands of dictionaries, glossaries, vocabularies, thesauri, word lists, wordnets, and other such published and unpublished sources. We subject the translations found there to a set of standards, so they may be combined into a unified network."

Here is a page linking to various web interfaces as well as actual downloadable data in csv, json, and xml formats.