We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [language]

For questions related to human language. Use 'programming' instead for questions related to programming languages.

Filter by
Sorted by
Tagged with
4
votes
0answers
104 views

List of inverse and polarity flip words required

I'm a computer science student and I'm doing a research on sentimental analysis with a large database. Therefore, i need to create dictionaries to mapping words. Now i'm facing the problem of "...
3
votes
2answers
252 views

Swedish Noun Gender List

I'm looking for a lexical resource on Swedish noun genders. In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
3
votes
2answers
135 views

Is there any picture corpus?

By "picture corpus" I mean the picture collection each illustrating one word in the usually used vocabulary, such as words in Google 10000 English I found on GitHub. For instance: vocabulary ...
3
votes
2answers
2k views

Offensive words - English and Spanish

Have a system generating four letter code and in testing just got up to AA-BJ ... and the BJ got me thinking. In a while the one after AR-SD could cause offence in the UK! Does anyone know of a list ...
3
votes
1answer
35 views

Where can I find subtitled recordings of university courses?

I'm doing a research on the language used by university professors and I'm looking for closed captioned videos from real university classes.
3
votes
3answers
147 views

For natives of each language, most common second language and proportion of people fluent in it

I am trying to prioritize what languages my software should be translated to. Regardless of other factors, I want to first look at how many people are native speakers of each language, example: ...
3
votes
1answer
210 views

source of character-level ngrams that include spaces

I'm looking for a source of character-level ngrams and counts that include spaces. I've found character-level ngram datasets, but they don't include spaces. Ideally, the dataset will include up to 8-...
3
votes
1answer
266 views

Need dataset of english letters

I have done one assignment of classifying digits using Back propagation Neural Network. Now I want to test it on english letters. Can anybody tell me from where can I get the dataset of English ...
3
votes
1answer
927 views

Where can I find open etymology databases?

Are there any open and directly accessible, freely available, English etymology dictionary databases like the one used by online etymology dictionary? It could be accessible either directly via ...
3
votes
0answers
143 views

Database of Diacritics

I'm working on a project to map Unicode English characters to diacritic characters in Javascript. When the user enters e.g. "doner" the search engine should also find e.g. "döner". So it should be ...
3
votes
0answers
56 views

American English pronunciation dictionary in text form showing syllables

Looking for pronunciation and syllabification data for American English. I want to analyze pronunciation and spelling rules/heuristics to see how accurate they are and identify exceptions. The goal ...
2
votes
3answers
1k views

Where can I find a list of latin words?

I'm currently searching for a big list of machine readable latin words. I need it in a (.txt) file and preferably no definitions with the words, and each word be on a different line. For example: ...
2
votes
3answers
111 views

Single-word Translations in All Languages

I am looking for a dataset of single-word translations, essentially a translation dictionary for all languages. I don't need all words, but simply a set of very common words: "yes", "no", "true", "...
2
votes
1answer
23 views

Source of multilingual chat/online/contemporary corpus data?

I need to develop autocorrection and prediction for mobile keyboard app that will work for some 8-10 most common Latin languages + Russian (English, German, Spanish, Portugues, French, ...). For start,...
2
votes
1answer
28 views

Looking for gis data on world linguistics by location

Looking for geospatial information on linguistics. Looking for world wide data with national and subnational data. Data should include what regions speak which language(s). Does this data exist?
2
votes
2answers
94 views

English / Swedish translation data?

I'm looking for a dataset similar to the "parallel data" (i.e. one file is English and one file is Swedish) found in the WMT translation task. Their datasets include news commentary, European ...
2
votes
2answers
31 views

Common words missing from typical spellcheckers

I find the spellcheckers included with most software to be woefully inadequate. Many common words are not included in your typical spellchecker. For example, try spellchecking these common words: ...
2
votes
1answer
210 views

Context-aware dataset of movies or music

Looking for access to context aware dataset(s) of movies or music. Context aware dataset should contain user related information, such as user gender, mood, time, social, location, weather, emotions ...
2
votes
1answer
47 views

List of languages with active or native speakers

I'm looking for a simple list that has a count of speakers per language. It can be either active speaker or native speakers, the exactness doesn't matter so much to me. Any format, any license. It's ...
2
votes
1answer
42 views

Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?

I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org manually is quite cumbersome at that scale. Is there a ...
1
vote
1answer
1k views

Dataset of common typos in English

I am looking for a dataset of common typos in English. Ideally, it should mention the frequency of typo.
1
vote
2answers
57 views

Dataset where words are associated to colors

I am looking for a dataset mapping concrete words with common colors. The only one i've found is this, but it has less than 100 entries. EDIT: example: red_words_list = [ "apple", "blood", "rose", "...
1
vote
1answer
79 views

Find geographic data on the languages spoken in Russia

Does anyone have any ideas for where I can get data on the number or percentage of speakers of a given language in the Russian Federation at some geographic level smaller than federal subject? ...
1
vote
1answer
38 views

Should the testing of my learning algorithm be restricted only on standard datasets or can I use any dataset to publish my results?

I've trained a speech recognition model using the LibriSpeech database which isn't as widely used as other datasets like TIMIT or MNIST. Does that factor in any way or can I publish the results ...
1
vote
1answer
80 views

List of vocabulary (kanjis) for the JLPT1 exam

I am preparing for the Japanese Language Proficiency Test N1. I am looking for a list of vocabulary (kanjis) that I should probably study for this test. I know there is no official list, but is ...
1
vote
0answers
22 views

Paragraph Similar Pairs Dataset

I have been working on a project where I need to find whether two paragraphs convey the similar meaning. Same as Quora Question Pairs on Kaggle, but instead of questions, paragraphs. But I could not ...
1
vote
0answers
63 views

Is there a large database of unique surnames and their (origin) language?

Let me explain the phrases I use in my request: large: I'm looking for multiple languages (especially european languages if this constrain helps) and at least 2K surnames per language. But the more ...
1
vote
0answers
20 views

Dataset for English word trigrams

Does anyone know where to find such a dataset? There is Google's Ngram viewer but it is too big to be useful. I'm looking for something smaller. Perhaps with only the 100k most common English words ...
1
vote
0answers
17 views

Estonian named entity recognition data

I'm looking for estonian named entity recognition data. Till now I am unable to find one. Although Estonia has 90% of it's Govt services online, I can't find their NER data anywhere. I tried estnltk, ...
1
vote
0answers
19 views

Historical data of languages spoken in country by percent population?

I am looking for a dataset that contains the languages spoken in each country and their respective percentages of the population, going back as far back into the historical record as possible. I ...
1
vote
0answers
55 views

Where is a large list of declarative and imperative sentences

For a project I need a bunch (+500 each) grammatically correct sentences (English) that are declarative (basic or simple), imperative (command), and interrogative (question) (sorted by group or ...
1
vote
0answers
21 views

What are methods to create open text data from copyrighted texts?

Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
1
vote
0answers
27 views

Hierarchical semantic distances dataset

I need a data (preferably human judgment) for the structural/hierarchical semantic distance between many couples (at least hundreds) of word. For example, d(computer, television) < d(radio, ...
1
vote
0answers
54 views

Looking for open dataset mapping informal company names to ticker

We've looked for trained data for named entity recognition (in particular for company names). The end goal is to feed in a list of entity names (with potential misspellings, acronyms, punctuation etc) ...
1
vote
0answers
166 views

Labeled dataset for sentiment analysis

I am doing my research on sentiment analysis for Standard Mandarin. I don't have any benchmark dataset. I need usable labeled dataset for my research.
1
vote
0answers
65 views

Mailing list datasets

Related to this question: Obtaining personal mail corpus I'm looking for a list of archived public mailing lists from diverse communities and in diverse languages. For example, a list would look ...
1
vote
0answers
27 views

Tourism/survival sentences in most languages

I am looking for the first 50 or 100 sentences/words that an English-only-speaking tourist would learn when staying in a non-English-speaking country for a few weeks. For instance, the first few ...
0
votes
3answers
515 views

French word list?

I would like to make an auto-completion tool for people who are don't type very well, like what exists on phones. So I do not need just a dictionary but a list of all words, like all conjugations of ...
0
votes
1answer
94 views

Database of Japanese words difficulty

I have a huge database of Japanese words and want to split them into flashcard decks of different difficulty levels. (actually for a GSoC project) For this, I need to know the "difficulty" of each ...
0
votes
2answers
37 views

Online Dictionary for Scraping

Does anybody know if there is an extensive online dictionary other than Wiktionary that lists many words on a single page, as opposed to having to search for a single word? None of the online ...
0
votes
1answer
16 views

Open dictionary for English with International Phonetic Alphabet for every word

I'm seeking a dictionary that contains the IPA transcription for every word. Is such a resource available for English?
0
votes
1answer
191 views

Torrent for the fastText pre-trained models?

I would like to download the pre-trained models for ~100 languages, which are ~3GB each. (See https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md) Is there a utility or ....
0
votes
0answers
25 views

Datasets for emotion recognition in German dialogues

I am currently working on the topic of emotion recognition in German dialogues (Speech). So far I only managed to find a single dataset related to this topic: Berlin Database of Emotional Speech. Is ...
0
votes
0answers
10 views

How to obtain multiple IGE datasheets simultaneously?

I am undertaking a study on the use of Galician based on certain sociodemographic variables. The IGE provides a good amount of data in this respect, but they are distributed in a large number of ...
0
votes
1answer
78 views

How to check whether a word is abbreviated or not using Python? [closed]

How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation. Is there any dictionary present for this?
0
votes
1answer
62 views

Looking for word frequency data for the Google Books ngrams

I have downloaded all the single words from Google Books ngram data (http://storage.googleapis.com/books/ngrams/books/datasetsv2.html) and I could write code to aggregate it all generate word ...
0
votes
0answers
100 views

Recent dump of names from facebook.com/directory?

For years all names of all Facebook users were listed at https://www.facebook.com/directory. Even without the links to the profiles themselves, it was a very useful dataset of diverse names and ...