Questions tagged [language]
For questions related to human language. Use 'programming' instead for questions related to programming languages.
0
votes
1answer
10 views
Looking for word frequency data for the Google Books ngrams
I have downloaded all the single words from Google Books ngram data (http://storage.googleapis.com/books/ngrams/books/datasetsv2.html) and I could write code to aggregate it all generate word ...
1
vote
0answers
19 views
Where is a large list of declarative and imperative sentences
For a project I need a bunch (+500 each) grammatically correct sentences (English) that are declarative (basic or simple), imperative (command), and interrogative (question) (sorted by group or ...
0
votes
0answers
83 views
Recent dump of names from facebook.com/directory?
For years all names of all Facebook users were listed at
https://www.facebook.com/directory.
Even without the links to the profiles themselves, it was a very useful dataset of diverse names and ...
1
vote
0answers
20 views
What are methods to create open text data from copyrighted texts?
Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
0
votes
1answer
12 views
Open dictionary for English with International Phonetic Alphabet for every word
I'm seeking a dictionary that contains the IPA transcription for every word. Is such a resource available for English?
3
votes
1answer
39 views
List of languages with active or native speakers
I'm looking for a simple list that has a count of speakers per language. It can be either active speaker or native speakers, the exactness doesn't matter so much to me.
Any format, any license. It's ...
2
votes
0answers
33 views
Audio data set of (telephone) conversations and their transcriptions
I need records of telephone conversations (or simple conversations between two persons) with their transcripts. I need English and French datasets.
2
votes
1answer
77 views
Find geographic data on the languages spoken in Russia
Does anyone have any ideas for where I can get data on the number or percentage of speakers of a given language in the Russian Federation at some geographic level smaller than federal subject?
...
0
votes
0answers
33 views
English Poetry Dataset categorized by meters
Is there a dataset that contains large number of English poems ?
It should be categorized by meters.
I couldn't find any resource online, i also can web scrape any site that have my needed dataset.
1
vote
2answers
52 views
Dataset where words are associated to colors
I am looking for a dataset mapping concrete words with common colors.
The only one i've found is this, but it has less than 100 entries.
EDIT: example:
red_words_list = [ "apple", "blood", "rose", "...
2
votes
2answers
150 views
Swedish Noun Gender List
I'm looking for a lexical resource on Swedish noun genders.
In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
1
vote
0answers
25 views
Hierarchical semantic distances dataset
I need a data (preferably human judgment) for the structural/hierarchical semantic distance between many couples (at least hundreds) of word.
For example, d(computer, television) < d(radio, ...
2
votes
0answers
45 views
Looking for open dataset mapping informal company names to ticker
We've looked for trained data for named entity recognition (in particular for company names). The end goal is to feed in a list of entity names (with potential misspellings, acronyms, punctuation etc) ...
0
votes
1answer
149 views
Torrent for the fastText pre-trained models?
I would like to download the pre-trained models for ~100 languages, which are ~3GB each.
(See https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md)
Is there a utility or ....
2
votes
1answer
36 views
Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?
I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org manually is quite cumbersome at that scale.
Is there a ...
3
votes
2answers
58 views
Corpus of words divided by language
I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language.
I don't really care for translations, ...
2
votes
2answers
63 views
English / Swedish translation data?
I'm looking for a dataset similar to the "parallel data" (i.e. one file is English and one file is Swedish) found in the WMT translation task. Their datasets include news commentary, European ...
0
votes
3answers
111 views
French word list?
I would like to make an auto-completion tool for people who are don't type very well, like what exists on phones.
So I do not need just a dictionary but a list of all words, like all conjugations of ...
6
votes
1answer
295 views
Dataset of Sentences Classified by Type?
Where can I find a labeled dataset of sentences classified by type (interrogative and declarative)? I'm particularly lookings for Russian sentences, but other open and labeled datasets of other ...
2
votes
3answers
650 views
Where can I find a list of latin words?
I'm currently searching for a big list of machine readable latin words. I need it in a (.txt) file and preferably no definitions with the words, and each word be on a different line. For example:
...
1
vote
0answers
144 views
Labeled dataset for sentiment analysis
I am doing my research on sentiment analysis for Standard Mandarin. I don't have any benchmark dataset. I need usable labeled dataset for my research.
4
votes
1answer
34 views
Where can I find subtitled recordings of university courses?
I'm doing a research on the language used by university professors and I'm looking for closed captioned videos from real university classes.
4
votes
1answer
39 views
I am looking for good german <-> czech dictionary in text format(or something else)
I am looking for good german <-> czech dictionary in text format (or something I can easy read).
The Idea behind is I would like to program a program for myself for learning czech, but the sources ...
5
votes
1answer
174 views
Wikidata label language: How to fallback to ANY language?
In Wikidata, this shows labels in English, and falls back to Russian if no English is available:
SERVICE wikibase:label {
bd:serviceParam wikibase:language "en, ru" .
}
QUESTION: How to fall back ...
5
votes
1answer
293 views
Audio Datasets Featuring Different Speakers Saying the Same Sentence (English)?
Seeking a dataset of audio files consisting of recordings featuring different people saying the same sentence in English for deep neural network (DNN) training and testing.
4
votes
1answer
814 views
Dataset of English Poetry
Is there a dataset containing a group of poems in English? I couldn't find any resource online. I could web scrape poetry sites, but I want to avoid copyright issues and even a dataset of older poems ...
4
votes
0answers
129 views
Database of Diacritics
I'm working on a project to map Unicode English characters to diacritic characters in Javascript. When the user enters e.g. "doner" the search engine should also find e.g. "döner". So it should be ...
3
votes
1answer
164 views
Need dataset of english letters
I have done one assignment of classifying digits using Back propagation Neural Network. Now I want to test it on english letters.
Can anybody tell me from where can I get the dataset of English ...
3
votes
1answer
53 views
Is there any picture corpus?
By "picture corpus" I mean the picture collection each illustrating one word in the usually used vocabulary, such as words in Google 10000 English I found on GitHub.
For instance: vocabulary
...
1
vote
1answer
37 views
Should the testing of my learning algorithm be restricted only on standard datasets or can I use any dataset to publish my results?
I've trained a speech recognition model using the LibriSpeech database which isn't as widely used as other datasets like TIMIT or MNIST. Does that factor in any way or can I publish the results ...
5
votes
2answers
94 views
Are there translation probabilities for major language pairs?
Are there reasonable alignment probability files for major language pairs?
Specifically, given a target language and a target word/phrase, what are the translation equivalents in other major ...
1
vote
1answer
695 views
Dataset of common typos in English
I am looking for a dataset of common typos in English. Ideally, it should mention the frequency of typo.
6
votes
1answer
81 views
How to collect hand-writting data
I'm working on machine learning project that requires I collect hand-writing data of alphabets from an uncommon language (I'm 100% there is no available data out there). My question is how should I ...
6
votes
4answers
3k views
Database world cities with their names in all languages of the world
I'm looking for a database of cities in the world with their names in all languages of the world. Where it is possible to download?
4
votes
1answer
2k views
ASCII Character Frequency Analysis
I've been looking for a table of the frequencies of all 128 ASCII characters, not just letters. The best table I've been able to find is this one, but that one only includes printable ASCII characters,...
4
votes
2answers
83 views
Keyboard keystrokes with timing
Similar to this question: Where to find key log data for keyboard usage?
...but it doesn't have the answers I am looking for.
I am researching keyboard layouts. I want to get timing data for each ...
5
votes
2answers
567 views
German news/text data set
I am looking for (possible buy) a dataset of German news data (i.e., the largest daily newspapers), where the data span at least ten years back up until now/recent.
Any tips?
7
votes
1answer
115 views
Corpus of human-scored machine translations?
Most of the parallel corpora (Opus, EuroParl, OpenSubtitles) have only human translations.
src_txt | human_trans_of_src_txt_into_trg_lng
(We assume all the translations are good.)
Is there a corpus ...
3
votes
0answers
48 views
American English pronunciation dictionary in text form showing syllables
Looking for pronunciation and syllabification data for American English. I want to analyze pronunciation and spelling rules/heuristics to see how accurate they are and identify exceptions. The goal ...
2
votes
2answers
30 views
Common words missing from typical spellcheckers
I find the spellcheckers included with most software to be woefully inadequate.
Many common words are not included in your typical spellchecker.
For example, try spellchecking these common words:
...
10
votes
6answers
8k views
Speech audio files dataset with language labels
I am working on building a language classifier in speech/audio samples. I have been trying to find a dataset which may have considerable number of speech samples in various languages. The audio files ...
4
votes
0answers
99 views
List of inverse and polarity flip words required
I'm a computer science student and I'm doing a research on sentimental analysis with a large database. Therefore, i need to create dictionaries to mapping words.
Now i'm facing the problem of "...
6
votes
1answer
113 views
How can I get a list of synonyms that work across all contexts?
Many synonyms are context specific. For example "force" is a synonym for "drive" in the context of urging or inspiring, but not in the context of journeying by vehicle.
Other synonyms seem to work ...
2
votes
2answers
99 views
Single-word Translations in All Languages
I am looking for a dataset of single-word translations, essentially a translation dictionary for all languages. I don't need all words, but simply a set of very common words: "yes", "no", "true", "...
5
votes
2answers
352 views
Text data sources in German, primarily news articles
Can anyone recommend good sources for getting archived news texts in German language?
Good source would satisfy the following conditions:
recent in date (for example 2005-2015)
variety of topics (...
4
votes
3answers
210 views
All Demonym's in their native language
COUNTRY - ? / ?
US - American / Americans
UK - English / English
DE - Deutscher / Deutsche
Anyone an idea where to find this? We're trying to find any sort of source for this for weeks now.
4
votes
3answers
2k views
Dataset for english words of dictionary for a NLP project
I am working on a NLP project, and for that I need a dataset of English words (words typically found in dictionary). Could somebody please guide me how to put together such dataset?
Is there any ...
5
votes
1answer
112 views
Worldwide holidays, and their names in the local dominant language
Might be slightly OT here but I'll give it a try since the topic is probably well known to many here:
We're looking for a database or system to get current and upcoming holidays by location, ...
4
votes
2answers
59 views
Source of aggressively toned texts for machine learning
I was looking for a source where there would be an extended amount of texts with aggressive tone. It can be intertwined with non-aggressive texts, but it would be good if the ratio of aggressive texts ...
3
votes
2answers
1k views
Offensive words - English and Spanish
Have a system generating four letter code and in testing just got up to AA-BJ ... and the BJ got me thinking.
In a while the one after AR-SD could cause offence in the UK!
Does anyone know of a list ...
