We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [language]

For questions related to human language. Use 'programming' instead for questions related to programming languages.

Filter by
Sorted by
Tagged with
6
votes
1answer
485 views

Audio Datasets Featuring Different Speakers Saying the Same Sentence (English)?

Seeking a dataset of audio files consisting of recordings featuring different people saying the same sentence in English for deep neural network (DNN) training and testing.
4
votes
1answer
2k views

Dataset of English Poetry

Is there a dataset containing a group of poems in English? I couldn't find any resource online. I could web scrape poetry sites, but I want to avoid copyright issues and even a dataset of older poems ...
3
votes
0answers
143 views

Database of Diacritics

I'm working on a project to map Unicode English characters to diacritic characters in Javascript. When the user enters e.g. "doner" the search engine should also find e.g. "döner". So it should be ...
12
votes
3answers
3k views

Is a spoken digit dataset available?

Is a labeled dataset of spoken digits, that means of people saying "zero", "one", "two", three", "four", "five", "six", "seven", "eight", or "nine" available? I would also be interested in such a ...
3
votes
1answer
266 views

Need dataset of english letters

I have done one assignment of classifying digits using Back propagation Neural Network. Now I want to test it on english letters. Can anybody tell me from where can I get the dataset of English ...
6
votes
2answers
1k views

German news/text data set

I am looking for (possible buy) a dataset of German news data (i.e., the largest daily newspapers), where the data span at least ten years back up until now/recent. Any tips?
1
vote
1answer
38 views

Should the testing of my learning algorithm be restricted only on standard datasets or can I use any dataset to publish my results?

I've trained a speech recognition model using the LibriSpeech database which isn't as widely used as other datasets like TIMIT or MNIST. Does that factor in any way or can I publish the results ...
5
votes
2answers
100 views

Are there translation probabilities for major language pairs?

Are there reasonable alignment probability files for major language pairs? Specifically, given a target language and a target word/phrase, what are the translation equivalents in other major ...
9
votes
5answers
334 views

Data set of news articles and scientific journals

I am working on a project in NLP area and in order to test the result I need to compare it against different data sources. I already have NYTimes data source. So, I am looking for another news source ...
12
votes
7answers
1k views

List of English words

I'm writing a program that requires a list of English words, such as that used by a spell checker. I have been unable to find any such word lists whatsoever, even after trying 2 search engines.
1
vote
1answer
1k views

Dataset of common typos in English

I am looking for a dataset of common typos in English. Ideally, it should mention the frequency of typo.
6
votes
1answer
81 views

How to collect hand-writting data

I'm working on machine learning project that requires I collect hand-writing data of alphabets from an uncommon language (I'm 100% there is no available data out there). My question is how should I ...
4
votes
2answers
87 views

Keyboard keystrokes with timing

Similar to this question: Where to find key log data for keyboard usage? ...but it doesn't have the answers I am looking for. I am researching keyboard layouts. I want to get timing data for each ...
5
votes
1answer
3k views

ASCII Character Frequency Analysis

I've been looking for a table of the frequencies of all 128 ASCII characters, not just letters. The best table I've been able to find is this one, but that one only includes printable ASCII characters,...
12
votes
1answer
6k views

Large list of quotes

I am looking for a large database / list of quotes I can use. Preferably famous quotes / quotes by famous people. I have tried looking for a brainyquote API or something, but couldn't find one. If ...
3
votes
0answers
56 views

American English pronunciation dictionary in text form showing syllables

Looking for pronunciation and syllabification data for American English. I want to analyze pronunciation and spelling rules/heuristics to see how accurate they are and identify exceptions. The goal ...
2
votes
2answers
31 views

Common words missing from typical spellcheckers

I find the spellcheckers included with most software to be woefully inadequate. Many common words are not included in your typical spellchecker. For example, try spellchecking these common words: ...
4
votes
3answers
3k views

Dataset for english words of dictionary for a NLP project

I am working on a NLP project, and for that I need a dataset of English words (words typically found in dictionary). Could somebody please guide me how to put together such dataset? Is there any ...
4
votes
0answers
104 views

List of inverse and polarity flip words required

I'm a computer science student and I'm doing a research on sentimental analysis with a large database. Therefore, i need to create dictionaries to mapping words. Now i'm facing the problem of "...
6
votes
1answer
125 views

How can I get a list of synonyms that work across all contexts?

Many synonyms are context specific. For example "force" is a synonym for "drive" in the context of urging or inspiring, but not in the context of journeying by vehicle. Other synonyms seem to work ...
10
votes
1answer
800 views

Where can I find a part-of-speech corpus free for commercial use?

I'm looking to teach myself more about NLP. I started with NLTK and I see that it has potential to eventually become something I could get paid for. From there, my journey continued to reading this ...
5
votes
1answer
978 views

Longman English Dictionary Database

Is Longman English Dictionary as database available for public or should I digitalise it? I plan to create an app based on Longman English Dictionary, but I need the database somehow. Any idea how?
7
votes
2answers
306 views

French equivalent of the brown corpus

The brown corpus is a collection of text where each element is already gramatically tagged. It contains about one million words and is often used to train statistical PoS taggers. For the french ...
3
votes
2answers
2k views

Offensive words - English and Spanish

Have a system generating four letter code and in testing just got up to AA-BJ ... and the BJ got me thinking. In a while the one after AR-SD could cause offence in the UK! Does anyone know of a list ...
5
votes
1answer
122 views

Worldwide holidays, and their names in the local dominant language

Might be slightly OT here but I'll give it a try since the topic is probably well known to many here: We're looking for a database or system to get current and upcoming holidays by location, ...
4
votes
2answers
61 views

Source of aggressively toned texts for machine learning

I was looking for a source where there would be an extended amount of texts with aggressive tone. It can be intertwined with non-aggressive texts, but it would be good if the ratio of aggressive texts ...
1
vote
1answer
80 views

List of vocabulary (kanjis) for the JLPT1 exam

I am preparing for the Japanese Language Proficiency Test N1. I am looking for a list of vocabulary (kanjis) that I should probably study for this test. I know there is no official list, but is ...
5
votes
2answers
1k views

Word list for Common European Framework of Reference for Languages (CEFR)

The Common European Framework of Reference for Languages: Learning, Teaching, Assessment, abbreviated as CEFR, is a guideline used to describe achievements of learners of foreign languages across ...
4
votes
2answers
47 views

Are there opendatasets of English terms describing nationalities, ethnicities?

Say one needs to parse text for any mention of nationality or ethnicity, or otherwise use these terms in the course of an NLP project. Is there a comprehensive resource of English words describing ...
2
votes
1answer
210 views

Context-aware dataset of movies or music

Looking for access to context aware dataset(s) of movies or music. Context aware dataset should contain user related information, such as user gender, mood, time, social, location, weather, emotions ...
1
vote
0answers
27 views

Tourism/survival sentences in most languages

I am looking for the first 50 or 100 sentences/words that an English-only-speaking tourist would learn when staying in a non-English-speaking country for a few weeks. For instance, the first few ...
10
votes
3answers
4k views

Dataset of sentences translated into many languages

I'm looking for a dataset of human translated sentences. The ideal dataset would look like this: 1, en, The weather is nice today. 1, de, Das Wetter ist heute schön. 1, es, El clima es agradable hoy....
3
votes
1answer
210 views

source of character-level ngrams that include spaces

I'm looking for a source of character-level ngrams and counts that include spaces. I've found character-level ngram datasets, but they don't include spaces. Ideally, the dataset will include up to 8-...
4
votes
2answers
480 views

Dataset for Named Entity Recognition on Informal Text

I'm currently searching for labeled datasets to use to train a model to extract named entities from informal text (think something similar to tweets). Because capitalization and grammar are often ...
11
votes
1answer
3k views

Database of English words difficulty

Data request I am looking for a database that describes the "difficulty" of each English word. Difficulty of a word is not easy to define, but it is probably a mix of: Level of study needed to ...
5
votes
2answers
293 views

Do you know a technique for text mining judicial decisions?

I'm trying to find a method to mine sepecific informations in judicial decisions (judge's name, court, area of law and neutral citation number[canadian way to cite decisions uniformly across the ...
6
votes
1answer
222 views

Language Similarity Heuristic

I'm looking for a matrix of language pairs that shows some measure of their similarity. My main issue is that I don't know of heuristics that work well across different writing styles. Wikipedia's ...
5
votes
2answers
2k views

Frequency Analysis Character Distribution Data

Is there a location I can download character distributions for frequency analysis used in decryption attempt validation? I am specifically interested in ASCII value [32, 126] frequency distribution ...
3
votes
1answer
927 views

Where can I find open etymology databases?

Are there any open and directly accessible, freely available, English etymology dictionary databases like the one used by online etymology dictionary? It could be accessible either directly via ...
5
votes
0answers
723 views

Where to download the Forvo database?

Forvo.com is a website where native speakers submit pronunciation of words. The audio data is under the BY-NC-SA license. I can't find any obvious place to download the whole data. There is an API, ...
4
votes
1answer
35 views

Which licence and format should I use for the Norwegian language “data” I am creating?

While learning a foreign language, I am creating my own list of verbs (with their variants when conjugated), list of idioms, nouns (with plural form), etc. As an example, let's focus on the verbs: I ...
0
votes
1answer
94 views

Database of Japanese words difficulty

I have a huge database of Japanese words and want to split them into flashcard decks of different difficulty levels. (actually for a GSoC project) For this, I need to know the "difficulty" of each ...
13
votes
1answer
2k views

Speech audio databases with phonemes labelled

Can anyone point me to opensource databases of speech audio with phonemes labelled, .wav or the like + label files, e.g. # time, phoneme 0.31200 ao 0.41200 th 0.54200 er ... I'd prefer English, ...
9
votes
1answer
622 views

Text message corpus for American english

I am doing a research project to identify some patterns in how people interact with each other using text messages. The scope of the project is confined to American users. Can someone suggest a good ...
21
votes
2answers
235 views

Data for word hierarchies

I'm looking for a dataset which will give me is-a and has-a type relations between senses (and give senses from words). WordNet seems to be the most popular model, but I'm trying to evaluate other ...
17
votes
2answers
3k views

IPA phonology database

Is there any open database of English words and their pronunciation in IPA? What about other languages (Arabic, French,...)? (Please also point out the license of the DB) Update: I tried to convert ...
9
votes
1answer
1k views

Database with English words with grammatical classification?

Is there open database with English words with grammatical classification? I mean the list of words, with the information, which part of speech this word is (f.e. verb, noun, pronoun etc.).