Questions tagged [language]
For questions related to human language. Use 'programming' instead for questions related to programming languages.
117
questions
1
vote
1
answer
31
views
Dataset for human-level artificial intelligence
I think a human-centric multi-modal dataset (i.e.: head mounted binocular cameras, steroes mics, text transciption, skeletal pose, skin area contact) would be a great starting point for building human-...
- 111
0
votes
1
answer
24
views
Extemporaneous speech, as text, tagged by speaker?
tl;dr; I'm looking for a corpus (or multiple of them) of un-scripted English speech, transcribed to text and tagged by who said what. Ideally, I'd like hundreds or even thousands of distinct speakers.
...
- 101
1
vote
2
answers
84
views
Russian Dictionary that contains stress marks
I am looking to automate the process of setting the stress of Russian words. There are already tools that do this (RussianGram or https://morpher.ru/accentizer/), but nothing open source which can be ...
- 131
1
vote
0
answers
32
views
Dataset with French words by CEFR level
As a German speaker learning French, I really enjoyed the "Fremdsprachentexte Reclam" books: they are French literature books for German learners, each book has a CEFR level (e.g. B2, C1...),...
- 11
1
vote
0
answers
38
views
How many *Non-Native* speakers of Egyptian Arabic are there world wide?
Every time I try to look this up, it always gives me the number for native speakers. I'm not looking for that. I specifically want to know how prevalent Egyptian Arabic is as a second language around ...
- 11
2
votes
1
answer
45
views
Where do I find a database of each city in the world and the languages spoken there? (Ex. Barcelona - Catalan, Spanish vs. Madrid - Spanish)
I'm looking to set up a dropdown of languages once the app picks up the city a person is in. Where would I find this database or API?
1
vote
0
answers
25
views
Label language, how to change to local names
I'm viewing Wikidata with Swedish set as my user interface language. This turns all the language names in the list of labels into Swedish, which makes the list hard to follow in my opinion since the ...
- 11
0
votes
1
answer
14
views
Language demographics of Swiss citizens
I am looking for demographics of languages spoken or not-spoken by Swiss citizens or residents.
Although I don't think it's possible to find this exact dataset, this is the core question: how many ...
- 17.2k
2
votes
1
answer
42
views
Is there a Simplified Chinese thesaurus file, like Moby for English?
I'm looking for a manually curated thesaurus/synonyms file for Simplified Chinese. Like the English Moby thesaurus, but in simplified Chinese. Does such a thing exist?
I'm aware that WordNet can be ...
- 121
2
votes
2
answers
54
views
Linguistics. German verbs
One might think that this question should be on the StackExchange German site, but they appear not to like requests for lists and such like. Hence this post ...
I'm looking for large electronic list ...
- 195
1
vote
0
answers
17
views
Peer to peer communication dataset for research
I am looking for a peer-to-peer communication dataset, chat data, emails, phone calls, etc... any kind of conversation text between pairs of people for research. It would be much useful if the genders(...
- 41
0
votes
2
answers
122
views
German nouns gender source
I want to create some sort of spell checking program for German language, for which i need a database with all the words and their gender description(only their gender description) to cross reference ...
- 9
1
vote
1
answer
37
views
Dataset of language families, sub-families and their relations other than Wikidata
I am interested in language families such as Indoeuropean, Romance languages, Afroasiatic and so on, not individual families (although if the dataset includes languages it doesn't hurt).
Wikidata and ...
- 125
1
vote
0
answers
14
views
Grammar for mathematics
I was wondering if there exits an grammar defining mathematical language. By that I mean the list of the tokens used and the syntactic rules.
- 11
1
vote
0
answers
16
views
Library of ancient Linear A and Linear B text images?
Is there an open-data library of text\language images for ancient Linear A and Linear B languages?
- 357
2
votes
2
answers
151
views
CEFR for English words by level
Are there any open source data for English words by CEFR level?
I found https://www.oxfordlearnersdictionaries.com/wordlists/oxford3000-5000 , which categorizes English words based on CEFR level, ...
- 71
0
votes
1
answer
81
views
English to Telugu and Telugu to English dictionary database
Data: Database of Telugu to English and vice versa language dictionary
Context: Need to make a Telugu to English and vice versa dictionary app.There should be quite some public domain dictionaries ...
1
vote
1
answer
44
views
Known datasets for long document analysis
Are there good baselines datasets or benchmarks for similarities/retrieval of long texts (extremely long documents, books, etc.)? Although significant both academically and applicational, I cannot get ...
- 111
1
vote
0
answers
22
views
Dictionary of European Languages
I'm looking for a database that contains translations of words in different European languages (mainly Romance). The most important languages for my project are in order Latin, English, Spanish and ...
- 111
1
vote
0
answers
22
views
translation dictionaries for european languages
I'm looking for open translation dictionaries of european languages. Most important would be english-french and english-spanish.
This question is a direct analogon to:
Translation dictionaries. The ...
- 133
1
vote
2
answers
51
views
Where can I get word lists for Indian languages, specifically Hindi and Tamil?
Is there a way to get word lists for Indian languages (specifically Tamil and Hindi), even for commercial use? It is for a game I am developing
- 33
0
votes
0
answers
114
views
Datasets for emotion recognition in German dialogues
I am currently working on the topic of emotion recognition in German dialogues (Speech). So far I only managed to find a single dataset related to this topic: Berlin Database of Emotional Speech.
Is ...
- 584
1
vote
0
answers
42
views
Paragraph Similar Pairs Dataset
I have been working on a project where I need to find whether two paragraphs convey the similar meaning. Same as Quora Question Pairs on Kaggle, but instead of questions, paragraphs. But I could not ...
1
vote
0
answers
108
views
Is there a large database of unique surnames and their (origin) language?
Let me explain the phrases I use in my request:
large: I'm looking for multiple languages (especially european languages if this constrain helps) and at least 2K surnames per language. But the more ...
- 111
0
votes
2
answers
179
views
Online Dictionary for Scraping
Does anybody know if there is an extensive online dictionary other than Wiktionary that lists many words on a single page, as opposed to having to search for a single word?
None of the online ...
- 121
1
vote
0
answers
27
views
Dataset for English word trigrams
Does anyone know where to find such a dataset? There is Google's Ngram viewer but it is too big to be useful. I'm looking for something smaller. Perhaps with only the 100k most common English words ...
- 111
2
votes
1
answer
43
views
Source of multilingual chat/online/contemporary corpus data?
I need to develop autocorrection and prediction for mobile keyboard app that will work for some 8-10 most common Latin languages + Russian (English, German, Spanish, Portugues, French, ...). For start,...
- 121
1
vote
0
answers
21
views
Estonian named entity recognition data
I'm looking for estonian named entity recognition data. Till now I am unable to find one. Although Estonia has 90% of it's Govt services online, I can't find their NER data anywhere. I tried estnltk, ...
1
vote
0
answers
43
views
Historical data of languages spoken in country by percent population?
I am looking for a dataset that contains the languages spoken in each country and their respective percentages of the population, going back as far back into the historical record as possible.
I ...
1
vote
1
answer
1k
views
How to check whether a word is abbreviated or not using Python? [closed]
How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation.
Is there any dictionary present for this?
SUHASINI KL
2
votes
1
answer
33
views
Looking for gis data on world linguistics by location
Looking for geospatial information on linguistics.
Looking for world wide data with national and subnational data. Data should include what regions speak which language(s).
Does this data exist?
- 51
1
vote
1
answer
330
views
Looking for word frequency data for the Google Books ngrams
I have downloaded all the single words from Google Books ngram data (http://storage.googleapis.com/books/ngrams/books/datasetsv2.html) and I could write code to aggregate it all generate word ...
- 11
1
vote
0
answers
168
views
Where is a large list of declarative and imperative sentences
For a project I need a bunch (+500 each) grammatically correct sentences (English) that are declarative (basic or simple), imperative (command), and interrogative (question) (sorted by group or ...
- 111
0
votes
0
answers
167
views
Recent dump of names from facebook.com/directory?
For years all names of all Facebook users were listed at
https://www.facebook.com/directory.
Even without the links to the profiles themselves, it was a very useful dataset of diverse names and ...
1
vote
0
answers
22
views
What are methods to create open text data from copyrighted texts?
Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
user6083
0
votes
1
answer
31
views
Open dictionary for English with International Phonetic Alphabet for every word
I'm seeking a dictionary that contains the IPA transcription for every word. Is such a resource available for English?
- 649
2
votes
1
answer
84
views
List of languages with active or native speakers
I'm looking for a simple list that has a count of speakers per language. It can be either active speaker or native speakers, the exactness doesn't matter so much to me.
Any format, any license. It's ...
- 17.2k
5
votes
1
answer
816
views
Audio data set of (telephone) conversations and their transcriptions
I need records of telephone conversations (or simple conversations between two persons) with their transcripts. I need English and French datasets.
- 51
1
vote
1
answer
96
views
Find geographic data on the languages spoken in Russia
Does anyone have any ideas for where I can get data on the number or percentage of speakers of a given language in the Russian Federation at some geographic level smaller than federal subject?
...
- 11
1
vote
2
answers
85
views
Dataset where words are associated to colors
I am looking for a dataset mapping concrete words with common colors.
The only one i've found is this, but it has less than 100 entries.
EDIT: example:
red_words_list = [ "apple", "blood", "rose", "...
- 111
3
votes
2
answers
430
views
Swedish Noun Gender List
I'm looking for a lexical resource on Swedish noun genders.
In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
- 251
1
vote
0
answers
27
views
Hierarchical semantic distances dataset
I need a data (preferably human judgment) for the structural/hierarchical semantic distance between many couples (at least hundreds) of word.
For example, d(computer, television) < d(radio, ...
- 111
1
vote
0
answers
77
views
Looking for open dataset mapping informal company names to ticker
We've looked for trained data for named entity recognition (in particular for company names). The end goal is to feed in a list of entity names (with potential misspellings, acronyms, punctuation etc) ...
- 11
0
votes
1
answer
206
views
Torrent for the fastText pre-trained models?
I would like to download the pre-trained models for ~100 languages, which are ~3GB each.
(See https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md)
Is there a utility or ....
2
votes
1
answer
217
views
Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?
I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org manually is quite cumbersome at that scale.
Is there a ...
3
votes
2
answers
77
views
Corpus of words divided by language
I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language.
I don't really care for translations, ...
- 133
2
votes
2
answers
174
views
English / Swedish translation data?
I'm looking for a dataset similar to the "parallel data" (i.e. one file is English and one file is Swedish) found in the WMT translation task. Their datasets include news commentary, European ...
- 123
0
votes
3
answers
2k
views
French word list?
I would like to make an auto-completion tool for people who are don't type very well, like what exists on phones.
So I do not need just a dictionary but a list of all words, like all conjugations of ...
- 210
4
votes
1
answer
859
views
Dataset of Sentences Classified by Type?
Where can I find a labeled dataset of sentences classified by type (interrogative and declarative)? I'm particularly lookings for Russian sentences, but other open and labeled datasets of other ...
- 173
2
votes
3
answers
3k
views
Where can I find a list of latin words?
I'm currently searching for a big list of machine readable latin words. I need it in a (.txt) file and preferably no definitions with the words, and each word be on a different line. For example:
...
- 37