Skip to main content
14 votes

Downloading the Panama Papers

No leak The Panama Papers were not leaked at all. They were handed over to a German media company which probably paid a good price (translation/summary here) for that deal. This means, the panama ...
q9f's user avatar
  • 290
10 votes
Accepted

Downloading the Panama Papers

The Panama Papers database has just been published at this link https://www.occrp.org/en/panamapapers/database.html This ICIJ database contains information on almost 320,000 offshore entities that ...
HDunn's user avatar
  • 332
9 votes

Paraphrase data sets

You definitely have the big ones with MSRP and PPDB. A few others, grouped by maintainers. CLiC at the Universitat de Barcelona Maintain several highly annotated corpora. These are available from ...
Frames Catherine White's user avatar
5 votes

Twitter open datasets

Check out these links below, and you can also search "twitter" in data.world for other related data. There's quite a few users who are already aggregating this data and creating analyses and cool ...
data.world's user avatar
4 votes

Paraphrase data sets

Google recently released paraphrase dataset: Paraphrase Adversaries from Word Scrambling (PAWS). https://github.com/google-research-datasets/paws This dataset is superior to many paraphrase datasets....
Proton Boss's user avatar
4 votes

Dataset of common typos in English

Here is a link to several corpora of commonly misspelled words in several varieties of English - list includes the Birbeck Spelling Error corpus
Brandon Loudermilk's user avatar
4 votes
Accepted

Looking for text-to-phoneme or pronunciation data for English words

#1 - For phonological equivalents of English words check out the Irvine Phonotactic Online Dictionary (IPhOD) which is a collection of English words and non-words primarily intended for work in ...
Brandon Loudermilk's user avatar
4 votes

Semantic dictionary of English language

Eurovoc is a large semantic "Multilingual Thesaurus of the European Union"
MickyInTheSky's user avatar
4 votes
Accepted

Where can I find a dataset containing legal documents?

CFPB Credit Card Agreements DB I think that is a service contract. docracy - open source legal contracts Requires sign up
albert's user avatar
  • 11.9k
3 votes

Dataset listing datasets for natural language processing

Such a list requires someone to maintain it. This one lists corpora and adds some info, covering some if not all of your criteria (in natural language): Corpora, Collections, Data Archives
Tom Anderson's user avatar
3 votes

Sentence segmentation dataset

The Pragmatic Segmenter segmentation library contains sentences for the common edge cases in English as well as some other languages. The README also includes links to a lot of research and other ...
diasks2's user avatar
  • 131
3 votes

Downloading the Panama Papers

Still waiting for the entire cache, but The Sunday Times has released a list with 37,000 names (10 April 2016). Search our database of 37,000 names linked to Mossack Fonseca companies in the tax ...
philshem's user avatar
  • 17.6k
3 votes

Downloading the Panama Papers

It seems the media is not releasing it to the public yet. There are some links floating around to a similar (but smaller) 2013 leak, but that is not the one that was released today. http://www....
mgalka's user avatar
  • 139
3 votes

Most common words in English

There are word frequency lists for a number of languages, including English, that were created by Hermit Dave from the data at opensubtitles.org. You can download them for free on his blog. The ...
Suzana's user avatar
  • 406
3 votes

Datasets: Is there any public data-set for FAQ (Frequently asked questions) in different domains?

What about StackOverflow's own dataset? https://archive.org/details/stackexchange
robinvdvleuten's user avatar
3 votes

Paraphrase data sets

You can also check out the Quora Question Pairs Dataset which was used in the recent Kaggle competetition.
Amir's user avatar
  • 173
3 votes

ASCII Character Frequency Analysis

I wrote some code to analyse the reuters21578 corpus (thanks @philshem for pointing me to it)! You can find the results here - ascii_freq.txt or ascii_freq.json. The code used to generate them can be ...
PiersyP's user avatar
  • 131
3 votes

German news/text data set

Event registry provides access to news articles from more than 100.000 news sources in 15 languages including German. The only problem is that you can only get access to the last 3 years of articles. ...
Gregor Leban's user avatar
3 votes

Data sets for short text classification

The following repository contains 20000 questions from StackOverflow. https://github.com/jacoxu/StackOverflow . The data is provided by Kaggle. There are true labels as well. So you can evaluate the ...
user15259's user avatar
3 votes

Tool to extract the main concepts/topics from web pages

Thanks Tasos for your very helpful DBpedia Spotlight Python code sample above. However, converting it from Python 2 to Python 3+ wasn't entirely straightforward so thought I'd post the revised version....
neilb's user avatar
  • 31
3 votes

Twitter open datasets

Here is a dataset of relative word frequency for the top 10,000 words in 890 million Tweets, divided by county: https://sites.google.com/site/wordmapperinfo/
wirefire's user avatar
  • 351
3 votes
Accepted

word2vec Analogy Task Data Set

The Syntactic task set can be downloaded from https://code.google.com/archive/p/word2vec/source/default/source FYI What percentage of links posted in published articles are dead?
Franck Dernoncourt's user avatar
3 votes

Natural language corpora dataset with time stamps

The Linguistic Data Consortium (LDC) has an online searchable database of English news articles (also Chinese & Arabic). If you sign up for an account, you should be able to access the LDC Online ...
Brandon Loudermilk's user avatar
3 votes
Accepted

Where can I find data set of tweets of users?

It's against Twitter's policy to provide datasets of more than user and tweet IDs. However, as lots of people are tweeting every day, you can construct your own dataset using the twitter api: https://...
Swier's user avatar
  • 1,453
3 votes

Dataset of documents and user libraries

While you may not have access to the Mendeley DataTEL dataset, have you considered other open datasets? Netflix has an open API that provides a very similar set of information that I think might fit ...
shadow_dev's user avatar
3 votes

Where can I find a dataset containing legal documents?

This dataset contains labeled and unlabeled legal contracts for contract element extraction. The labeled dataset POS tags as well as annotations for different contract elements. You can refer to the ...
Alok Ranjan's user avatar
3 votes

Where can I find machine readable transcribed text of the 2016 Presidential speeches and debates?

The GitHub repo 2016-campaign contains campaign speeches given by Hillary Clinton and Donald Trump during the 2016 presidential campaign. [and] the text of the three presidential debates, separated ...
philshem's user avatar
  • 17.6k
3 votes
Accepted

Reddit comments labeled data-set for sentiment analysis?

Reddit Comment Dataset Including Sentiment Data
albert's user avatar
  • 11.9k
3 votes

Is there any open data-set for sentence segmentation?

Some Corpora in the NLTK have a method sents() that returns each sentence as a list of words, for example the Brown corpus: import nltk nltk.download() # this opens a GUI to download all corpora ...
Suzana's user avatar
  • 406
3 votes

Is there any open data-set for sentence segmentation?

The British National Corpus, "a 100 million word collection of samples of written and spoken language from a wide range of sources", is free of charge and segmented, but i never used it.
Ettore Rizza's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible