13

Many questions are posted here in search of a specific dataset that is either no longer online, or has died from linkrot, or a combination of the two. What is the best way to get around this? Or rather, can these datasets be recovered?

Improve this question
1

1 Answer 1

Reset to default
9

The Memento Web and the Wayback Machine are two possible solutions:

Wayback Machine

The Wayback Machine by the Internet Archive is your best friend for all things that were once online, and even some things that still are, if you want to compare changes.

From Wikipedia:

The Wayback Machine is a digital archive of the World Wide Web and other information on the Internet created by the Internet Archive, a non-profit organization, based in San Francisco, California. It was set up by Brewster Kahle and Bruce Gilliat, and is maintained with content from Alexa Internet. The service enables users to see archived versions of web pages across time, which the Archive calls a "three dimensional index."

If you have a dead link, simply paste it into the search input element of Wayback Machine.

Alternatively, you can type https://web.archive.org/web/*/DEAD_LINK'S_URL directly into the address bar of your favorite browser.

Memento Web

Los Alamos National Lab has been offering the Memento Web project which unifies search across the archives.

Other

Some search engines may provide you with a cached version of the page where the dataset resided (possibly, the dataset files as well).

If the steps above fail, you may consider

  1. looking for papers which used the dataset (at arXiv or in academic journals),
  2. contacting their authors,
  3. looking at various bulletin boards for those who might have downloaded the dataset and contacting them.

You can also have a look at http://academictorrents.com to see if your dataset is present there.

Improve this answer
1
  • In addition to this, a number of countries have implemented "digital legal deposit" and introduced national web archives. In the UK, for example, there is the UK Web Archive, which allows free access to archived web pages over the internet where licensing allows, and full access to the restricted parts of the archive from the British Library reading room in London and a handful of other access points around the UK.
    – Jez
    Commented Mar 19, 2015 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.