8

I'm looking for cases where data (including software) that once was open is not anymore, or was not for periods of several months or more. I can imagine several scenarios for this, including

  1. Technical problems (e.g. server failures, bandwidth issues)
  2. Legal problems (e.g. uncertainties over applicability of EU database rights)
  3. Social issues (e.g. new management at data producer, financial problems at data host)

I am interested in all of these and would appreciate pointers to concrete examples and attempted remedies. I have seen this related question that addresses the symptom of missing once-open data, but not the underlying causes.

As for link rot, I am right now only interested in cases where the data is actually not available any more, as opposed to not available on the original location, though distinguishing the two may of course be difficult.

Improve this question
3
  • Are we talking about link rot and poor maintenance? Because there's a ton of that on data.gov and most other 'data portal' sites that link to rather than archive the data.
    – Joe
    Commented May 26, 2015 at 14:22
  • 1
    Related : opendata.stackexchange.com/q/1148/263
    – Joe
    Commented May 26, 2015 at 14:29
  • While poor maintenance may be a frequent element in cases of once-open data, I am more interested in conscious decisions leading to the removal, rather than neglect. That thread about the government shutdown is indeed interesting in this regard.
    – Daniel Mietchen
    Commented May 31, 2015 at 21:27

6 Answers 6

Reset to default
4

Here's an example from the NIH in 2008. Basically, the NIH learned that in some cases a clever algorithm could identify medical patients from two sets of open medical data. This is an early case in which attempted anonymization turned out to be insufficient. http://articles.latimes.com/2008/aug/29/local/me-dna29

FYI, here's my blog post about it at the time. http://legacy.earlham.edu/~peters/fos/2008/08/nih-takes-two-oa-dna-databases-offline.html

Improve this answer
4

One you'll probably remember - Wikimedia released (CC0) search logs in 2012, realised they were insufficiently anonymised, and took them down the same day:

https://blog.wikimedia.org/2012/09/19/what-are-readers-looking-for-wikipedia-search-data-now-available/

There's probably quite a few more cases like this, where the dataset has been removed shortly after release by the maintainers, but many will be pretty much undocumented...

Improve this answer
1
3

I can think of one instance that once open data was subsequently removed from public use in future iterations. Unfortunately I don't remember the specifics, but it involved the outcome of trials for criminal cases in a Midwestern state I can't at this moment recall. Not all of it was redacted in more recent releases, but the name of the judge and any other information that would identify the id of the trial were removed.

In this state judges were elected by popular vote, but unfortunately these judges found that the information from the publicly-released data were being used as a political attack against them. Opponents would argue that certain judges were soft on crime using this information. In this specific instance, open data was removed because people were using it for malicious purposes against other individuals.

Improve this answer
1
  • Yes, "don't remember the specifics" seems to be a theme around this topic - got several of these before asking here. Judges rarely played a role in open data scenarios that I am familiar with, so thanks for this angle.
    – Daniel Mietchen
    Commented May 25, 2015 at 16:49
3

Not sure if this fits your requirement, but the Enron Email Dataset comes to mind. There are multiple locations where you can learn about the dataset, i.e. here: https://www.cs.cmu.edu/~./enron/

The initially published dataset got cleaned because some information was too sensitive.

Improve this answer
1
  • I had been vaguely aware of that case, but it hadn't come to my mind when I was pondering the question, so thanks for the pointer. I could not find any hints that the data were openly licensed, but the issues discussed around it are probably relevant to many open datasets.
    – Daniel Mietchen
    Commented May 31, 2015 at 21:33
2

Social Media APIs are prime examples here; literally most of them have done this, with the exception of flickr.

Details differ for each, but essentially these companies make API consumption easy/cheap/something else to entice developers to mash it up with something, to generate excitement about their service(s). Then at some point after they decide to monetize their platform via consumption.

On one hand, it makes sense because you can't scale with unlimited consumption....on the other hand, considering it was offered in most cases, and lots of people put hard work and time into building things to effectively make their product better, its totally a dev killer.

Twiter's whitelist is a great example, although I'll go ahead and say every one of their API changes post 2008-09 is the same. Also when they nixed JSONP is another example.

Facebook killing off users RSS feeds is another example, but again, all of their API changes can be seen as the same.

Netflix, Skype, and Google Translate are all more examples, although again, practically every social media API i can think of has done this. Most recently LinkedIn killed their Open API (no irony there!!!! :) ).

EDIT: textbook example of another one bites the dust: twitter kills politwoops data api functionality:
http://tktk.gawker.com/twitter-just-killed-politwoops-1708842376

Improve this answer
2
  • 1
    Are any of these APIs really "open", though? If memory serves they've mostly had explicit warnings that effectively say "you can use this until we decide to turn it off, no recourse"
    – Andrew is gone
    Commented Jun 8, 2015 at 18:06
  • idk anything about those. if you know of one specifically....
    – albert
    Commented Jun 8, 2015 at 18:32
1

Here's a relatively well known one:

AOL issued an apology yesterday for posting on a public Web site 20 million keyword searches conducted by hundreds of thousands of its subscribers from March to May. But the company's admission that it made a mistake did little to quell a barrage of criticism from bloggers and privacy advocates who questioned the company's security practices and said the data breach raised the risk of identity theft.

…

Source: AOL Takes Down Site With Users' Search Data, by Ellen Nakashima, Washington Post, 8 Aug 2006.

Improve this answer
3
  • 1
    This seems to have been a case where some data was public for a while, but it was not open in the sense that I am aiming at, which I have now clarified by linking the question to the Open Definition.
    – Daniel Mietchen
    Commented May 25, 2015 at 19:25
  • @DanielMietchen : the data was open -- they had attempted to anonymize it to make it available for people doing research, but it was shown that there was quite a bit of identity information that could be extracted from it. See techcrunch.com/2006/08/06/… .
    – Joe
    Commented May 26, 2015 at 14:27
  • OK, thanks for the additional pointer. Do you know what license they had used or whether they had put it into the public domain?
    – Daniel Mietchen
    Commented May 30, 2015 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.