20

IMDb offers a great deal of useful structured information for research.

There're multiple ways to get small pieces of its database:

But these ways prevent from reaching a deeper study of relations within the DB, for instance, for economic research. Say, random sampling of a fraction of the DB may miss important relations.

Is there a better way to get mass IMDb data for research purposes?

  • 2
    What relations do you want to investigate, but can't? – fgregg Sep 3 '13 at 19:42
  • 3
    Are you sure the data in "alternative interfaces" is a subset in the sense you're interpreting? It may just be a subset of the kind of data they have, but, for example, their movie list from there may be everything they have. (I suspect it is complete. For example, their movie list I downloaded now has 2,780,337 titles, which is consistent with their stats: imdb.com/stats) – BurntSushi5 Jan 23 '14 at 0:10
  • I found this copy being referred to in research papers, but do not know the source. – ingomueller.net Dec 6 '18 at 18:03
11

Not sure if this would classify as a comment or an answer, but it's useful information nonethelss:

So in reading this question I HAVE to point this out - ever heard of the paper?:

Arvind Narayanan and Vitaly Shmatikov. "Robust De-anonymization of Large Datasets (How to Break Anonymity of the Netflix Prize Dataset)". The University of Texas at Austin February 5, 2008.

Full text is at: http://arxiv.org/pdf/cs/0610105v2.pdf

It's quite a famous paper and was even on the news when it got published.

Here's the abstract:

We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary’s background knowledge. We apply our de-anonymization methodology to the Netflix Prize dataset, which contains anonymous movie ratings of 500,000 subscribers of Netflix, the world’s largest online movie rental service. We demonstrate that an adversary who knows only a little bit about an individual subscriber can easily identify this subscriber’s record in the dataset. Using the >>>Internet Movie Database<<< as the source of background knowledge, we successfully identified the Netflix records of known users, uncovering their apparent political preferences and other potentially sensitive information.

I looked at their citations for clues but they only thing they cite verbatim is:

IMDb. The Internet Movie Database. http://www.imdb.com/, 2007.

This is also quite a while ago too. However going through the full text of that article, you may be able to glean some clues as to how they got their data and replicate those - so this could potentially help you.

  • 3
    +1 I think it's a useful article in many respects. Saw it some time ago. But the article's authors had to work "with a very small sample of a few dozen IMDb users" due to IMDb's limitations on crawling. PS: A couple ideas. (1) The Netflix dataset may be an alternative (supplement) to IMDb itself. (2) It's a potentially interesting method of revealing preferences and making sociological surveys. PPS: Partly relevant work about anonymous dataset and identification from The Privacy Bounds of Human Mobility at Media Lab. – Anton Tarasenko Sep 5 '13 at 10:04
1

As was alluded to in one in a comments you should check out http://www.imdb.com/interfaces which would allow you to either download manually via ftp or through a terminal interface. I am pretty sure the restrictions on the data only pertain to commercial usages but you should verify that before diving in head first. Also, you could check out http://www.themoviedb.org or http://www.omdbapi.com. themoviedb API is free but has api request limits, and it does not natively include the IMDBid in a typical search which can make integrating data from multiple sources difficult. OMDb API used to be totally free but now restricts api access though the cost for 100,000 requests is 1$/month so it is very reasonable and it does include the IMDBid in its general search.

UPDATE:

So as @lonstar, likely in order to save on costs, imdb now requires that users foot the bill for downloading by using a S3 Pay Account. However, there is a mirror site that is still operational hosted by Freie Universitat Berlin, ftp://ftp.fu-berlin.de/pub/misc/movies/database/temporaryaccess/ .

Although, given that the name of the ftp parent directory is "temporaryaccess" it may not be long for this world. :(

  • looks like no more FTP - S3 requester-pays now. Also, it seems to be just titles available, not plots, stars, etc. See imdb.com/interfaces. – lonstar Jul 5 '17 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.