Open text document corpus for information retrieval evaluation

Question

INTRODUCTION: document collections (corpora) for evaluation of information retrieval (search engine) systems are pretty often behind a paywall. A notorious example is the TREC conference (http://trec.nist.gov/). Apart from money, they ask for affiliation, making the data unavailable for the hobbyist/open-source. A few older corpora exist (http://www.daviddlewis.com/resources/testcollections/reuters21578/) but these are "antique" in ways that make then less useful for the evaluation of current systems. Others like the Wikipedia dumps(1), the European Union multilingual corpora (2)(often used in automatic translation) or the RFCs (3) could be used but are not standard in the research community.

QUESTION: does any open text document corpus of a decent size and quality in English exist, that is widely used for evaluation of information retrieval systems? The widespread usage puts the result of one's own evaluation into a meaningful context.

Answer 1

You might look at How can I Share my Data Sets Without Worrying About Copyright Issues?

I don't know if the author is willing to share or not. He was concerned about copyright issues.

Answer 2

If you're interested in Twitter search, you can prep your own tweet IR collection using the open tools described on the TREC Microblog (MB) 2011/2012 pages. The test queries and judgements are also openly available from TREC, as is the tool for evaluating (trec_eval)