Is there a freely available test dataset of research papers in computer science field? I need to evaluate my text retrieval system which uses a computer science domain ontology called CSO, so the dataset should come with a ground truth/relevance judgments for text retrieval task and/or classification task.
There are some freely available collections of research papers, but whether they are annotated in the way that you need is a different story. Unless the kind of task that you're targeting has been done before, it's likely you'll have to do some annotation work yourself.
I'm aware of the ACL Anthology Corpus, which is about CS but more specifically NLP papers.
Another option to investigate would be Medline and/or PMC data: while this is about biomedical research, some of the journals it indexes overlap with computer science. This it's so massive there might be enough CS data in there. The advantage of Medline is that the abstracts are indexed with Mesh descriptors which are often used as a kind of gold standard, although I don't know if it would contain enough CS concepts.