I am trying to test one name entity tagger. I am using NLTK's nltk.HiddenMarkovModelTagger.train(trains) which requires data in Brown corpus's tagged_sents() format.

I was looking for a free benchmark entity tagged corpus in the said format of Brown corpus.

I could not find anyone.

I found conll2000 in NLTK itself from which I am trying to extract data as (w,c) format as in http://pastebin.com/AttNEbSe and test.

I am thinking of an alternative. I found two online NERs, like Stanford Online NER Cognitive Computing Group NER

If I paste data generate NER like http://pastebin.com/x1bVW6u7 and create data in said format, may I use either of them as benchmark data?

Please suggest

  • Tagging is very expensive and only a few institutions can afford it. I doubt that you will find a free alternative or a free Brown-like corpus.
    – marfi
    Jan 25 '17 at 6:44
  • I understand it. I am trying to produce sizeable data to do Benchmark tests at present. Full corpus I can not work out this way. If I need that I may have to talk with my Corpus Linguistics friends.
    Jan 26 '17 at 16:37
  • Maybe this question can help you. Feb 13 '17 at 18:01
  • Thanks for your time and effort. I posted this question after going through that link. Don't worry my local ML fraternity verified I am right if I create benchmark test data in this way.
    Feb 14 '17 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.