I am looking for datasets that contain texts as well as notations for sentence boundaries. I plan to use it to assess a sentence segmentation system. I am mostly interested in texts in English at the moment, but curious about other languages as well.
asked Apr 5, 2016 at 18:09
Add a comment
|
1 Answer
The Pragmatic Segmenter segmentation library contains sentences for the common edge cases in English as well as some other languages. The README also includes links to a lot of research and other segmentation libraries.
You can also access the Brown Corpus from the NLTK Corpora page. That dataset has been used to test segmentation libraries.
