0

I'm looking to make a ML model to attempt to determine if a given sentence/paragraph fits certain parameters. Right now the main one I'm looking at is particularly harmful sentences, so content that exhibits racism/sexism/violence or threats.

I'd also like to be able to determine other things, but for now these are my main priority.

Are there any public data sets, or even ideas for things I can scrape in order to get this data? Here's an example of what I'd like to be able to get after cleaning the data

Content, Sexist, Racist, Violent
Someone needs to teach these pigs a lesson., 0, 0, 1
The batter crushed that ball!, 0, 0, 0

At the same time I'd rather let violent statements pass the filter than limit freedom of speech. So I'd like to be a little conservative with the model and want to make sure to be careful with its training. I'd rather minimize false positives instead of false negatives. So if possible I'd like to get data that contains similar language but doesn't fit the description such as

Black people are more likely to commit crimes., 0, 0, 0

I'm trying to censor messages where I can, so I'd also need a quite large data set in order to cover as many edge cases as possible.

|improve this question
  • half joking, but Trump's twitter feed? nytimes.com/interactive/2016/01/28/upshot/… – philshem♦ Jul 8 at 6:35
  • That's not a bad suggestion. Look for social media postings of individuals and organizations who are known for using sexist, racist and violent statements. There's been a lot of media attention to white supremacist organisations lately, so it shouldn't be hard to find some of their online hangouts (eg, one I've heard of but don't know how it's spelled is four chan). That will bias your model towards one type/flavor of racism, so try to figure out what other types might exist and how to find them. Be aware that researching this will lead to some unpleasant youtube recommendations. – csk Jul 8 at 15:33
  • look up natural language processing; there's tons of nlp datasets out there, particularly ones post 2016 us federal election, doing basically what you want. – albert♦ Jul 8 at 21:40
  • 1
    how can speech be violent? i'm being 100% serious. what is your methodology for violent speech? – albert♦ Jul 8 at 21:53
2

Hate Speech and Offensive Language repository for the paper "Automated Hate Speech Detection and the Problem of Offensive Language", ICWSM 2017.

There is a plethora of datasets like this available as open data.

You may already know this and/or have done this, but it seems like this needs to be fleshed out in much more detail.
For example, "particularly harmful sentences" - what does that mean?

Harmful to whom?
Harmful how?
Harmful always/certain context(s)?

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.