Twitter - Labelling system for sentiments and other things

Question

I have build a program that log tweets from stream by hashtag into a MongoDB database and a website to label them.

The website loads a random tweet and allows me to label them with three buttons (Negative, Neutral, Positive). My goal was to make a huge database of tweets and labels for machine learning purpose.

I was wondering if putting it online/making it available, others will find it interesting and allow them to classify tweets to add to the data. I don't want to build a full website that no one will use. I was thinking about give it as an open source platform.

Do you think this project is interesting? Is it worth finishing it?

As I store the full tweet I thought that in the future I will allow different classification e.g. (Pos, Neutrale, Neg / Categories ex: US. Politics, etc.) and people will be able to build a training set off of it, for example Negative/Positive tweets about location (browsing tweets with geolocation) etc.

In the future, maybe give incentive for people to rate them, add advertisement to finance it maybe. For example a tree learning for label questions (Ex : Categorie "US. Politics" -> "Is it about Hillary Clinton ?" -> Is it positive ? ... a bit like akinator)

Any comment / critic / encouragement / support is welcome.

I hope I was clear in my explanation and that I am not off topic on this forum.

Answer 1

Yes, please share this information with the world.
Sentiment analysis is always desired, I see plenty of questions regarding sa, as well as seeking twitter datasets on here regularly.
Don't worry about not wanting to build a site/maintain something/etc., sharing the data is entirely enough.
As far as storage/sharing, you have a plethora of options, here's what I would do:
store a compressed version of the dataset on my website and write a blogpost about it, keep the data versioned in a GitHub repository, as well as uploaded it to Open Data SE's datahub.io datasets account for posterity.

EDIT/UPDATE: adding commentary for allowing user labelling/downloading/access.
Crowd Sourcing is your friend here, particularly when the topic is niche; users will be less, but (imo) much more likely to complete/follow-through desired actions. Creating an open source platform that allows for users to sign-in, select/submit form controls, and access certain features certainly sounds reusable, providing benefit(s) to the open source community.
Being that you don't want to get boggled down in the technical details, I would search GitHub rigoursly for repositories like this...you can probably find multiple repos that already do this entirely. I encourage this for two reasons: a) again, you don't want to waste time on the site when the real prize is the dataset. b) if you're not going to reinvent the wheel, I think it is optimal to use what is already out there when possible; open source portals tend to have entirely too many repos that do very similar things, while not being optimized at all. No point in muddying up the waters anymore than they already are.
If I had to roll my own, I'd hobble some spaghetti code (copy/pasta) together along the lines of this:
- index document consisting of a <form> for users to sign in with. I'd use social login libraries to offer users easy options for log-in, which also makes your job of maintaining the backend easier. Offer a number of options, not just facebook/google: I'd go for Mozilla Persona/StackOverflow/OpenID/Twitter/Disqus/Identica/etc.; your options may vary, the only thing that matters is using logins that your users will most likely have.
- labelling <form> for users to label sentiments of tweets. You can display this in any manner you choose, I would: on login <form> submission success, ajax in the labelling <form>, and serve up tweets to be analyzed. - keep a running <progress> somewhere in the post login document that displays to the user percentage completed towards accessing dataset. - set up the backend to allow for multiple user sessions; i'm assuming that most will not complete 100 in one sitting.
- set up an extremely lightweight backend. emphasis on extremely lightweight. currently I'd most likely opt for postgres and connect in django. Back to lightweight: you don't need to collect much data here: user inputs, db of tweets, tallying inputs of tweets, user progression towards access to data...I may be missing some, but those are what stand out immediately.