Open Data Stack Exchange is a question and answer site for developers and researchers interested in open data. It only takes a minute to sign up.
Sign up to join this community
I have a dataset of tweets annotated by a group of people and I want to distribute it in Github without the user id or Tweet id, do you know if that possible? only the tweet_text and the label.
Given that the work where I collected these tweets was accepted recently in a NLP research conference. Thus, I want to add a link for the dataset in the article.
An accurate answer will depend on how many tweets you want to release: MBs, GBs, TBs?
You can take a look at the awesome-twitter-data list to see what other researchers are doing.
One option is to publicly release the tweet_id and user_id and then also a small script for downloading those tweets/users, with instructions for generating and adding your own authentication tokens. This would make more sense for the public (github) and would be fully compliant with the ToS. For the conference, you could additionally post an ephemeral link to the full dataset (text) download, explaining that the link will expire in N months.
Instead of Github, you can also share the data on archive.org, which would not be related to you Github account. See, for example, how ArchiveTeam (not affiliated with archive.org) is storing twitter-stream data: https://archive.org/search.php?query=twitterstream
If you do create a repo with tweet texts, in general, the likely "worst case" would be a takedown notice, which you could comply with if it ever comes to that. If the data is hosted at archive.org, then they could deal with the takedown notice.
Once you've published the dataset, please add it to the list above (and as an answer here!)
