4

There have been a number of stories in the last couple of dats claiming a team of developers were able to scrape practically the entirety of the Parler social media platform before it went offline.

For example:

https://gizmodo.com/every-deleted-parler-post-many-with-users-location-dat-1846032466

https://www.buzzfeednews.com/article/johnpaczkowski/amazon-parler-aws

But so far I have been unable to find any evidence this data set actually exists. There are a couple of data sets labeled 'parler' on archive.org but they certainly aren't anywhere close to 70TB of data.

Does anyone know the status of this dataset? Is it available anywhere? Does it really exist?

Improve this question
1
  • the data is now available, see the answer below. I'll update once there is a torrent link
    – philshem
    Commented Jan 20, 2021 at 16:45

1 Answer 1

Reset to default
4

update: the full dataset is available here: https://ddosecrets.com/wiki/Parler

Files are accessible from two Amazon S3 buckets, ddosecrets-parler (32.1TB) and ddosecrets-parler-images (235GB).

At this time, we only have a partial scrape of text posts (1.6 million), which was provided by a 3rd party. The 18 GB torrent can be downloaded here: https://ddosecrets.com/images/d/de/Parler_2020-01-06_posts-partial.torrent

At the moment only the metadata is available. This means not the content itself, but the creation, id, creation date, and some gps tagging. The full dataset (70+ TB) is being processed by https://www.archiveteam.org/index.php?title=Parler and will be available soon.

Grab status: https://tracker.archiveteam.org/parler/

in the meantime, the metadata:

https://donk.sh/metadata.tar.gz (check for mirrors on this twitter thread)

Also a magnet torrent link:

magnet:?xt=urn:btih:1723E27BC79186C4574FF056DDB458D771C26E2F&dn=metadata.tar.gz&tr=udp%3a%2f%http://2ftracker.openbittorrent.com%3a1337%2fannounce&tr=udp%3a%2f%http://2ftracker.opentrackr.org%3a1337%2fannounce

MD5: 77299fb15bfa2356b542cdeec944e5c0

Jupyter notebook to parse gps-tagged metadata (videos): https://gist.github.com/kylemcdonald/8fdabd6526924012c1f5afe538d7dc09 (tweet)

I also made a parser for a different (meta)data source (this one was only gps-tagged videos in csv format): https://github.com/philshem/parler-switzerland

Improve this answer
1
  • 1
    That's great, thank you. I was beginning to think it was just an internet legend! I will keep an eye on those links for the full data set
    – Matt
    Commented Jan 12, 2021 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.