It would be helpful if there were an instance that migrated all of this to Lemmy so that we could access it from any other instance, instead of having to download it for local browsing.
This was data from pushshift before Reddit nuked it in March. You can find this torrent (called “Reddit comments/submissions 2005-06 to 2022-12”) and others, including 2023-01 and 2023-02, on https://academictorrents.com by user Watchful1.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !piracy@lemmy.dbzer0.com
⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.
What’s the context and background here? It would be nice to know what’s in some of these 4GB compressed files before downloading them.
Agreed, what’s in these? Raw text? Image metadata?
Nothin’ but JSON compressed with zstd. You can also grab individual subreddits at https://the-eye.eu/redarcs/
It has json files with every written post on Reddit.