I was able to get a list of the most recent anime from aniwave using this reddit thread Goofhey made: https://old.reddit.com/r/animepiracy/comments/1f2xbg7/archived_aniwaves_12000_anime_pages_on_wayback/ and scraping all 411 pages archived in the wayback machine. Back in March I built a web scraper using python requests and beautiful soup and got a list of all of aniwaves current anime sorted in alphabetical order. I compared that list to what was most recently saved in wayback machine by Goofhey. I discovered that some anime were missing. I guess its because the pages saved by Goofhey in the wayback machine were sorted by recently updated and since recently updated is constantly changing it caused some anime to be excluded but I think I got all or most of them by combining both list. Then a using a Disqus scraper I made I fed it links from the list I made and downloaded the comments. I tested the scraper on various sites(myasiantv, gogoanime, aniwave) the scraper can most likely work on most websites that use disqus with a bit of tweaking.
I also managed to get all of Gogoanime’s old comments from before 2021 going all the way back to 2014/2015. Something interesting I found is that a few copycat websites(6anime, gogoanimes) still have all of gogoanimes old comments from before 2021.
Most commented pages on each site sorted from most(Aniwave) to least(Anitaku) amount of comments:
Aniwave(9anime): Attack on Titan The Final Season Part 3 Episode 1
Gogoanime Old comments: Yuri on Ice Category page
Anitaku(Gogoanime): Kimetsu no Yaiba Yuukaku Hen Episode 10
Folders were compressed into tarballs with zstd level 9 compression:
Aniwave(9anime): TOTAL GB UNCOMPRESSED: 23.7 GiB TOTAL GB COMPRESSED:1.4 GiB
Gogoanime: TOTAL GB UNCOMPRESSED: 16.4 GiB TOTAL GB COMPRESSED: 769.5 MiB
Anitaku(Gogoanime): TOTAL GB UNCOMPRESSED: 7.2 GiB TOTAL GB COMPRESSED: 326.7 MiB
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !piracy@lemmy.dbzer0.com
⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.
Awesome. May I ask the Disqus scraper you used?
it my own custom scraper that I have been slowly working on since January.