• 0 Posts
  • 3 Comments
Joined 1Y ago
cake
Cake day: Oct 21, 2023

help-circle
rss

syncthing falls down when you hit millions of files.

Platform agnostic? Rsync from the perspective of duplicating the client-presented data.

Or rclone, another great tool. I tend to use this for storage providers instead of between self hosted systems (or data center fully self-managed systems.)

If the NAS uses zfs then zfs send/recv is the best, because you can send only the changed blocks. Want to redo the names on every single movie? No problem! I do recommend sanoid or syncoid, I don’t remember which. ZFS snapshots are not intuitive, make sure to do LOTS of testing with data you don’t care about, and which is small.

In terms of truly duplicating the entire NAS, we can’t help without knowing which NAS you’re using. Or since this is selfhosted, which software you used to build a NAS.


Backblaze b2 and rsync.net (which shouldn’t be a problem for rclone)

Neither is sketchy in the slightest and run solid services with many options.


Have you isolated source formats? Does it do this with h264 and hevc?

Tried stopping other containers and/or all other services? Could have less to do with pure power and more with sheer volume (scheduling.)

Any issues if you use the current server as a file server and play the video on another machine? Any modifications to playback speed?

If none of that works you might try iperf3 to check network speeds. Or fio for disk speeds. Run tests covering all kinds of situations, you’re not so much looking for max speeds but instead for inconsistencies.

Don’t get too lost in the small stuff. Isolate systems. Might even move or clone the boot drive to another machine as a test. Try a different switch. That kind of thing.

Good luck!