I just got my home server up and running and was wondering what you guys recommend for backups. I figure it will probably be worth having backups on cloud servers tjay are external, are there any good services yall use for that?
I hate to ask the scary question, but have you tried to restore your backups before? I used Duplicati and discovered that none of my backups were usable and ended up switching to Duplicacy.
I have an unraid server which hosts an docker image of Duplicacy. It is paid though for the web interface. And it backs up to Backblaze B2. I have roughly 175GB backed up, for which I pay $0.87 a month.
Do you have other clients backing up to your unraid? I’m looking for a complete solution to backing up end user workstations (windows, Mac and Linux) to my unraid server then backing up my unraid server to something like wasabi, Amazon, backblaze, etc. Preferably a single solution.
Somehow “took me a while to wrap my head around it” doesn’t make me feel comfortable. Apart from git-annex themselves saying that they aren’t a backup system and just a building block to maybe create one, a backup system should imho be dead simple sind easy to understand.
Once you actually start using it it is dead simple and integrates extremely well with stuff you (might) already do.
I have a Git repo which contains my dotfiles + every “large” (annexed) file I want to back up under my home directory.
Git annex automatically tracks where all annexed files are, how many copies there are on various repos, etc.
I add and modify files using mostly standard git commands.
It supports pretty much anything as a “remote”.
It’s extremely simple to restore backups locally or remotely.
Basically Git annex is the Git of backup solutions IME, allowing you extreme flexibility to do exactly what you want, provided you take the time to learn how to do what you want.
Features that are important to me are things like an easy overview of all backup jobs (ideal via a web UI), snapshots going back every day for a week and after that every month. Backup to providers like Backblaze or AWS and the ability to browse these backups and individual snapshots.
I’d assume that you can build all of this with git annex in some way. But I really want something that works out of the box. E.g. install the backup software give it some things to backup and an B2 bucket and then go.
What I’m curious about is that the git-annex site explicitly days that they aren’t a backup system, but you describe it as such.
I don’t care about stuff working OOTB - half the fun is messing around with things IMO.
I also don’t care about web UIs and similar features (I always got the impression from selfhosting communities that this is considered important but I never really understood why - I don’t spend all day staring at statistics, and when I need some info I can get it through the terminal usually).
Also, first sentence on Git Annex’s website:
git-annex allows managing large files with git, without storing the file contents in git. It can sync, backup, and archive your data, offline and online. Checksums and encryption keep your data safe and secure.
Not sure why you’re saying it’s not a backup solution.
Efit: I guess the “what git-annex is not” page says this.
To quote a comment by the creator on the same page:
It’s definitely possible to use git-annex in backup-like ways, but what I want to discourage is users thinking that just putting files into git-annex means that they have a backup. Proper backups need to be designed, and tested. It helps to use software that is explicitly designed as a backup solution. git-annex is more about file distribution, and some archiving, than backups.
So basically he says this just so people won’t yell at him when they fail to use it as a backup solution correctly.
I don’t care about stuff working OOTB - half the fun is messing around with things IMO.
I generally agree. Backups for me are just something I don’t want to tinker with. It’s important to me that they work OOTB, are easy to grasp and I have a good overview.
The web interface is important to me because it gives me that overview from any device I’m currently using without needing to type anything into a terminal. The OOTB is important to me since I want to be able to easily set this all up again even without access to my Ansible setup or previous configuration.
To each their own. I’m not saying your way of doing this is wrong. It’s just not for me. This is just my reasoning / preferences. It’s also the reason something like borg wasn’t my chosen solution, even though it’s generally considered great.
I had never heard of rsync.net until now. I like the idea but it seems more expensive than B2. $15/TB vs $5/TB. Am I doing the math wrong or reading it wrong?
I don’t see it on their website right now, but they offer a discount if you’re using something like restic/borg and only need scp/sftp access.
Their support is also super friendly. I’ve had an account forever and got moved to the 100+ TB pricing even though I have < 50TB stored. YMMV but it doesn’t hurt to ask if they have any additional discounts.
Also keep in mind that B2 charges for bandwidth too. It’s $5/TB for storage, but $10/TB to download that same data.
rsync.net and learn to use Borg; they’re stupid cheap if you’re technically proficient enough to handle the Borg setup yourself. Like, charge by the gigabyte, but it’s 1.5¢/GB at the most expensive, and cheaper in bulk
I second restic. Have been using it for a year now and have been generally very happy. Actually had to use it in a couple occasions to restore directory content and even recover a complete workstation drive. I have had relatively easy success in both scenarios.
+1 for backblaze. I use docker for everything and mounted volumes directly in the folder alongside a docker compose file. So I just tar my services directory with everything in it, and pipe it to rclone which connects to backblaze and has a “cat” feature so you can pipe data directly to the destination.
I do once a day rsync my data to another drive. I can restore a file, if I accidentaly deleted it. Important stuff goes encrypted via rclone additionaly to a hetzner storagebox.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Duplicati, to a friend’s home server who lives in another town.
I hate to ask the scary question, but have you tried to restore your backups before? I used Duplicati and discovered that none of my backups were usable and ended up switching to Duplicacy.
This is why I switched to restic.
An important question though.
I have, when I first set it up, and again once when I needed to.
It works just fine for me, but I’ve heared scary storries so now Im using:
+1 for Duplicacy. It just works, truly does. Duplicati on the other hand seems to work, but has a tendency to fail on restore, just as you described.
How would one realistically go about testing their backup? Do you need a bunch of empty drives?
You don’t need to do full restores, spot check random files.
Backups and archived files go to my home server which then backups to backblaze b2.
My setup exactly, with the addition of using M-Discs to backup my core important stuff.
Backblaze.
I have an unraid server which hosts an docker image of Duplicacy. It is paid though for the web interface. And it backs up to Backblaze B2. I have roughly 175GB backed up, for which I pay $0.87 a month.
Paid for the web interface as well. I really like that it’s super simple and just does it’s job. That would be the one I’d also recommend.
Do you have other clients backing up to your unraid? I’m looking for a complete solution to backing up end user workstations (windows, Mac and Linux) to my unraid server then backing up my unraid server to something like wasabi, Amazon, backblaze, etc. Preferably a single solution.
Yes, I have another server automatically rsyncing important config files to a nfs share. And my pc has a samba share where I manually backup files to.
Look into Veeam. The free version should be enough for this workflow.
This is almost my exact backup workflow, with another location in between. Duplicacy is great, highly recommend.
Git Annex.
Took me a while to wrap my head around it, but nothing comes close to it once you set it up.
Edit: should have read the post more carefully, I use Git Annex both locally and on a VPS I rent from openbsd.amsterdam for off-site backups.
Somehow “took me a while to wrap my head around it” doesn’t make me feel comfortable. Apart from git-annex themselves saying that they aren’t a backup system and just a building block to maybe create one, a backup system should imho be dead simple sind easy to understand.
Once you actually start using it it is dead simple and integrates extremely well with stuff you (might) already do.
I have a Git repo which contains my dotfiles + every “large” (annexed) file I want to back up under my home directory.
Git annex automatically tracks where all annexed files are, how many copies there are on various repos, etc.
I add and modify files using mostly standard git commands.
It supports pretty much anything as a “remote”.
It’s extremely simple to restore backups locally or remotely.
Basically Git annex is the Git of backup solutions IME, allowing you extreme flexibility to do exactly what you want, provided you take the time to learn how to do what you want.
Features that are important to me are things like an easy overview of all backup jobs (ideal via a web UI), snapshots going back every day for a week and after that every month. Backup to providers like Backblaze or AWS and the ability to browse these backups and individual snapshots.
I’d assume that you can build all of this with git annex in some way. But I really want something that works out of the box. E.g. install the backup software give it some things to backup and an B2 bucket and then go.
What I’m curious about is that the git-annex site explicitly days that they aren’t a backup system, but you describe it as such.
I don’t care about stuff working OOTB - half the fun is messing around with things IMO.
I also don’t care about web UIs and similar features (I always got the impression from selfhosting communities that this is considered important but I never really understood why - I don’t spend all day staring at statistics, and when I need some info I can get it through the terminal usually).
Also, first sentence on Git Annex’s website:
Not sure why you’re saying it’s not a backup solution.
Efit: I guess the “what git-annex is not” page says this.
To quote a comment by the creator on the same page:
So basically he says this just so people won’t yell at him when they fail to use it as a backup solution correctly.
I generally agree. Backups for me are just something I don’t want to tinker with. It’s important to me that they work OOTB, are easy to grasp and I have a good overview.
The web interface is important to me because it gives me that overview from any device I’m currently using without needing to type anything into a terminal. The OOTB is important to me since I want to be able to easily set this all up again even without access to my Ansible setup or previous configuration.
To each their own. I’m not saying your way of doing this is wrong. It’s just not for me. This is just my reasoning / preferences. It’s also the reason something like borg wasn’t my chosen solution, even though it’s generally considered great.
I understand your position, though I always have access to a terminal pretty much so I still don’t see the point of a web UI.
Though I realize I’m in the minority here.
I use wasabi s3, I back up to that using restic.
rsync.net is great if you need something simple and cheap. Backblaze B2 is also decent, but does have the typical download and API usage cost.
I use rsync and backblaze b2.
I use it for version control and cost, about £2 for 750GB
How are you using rsync with B2? Are you mounting the bucket locally?
Sorry I’m using rclone.
I had never heard of rsync.net until now. I like the idea but it seems more expensive than B2. $15/TB vs $5/TB. Am I doing the math wrong or reading it wrong?
I don’t see it on their website right now, but they offer a discount if you’re using something like restic/borg and only need scp/sftp access. Their support is also super friendly. I’ve had an account forever and got moved to the 100+ TB pricing even though I have < 50TB stored. YMMV but it doesn’t hurt to ask if they have any additional discounts.
Also keep in mind that B2 charges for bandwidth too. It’s $5/TB for storage, but $10/TB to download that same data.
Sure but backup is mostly data in (free on B2). Data out is rare, if ever.
If i wasn’t backing up 12TB+ I would actually go with rsync for the features though.
Borgbase looks interesting, too.
I’ve never heard of it either, but I came to the same conclusion as you
Yeah rsync.net has always been pricey.
When I researched what to use for my backup I found rsync.net. They have some nice features nobody else seems to support, like they support ZFS send/receive https://www.rsync.net/products/zfsintro.html
But in the end the price made me go with borgbase.com
Borgbase with Borgmatic (Borg) as the Software. As far as I know the whole Borgbase Service is from a Homelab guy (with our needs in mind).
Also 3-2-1 rule!
Also team borgmatic here. ;)
rsync.net and learn to use Borg; they’re stupid cheap if you’re technically proficient enough to handle the Borg setup yourself. Like, charge by the gigabyte, but it’s 1.5¢/GB at the most expensive, and cheaper in bulk
Restic or Kopia, both to Backblaze.
I second restic. Have been using it for a year now and have been generally very happy. Actually had to use it in a couple occasions to restore directory content and even recover a complete workstation drive. I have had relatively easy success in both scenarios.
I’ve always found them pretty similar. How’d you chose one or another?
I know Restic before Kopia and made a set of systemd units to run Restic backups on my home server and office workstation (both online 24/7).
Kopia seems much nicer for a regular user, so I use it on my and family laptops. I used to use Duplicati there, but that project seems dead.
Thank you :)
+1 for backblaze. I use docker for everything and mounted volumes directly in the folder alongside a docker compose file. So I just tar my services directory with everything in it, and pipe it to rclone which connects to backblaze and has a “cat” feature so you can pipe data directly to the destination.
Restic and then rclone to backblaze? Or is there a way to restic directly to backblaze?
I do prefer having a local copy of my backups (and therefore i use rclone), but afaik restic does support b2 directly…
I do once a day rsync my data to another drive. I can restore a file, if I accidentaly deleted it. Important stuff goes encrypted via rclone additionaly to a hetzner storagebox.
see also this previous discussion:
https://kbin.social/m/selfhosted@lemmy.world/t/182362/What-are-your-backup-solutions
I use SyncThing to backup our cell phones to my on-prem server, and then use BackBlaze Personal Backup for a cloud copy.
External HDD in my wifi network. It runs Samba. I can just drag and drop folders and it transfers over wifi.
Are you using a Synology NAS?
Veeam backup and replication at home and at work. At home a copy goes to a NAS, another copy goes to backblaze b2 currently.