Edit: Results tabulated, thanks for all y’alls input!
Backup while it is expected to be idle @MangoPenguin@lemmy.blahaj.zone @khorak@lemmy.dbzer0.com @dandroid@sh.itjust.works
@Darkassassin07@lemmy.ca suggested adding a real long-ass-backup-script to run monthly to limit overall downtime
Shutdown all containers -> backup @PotatoPotato@lemmy.world
Leveraging NixOS impermanence, reboot once a day and backup @thejevans@lemmy.ml
(it seems pg_dumpall
for Postgres and mysqldump
for mysql (though some images with mysql don’t have that command for meeeeee))
Dump Postgres via pg_dumpall
on a schedule, backup normally on another schedule @RegalPotoo@lemmy.world
Dump mysql via mysqldump and pipe to restic directly @youRFate@feddit.de
Dump Postgres via pg_dumpall
-> backup -> delete dump @2xsaiko@discuss.tchncs.de @SteveDinn@lemmy.ca
Make your own docker image (https://gitlab.com/trubeck/postgres-backup) and set to run on a schedule, includes restic so it backs itself up @Undaunted@discuss.tchncs.de (thanks for uploading your scripts!!)
Add docker image prodrigestivill/postgres-backup-local
and set to run on a schedule, backup those dumps on another schedule @brewery@lemmy.world @Lem453@lemmy.ca (also recommended additionally backing up the running database and trying that first during a restore)
LVM snapshot -> backup that @butitsnotme@lemmy.world
ZFS snapshot -> backup that @ikidd@lemmy.world (real world recovery experience shows that databases act like they’re recovering from a power outage and it works)
(I assume btrfs snapshot will also work)
pg_dumpall
, zips, then rclone them @DeltaTangoLima@reddrefuge.comI’ve searched this long and hard and I haven’t really seen a good consensus that made sense. The SEO is really slowing me on this one, stuff like “restic backup database” gets me garbage.
I’ve got databases in docker containers in LXC containers, but that shouldn’t matter (I think).
I’ve seen:
None seem turnkey except for the first, but since so many other options exist I have a feeling the first option isn’t something you can rest easy with.
I’d like to minimize backup down times obviously, like what if the backup for whatever reason takes a long time? I’d denial of service myself trying to backup my service.
I’d also like to avoid a “long ass backup script” cause autorestic/borgmatic seem so nice to use. I could, but I’d be sad.
So, what do y’all do to backup docker databases with backup programs like Borg/Restic?
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
I setup borg around 4 months ago using option 1. I’ve messed around with it a bit, restoring a few backups, and haven’t run into any issues with corrupt/broken databases.
I just used the example script provided by borg, but modified it to include my docker data, and write info to a log file instead of the console.
Daily at midnight, a new backup of around 427gb of data is taken. At the moment that takes 2-15min to complete, depending on how much data has changed since yesterday; though the initial backup was closer to 45min. Then old backups are trimmed; Backups <24hr old are kept, along with 7 dailys, 3 weeklys, and 6 monthlys. Anything outside that scope gets deleted.
With the compression and de-duplication process borg does; the 15 backups I have so far (5.75tb of data) currently take up 255.74gb of space. 10/10 would recommend on that aspect alone.
/edit, one note: I’m not backing up Docker volumes directly, though you could just fine. Anything I want backed up lives in a regular folder that’s then bind mounted to a docker container. (including things like paperless-ngxs databases)
Love the detail, thanks!!
I have one more thought for you:
If downtime is your concern, you could always use a mixed approach. Run a daily backup system like I described, somewhat haphazard with everything still running. Then once a month at 4am or whatever, perform a more comprehensive backup, looping through each docker project and shutting them down before running the backup and bringing it all online again.
Not a bad idea for a hybrid thing, especially people seem to say that a running database backup at least some of the time most of the time with no special shutdown/export effort is readable. And the dedupe stats are really impressive