See my other reply here.

See my other reply here.

See my other reply here.

I followed the guide found here, however with a few modifications.

Notably, I did not encrypt the borg repository, and heavily modified the backup script.

#!/bin/bash -ue

# The udev rule is not terribly accurate and may trigger our service before
# the kernel has finished probing partitions. Sleep for a bit to ensure
# the kernel is done.
#
# This can be avoided by using a more precise udev rule, e.g. matching
# a specific hardware path and partition.
sleep 5

#
# Script configuration
#

# The backup partition is mounted there
MOUNTPOINT=/mnt/external

# This is the location of the Borg repository
TARGET=$MOUNTPOINT/backups/backups.borg

# Archive name schema
DATE=$(date '+%Y-%m-%d-%H-%M-%S')-$(hostname)

# This is the file that will later contain UUIDs of registered backup drives
DISKS=/etc/backups/backup.disk

# Find whether the connected block device is a backup drive
for uuid in $(lsblk --noheadings --list --output uuid)
do
        if grep --quiet --fixed-strings $uuid $DISKS; then
                break
        fi
        uuid=
done

if [ ! $uuid ]; then
        echo "No backup disk found, exiting"
        exit 0
fi

echo "Disk $uuid is a backup disk"
partition_path=/dev/disk/by-uuid/$uuid
# Mount file system if not already done. This assumes that if something is already
# mounted at $MOUNTPOINT, it is the backup drive. It won't find the drive if
# it was mounted somewhere else.
(mount | grep $MOUNTPOINT) || mount $partition_path $MOUNTPOINT
drive=$(lsblk --inverse --noheadings --list --paths --output name $partition_path | head --lines 1)
echo "Drive path: $drive"

# Log Borg version
borg --version

echo "Starting backup for $DATE"

# Make sure all data is written before creating the snapshot
sync


# Options for borg create
BORG_OPTS="--stats --one-file-system --compression lz4 --checkpoint-interval 86400"

# No one can answer if Borg asks these questions, it is better to just fail quickly
# instead of hanging.
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no


#
# Create backups
#

function backup () {
  local DISK="$1"
  local LABEL="$2"
  shift 2

  local SNAPSHOT="$DISK-snapshot"
  local SNAPSHOT_DIR="/mnt/snapshot/$DISK"

  local DIRS=""
  while (( "$#" )); do
    DIRS="$DIRS $SNAPSHOT_DIR/$1"
    shift
  done

  # Make and mount the snapshot volume
  mkdir -p $SNAPSHOT_DIR
  lvcreate --size 50G --snapshot --name $SNAPSHOT /dev/data/$DISK
  mount /dev/data/$SNAPSHOT $SNAPSHOT_DIR

  # Create the backup
  borg create $BORG_OPTS $TARGET::$DATE-$DISK $DIRS


  # Check the snapshot usage before removing it
  lvs
  umount $SNAPSHOT_DIR
  lvremove --yes /dev/data/$SNAPSHOT
}

# usage: backup <lvm volume> <snapshot name> <list of folders to backup>
backup photos immich immich
# Other backups listed here

echo "Completed backup for $DATE"

# Just to be completely paranoid
sync

if [ -f /etc/backups/autoeject ]; then
        umount $MOUNTPOINT
        udisksctl power-off -b $drive
fi

# Send a notification
curl -H 'Title: Backup Complete' -d "Server backup for $DATE finished" 'http://10.30.0.1:28080/backups'

Most of my services are stored on individual LVM volumes, all mounted under /mnt, so immich is completely self-contained under /mnt/photos/immich/. The last line of my script sends a notification to my phone using ntfy.