I have set up a refurbished PC as a media PC with storage. The OS, Debian, is on an m.2 nvme disk of 256 GB. I have connected 2x4TB risks in zfs mirror mode to store my media.
Off late, while booting, I’ve noticed some messages that suggest that the health of nvme disk is not good. Searching the error, i realised that I should not rely on it. I’ve done a number of tweaks to set up my system the way I like that I want to save by creating an image of the OS drive on a fresh nvme disk of same size that I have.
How do I go about doing it? I could boot using a live USB and create the image on the HDDs but the live USB OS won’t recognise my zfs, right? Is using another external disk or another PC my only option here?
Thanks and cheers!
PS: The machine is a HP Elitedesk 800 G3 that has a wifi port that I’ve heard can be used as additional port for m.2 drives. Is it true?
don’t mess around with imaging to a file on the zfs, then restoring it. simply clone nvme -> nvme using a usb nvme adapter then replace the internal with the clone.
usb nvme adapters are not expensive and it likely won’t be the only time you need it. they are a handy accessory to have on hand if you have nvme storage.
If you want an image, it doesn’t matter what the underlying file system is. You should be able to use a tool like Clonezilla and get a 1:1 copy. Depending how you’ve set up partitioning, you could also use sgdisk to set up the proper partitions and zfs send/recv for the new data portion of the drive and install a boot loader. That’s probably the way I’d go in this instance.
I get the part that the cloning software does not care for the underlying OS. My worry is the fact that I’ll run the cloning software/command from a live USB which will not be able to detect the zfs mirror on my backup drive on its own and thus break the zfs mirror with bad consequences for the existing data. I could not find any commands to make the live USB OS discover and respect the existing zfs configuration.
It sounds like nobody actually understood what you want.
You have a non-ZFS boot drive, and a big ZFS pool, and you want to save an image of the boot drive to the pool, as a backup for the boot drive.
I guess you don’t want to image the drive while booted off it, because that could produce an image that isn’t fully self-consistent. So then the problem is getting at the pool from something other than the system you have.
I think what you need to do is find something else you can boot that supports ZFS. I think the Ubuntu live images will do it. If not, you can try something like re-installing the setup you have, but onto a USB drive.
Then you have to boot to that and zfs import your pool. ZFS is pretty smart so it should just auto-detect the pool structure and where it wants to be mounted, and you can mount it. Don’t do a ZFS feature upgrade on the pool though, or the other system might not understand it. It’s also possible your live kernel might not have a new enough ZFS to understand the features your pool uses, and you might need to find a newer one.
Then once the pool is mounted you should be able to dd your boot drive block device to a file on the pool.
If you can’t get this to work, you can try using a non-ZFS-speaking live Linux and dding your image to somewhere on the network big enough to hold it, which you may or may not have, and then booting the system and copying back from there to the pool.
Phew! I almost believed I was asking for something beyond the scope of linux-fu. English not being my first language may be part of the reason but still I think I covered everything that was relevant.
Yes, that’s exactly what I want and your post has given me the clarity I needed. M.2 wifi slots don’t support disks so that option is definitely out. I’m going to boot with the latest Ubuntu live OS on a USB and attempt what you’ve outlined.
I don’t have anything really critical on the zfs that is not backed up separately so I’m definitely going to attempt this and learn in the process.
Use an Ubuntu live USB, all recent versions of Ubuntu have ZFS drivers baked into the live environment. Then you should add your new SSD to the ZFS pool, and remove the old one from the ZFS pool. Your m.2 WiFi slot should be able to host the 2nd drive while you do this, but if not you can use an external USB housing for it, you’ll just have to make sure that the ZFS pool knows its UUID so that it knows it’s the same drive.
You need to add the new drive to your existing pool because ZFS stores data across all drives by default, similar to a RAID0. Then you remove the old drive and ZFS will automatically copy the data off the failing drive onto the healthy one and allow you to remove the failing drive with no data loss.
That depends on your use case, I just did a simple zpool with no redundancy because I wanted maximum speed/capacity and all my data is backed up on an external HDD. If you need redundancy, I would look online for how to configure that and what the optimal setup is.
Actually, I assumed you just had the SSD, if you have more than 256gb of free space between those HDDs, you can go ahead and remove the SSD from your zpool right now (unless your bootloader is there, then you’ll have to make an EFI system partition on one of the HDDs and install a bootloader first)
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
You could use something like Veeam Endpoint (free), it will image while the OS is running so the underlying ZFS system shouldn’t matter.
don’t mess around with imaging to a file on the zfs, then restoring it. simply clone nvme -> nvme using a usb nvme adapter then replace the internal with the clone.
I would like to avoid buying additional hardware if possible.
You can old nvme -> SATA -> new nvme with any old SATA drive you have lying around.
usb nvme adapters are not expensive and it likely won’t be the only time you need it. they are a handy accessory to have on hand if you have nvme storage.
I’ll definitely take this route if the wifi slot will not support the m.2 drive. Thanks for the suggestion.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
[Thread #758 for this sub, first seen 22nd May 2024, 23:15] [FAQ] [Full list] [Contact] [Source code]
If you want an image, it doesn’t matter what the underlying file system is. You should be able to use a tool like Clonezilla and get a 1:1 copy. Depending how you’ve set up partitioning, you could also use
sgdiskto set up the proper partitions andzfs send/recvfor the new data portion of the drive and install a boot loader. That’s probably the way I’d go in this instance.I get the part that the cloning software does not care for the underlying OS. My worry is the fact that I’ll run the cloning software/command from a live USB which will not be able to detect the zfs mirror on my backup drive on its own and thus break the zfs mirror with bad consequences for the existing data. I could not find any commands to make the live USB OS discover and respect the existing zfs configuration.
It sounds like nobody actually understood what you want.
You have a non-ZFS boot drive, and a big ZFS pool, and you want to save an image of the boot drive to the pool, as a backup for the boot drive.
I guess you don’t want to image the drive while booted off it, because that could produce an image that isn’t fully self-consistent. So then the problem is getting at the pool from something other than the system you have.
I think what you need to do is find something else you can boot that supports ZFS. I think the Ubuntu live images will do it. If not, you can try something like re-installing the setup you have, but onto a USB drive.
Then you have to boot to that and
zfs importyour pool. ZFS is pretty smart so it should just auto-detect the pool structure and where it wants to be mounted, and you can mount it. Don’t do a ZFS feature upgrade on the pool though, or the other system might not understand it. It’s also possible your live kernel might not have a new enough ZFS to understand the features your pool uses, and you might need to find a newer one.Then once the pool is mounted you should be able to
ddyour boot drive block device to a file on the pool.If you can’t get this to work, you can try using a non-ZFS-speaking live Linux and
dding your image to somewhere on the network big enough to hold it, which you may or may not have, and then booting the system and copying back from there to the pool.Phew! I almost believed I was asking for something beyond the scope of linux-fu. English not being my first language may be part of the reason but still I think I covered everything that was relevant.
Yes, that’s exactly what I want and your post has given me the clarity I needed. M.2 wifi slots don’t support disks so that option is definitely out. I’m going to boot with the latest Ubuntu live OS on a USB and attempt what you’ve outlined.
I don’t have anything really critical on the zfs that is not backed up separately so I’m definitely going to attempt this and learn in the process.
Thank you for taking the time to respond!
Use an Ubuntu live USB, all recent versions of Ubuntu have ZFS drivers baked into the live environment. Then you should add your new SSD to the ZFS pool, and remove the old one from the ZFS pool. Your m.2 WiFi slot should be able to host the 2nd drive while you do this, but if not you can use an external USB housing for it, you’ll just have to make sure that the ZFS pool knows its UUID so that it knows it’s the same drive.
Why do I need to add my nvme to the zfs pool? That doesn’t really make sense.
If the wifi slot does support m.2 drivers (I was just looking for some confirmation/document), it’ll solve my problem. Thanks nonetheless.
You need to add the new drive to your existing pool because ZFS stores data across all drives by default, similar to a RAID0. Then you remove the old drive and ZFS will automatically copy the data off the failing drive onto the healthy one and allow you to remove the failing drive with no data loss.
The SSD is 256GB while the two HDDs are 4TB each. What kind of zfs config/array do you suggest I create from them?
That depends on your use case, I just did a simple zpool with no redundancy because I wanted maximum speed/capacity and all my data is backed up on an external HDD. If you need redundancy, I would look online for how to configure that and what the optimal setup is.
Actually, I assumed you just had the SSD, if you have more than 256gb of free space between those HDDs, you can go ahead and remove the SSD from your zpool right now (unless your bootloader is there, then you’ll have to make an EFI system partition on one of the HDDs and install a bootloader first)
I think you want to see “zfs import” command, to get your pool back con the new OS.
Yes, the other answer also suggests this and I think this will do the trick. Thank you for your response.