I’m running a ZFS pool of 4 external USB drives. It’s a mix of WD Elements and enclosed IronWolfs. I’m looking to consolidate it into a single box since I’m likely to add another 4 drives to it in the near future and dealing with 8 external drives could become a bit problematic in a few ways.
There’s been recurrent questions about ZFS with USB. Does it work? How does it work? Is it recommended and so on. The answer is complicated but it revolves around - yes it works and it can work well so long as you ensure that anything on your USB path is good. And that’s difficult since it’s not generally known what USB-SATA bridge chipset an external USB drive has, whether it’s got firmware bugs, whether it requires quirks, is it stable under sustained load etc. Then that difficulty is multiplied by the number of drives the system has. In my setup for example, I’ve swapped multiple enclosure models till I stumbled on a rock-solid one. I’ve also had to install heatsinks on the ASM1351 USB-SATA bridge ICs in the WD Elements drives to stop them from overheating and dropping dead under heavy load. With this in mind, if a multi-bay unit like the OWC Mercury Elite Pro Quad proves to be as reliable as some anecdotes say, it could become a go-to recommendation for USB DAS that eliminates a lot of those variables, leaving just the host side since it comes with a cable too. And the host side tends to be reliable since it’s typically either Intel or AMD. Read ##Testing for some tidbits about AMD.
hdparm -I /dev/sda
seems to return the original drive information. The disks appear with their internal designations in GNOME Disks. The kernel maps them in /dev/disks/by-id/*
according to those as before. I moved my drives in it, rebooted and ZFS started the pool as if nothing happenedsmartctl -x /dev/sda
hdparm
results in about 400MB/s total bandwidthApr 01 00:31:08 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 01 00:31:08 host kernel: usb 6-3.4: reset SuperSpeed USB device number 12 using xhci_hcd
Apr 01 00:31:08 host kernel: scsi host11: uas_eh_device_reset_handler success
Apr 01 00:32:42 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 01 00:32:42 host kernel: usb 6-3.4: reset SuperSpeed USB device number 12 using xhci_hcd
Apr 01 00:32:42 host kernel: scsi host11: uas_eh_device_reset_handler success
Apr 01 00:33:54 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 01 00:33:54 host kernel: usb 6-3.4: reset SuperSpeed USB device number 12 using xhci_hcd
Apr 01 00:33:54 host kernel: scsi host11: uas_eh_device_reset_handler success
Apr 01 00:35:07 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 01 00:35:07 host kernel: usb 6-3.4: reset SuperSpeed USB device number 12 using xhci_hcd
Apr 01 00:35:07 host kernel: scsi host11: uas_eh_device_reset_handler success
Apr 01 00:36:38 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 01 00:36:38 host kernel: usb 6-3.4: reset SuperSpeed USB device number 12 using xhci_hcd
Apr 01 00:36:38 host kernel: scsi host11: uas_eh_device_reset_handler success
It appears to be only related to the drive being resilvered. I did not observe resilver errors
iostat
shows numbers in-line with the 500MB/s of the the USB 3.1 Gen 1 port it’s connected to: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device
314.60 119.9M 95.2k 0.0k 599.4M 476.0k 0.0k sda
264.00 119.2M 92.0k 0.0k 595.9M 460.0k 0.0k sdb
411.00 119.9M 96.0k 0.0k 599.7M 480.0k 0.0k sdc
459.40 0.0k 120.0M 0.0k 0.0k 600.0M 0.0k sdd
Apr 02 16:13:47 host kernel: scsi host11: uas_eh_device_reset_handler start
Apr 02 16:13:47 host kernel: usb 6-2.4: reset SuperSpeed USB device number 9 using xhci_hcd
Apr 02 16:13:47 host kernel: scsi host11: uas_eh_device_reset_handler success
tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device
281.80 130.7M 63.2k 0.0k 653.6M 316.0k 0.0k sda
273.00 130.1M 56.8k 0.0k 650.7M 284.0k 0.0k sdb
353.60 130.8M 63.2k 0.0k 654.0M 316.0k 0.0k sdc
546.00 0.0k 133.2M 0.0k 0.0k 665.8M 0.0k sdd
The OWC passed all of the testing so far with flying colors. Even though resilver finished successfully, there were silent USB resets in the logs with the OWC connected to CPU-provided ports. Multiple ports exhibited the same behavior. When connected to a B350 chipset-provided port on the other hand the OWC finished two resilvers with no resets and faster, 18 hours vs 32 hours. My hypothesis is that these silent resers are likely related to the known USB problems with Ryzen CPUs. The OWC itself passed testing with flying colors when connected to a chipset port.
I’m buying another one for the new disks.
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
Exactly. You’re connecting this to a PC of some sort, right? So why not just put the disks in the case? The OCW on uk amazon is £440, which isn’t far off what I spent on my server build.
440 pounds is insane, agreed. 😂
Yeah I get it then. So it depends on whether one has PCIe slots available, 3.5" bays in the case, whether they can change the case if full, etc. It could totally make sense to do under certain conditions. In my case there’s no space in my PC case and I don’t have any PCIe slots left. In addition, I have an off-site machine that’s an USFF PC which has no PCIe slots or SATA ports. It’s only available connectivity is USB. So in my case USB is what I can work with. As long as it isn’t exorbitantly expensive, a USB solution has flexibility in this regard. I would have never paid 440 pounds for this if that was the price. I’d have stayed with single enclosures nailed to a wooden board and added a USB hub. 🥹 Which is how they used to be: