I am currently running most of my stuff from an unraid box using spare parts I have. It seems like I am hitting my limit on it and just want to turn it into a NAS. Micro PCs/USFF are what I am planning on moving stuff to (probably a cluster of 2 for now but might expand later.). Just a few quick questions:
Running arr services on a proxmox cluster to download to a device on the same network. I don’t think there would be any problems but wanted to see what changes need to be done.
Which micro PCs are you running? I am leaving towards HP prodesk or Lenovo 7xx/9xx series around 200 each. I don’t really plan on getting more than 2-3 and don’t run too many things, but would want enough overhead if I switch stuff over to home assistant and windows and Linux VMs if needed.
Any best practices you recommend when starting a Proxmox cluster? I’ve learned over time it’s best to set it up correctly than try to fix stuff when it’s running. I wish I could coach myself from 7 years ago now. Would of saved a lot of headaches lol.
It’s 2024, avoid Proxmox and safe yourself a LOT of headaches down the line.
You most likely don’t need Proxmox and its pseudo-open-source bullshit. My suggestion is to simply with with Debian 12 + LXD/LXC, it runs VMs and containers very well. Proxmox ships with an old kernel that is so mangled and twisted that they shouldn’t even be calling it a Linux kernel. Also their management daemons and other internal shenanigans will delay your boot and crash your systems under certain circumstances.
What I would suggest you to use use instead is LXD/Incus.
LXD/Incus provides a management and automation layer that really makes things work smoothly - essentially what Proxmox does but properly done. With Incus you can create clusters, download, manage and create OS images, run backups and restores, bootstrap things with cloud-init, move containers and VMs between servers (even live sometimes).
Another big advantage is the fact that it provides a unified experience to deal with both containers and VMs, no need to learn two different tools / APIs as the same commands and options will be used to manage both. Even profiles defining storage, network resources and other policies can be shared and applied across both containers and VMs.
I draw your attention to containers (not docker), LXC containers because for most people full virtualization isn’t even required. In a small homelab if you can have containers that behave like full operating systems (minus the kernel) including persistence, VMs might not be required. Either way LXD/Incus will allow for both and you can easily mix and match and use what you require for each use case.
For eg. I virtualize the official HomeAssistant image with LXD because we all know how hard is to get that thing running, however my NAS / Samba shares are just a LXD Debian 12 container with Samba4, Nginx and FileBrowser. Sames goes for torrent client that has its own container. Some other service I’ve exposed to the internet also runs a full VM for isolation.
Like Proxmox, LXD/Incus isn’t about replacing existing virtualization techniques such as QEMU, KVM and libvirt, it is about augmenting them so they become easier to manage at scale and overall more efficient. I can guarantee you that most people running Proxmox today it today will eventually move to Incus and never look back. It woks way better, true open-source, no bugs, no BS licenses and way less overhead.
I’ve been on Proxmox for 6 or so months with very few issues and have found it to work well in my instance, I do appreciate seeing another alternative and learning about it too! I very specifically like Proxmox as it gives me an actual IP on my router’s subnet for my machines such as Home Assistant. So instead of the 192.168.122.1 it rolls a nice 192.168.1.X/24 IP which fits my range which makes it easier for me to direct my outside traffic to it. Does this also do this? Based on your screenshots, maybe not, IDK.
Thanks for the link! I’ve been running Proxmox for years now without any of the issues like the previous commenter mentioned. Not that they don’t exist, just that I haven’t hit them. I really like Proxmox but love hearing about alternatives. One day I might get bored and want to set things up new with a different stack and anything that’s more free/open is better in my book.
it gives me an actual IP on my router’s subnet for my machines
Yes you configure LXD/Incus’ networking to use a bridge and it will simply delegate the task to your router instead of proving IPs itself. One of my nodes actually runs the two setups at the same time, I’ve a bunch of containers on an internal range and then my Home Assistant VM getting an IP from my router.
I’ve no ideia if there’s a reasonable migration path but after running Proxmox for years I wouldn’t even want stuff that was tainted by it ever running on my pristine LXD nodes.
It’d be a pain in the rear to rebuild everything. This proxmox machine is the center of everything, even housing the disk all the config backups are on. I should probably not be doing that…
If you’re on a recent proxmox setup that uses LXC containers you might be able to export those containers using lxc-snapshot or some other method and move them to the LXD node… May work just fine, may require other adjustments.
Personally the move was worth it. I’m not gonna lie, a ton of my most complex solutions are setup using cloud-init and Ansible so moving from one solution to another was mostly running those again and watch the machines get re-created.
create clusters, download, manage and create OS images, run backups and restores, bootstrap things with cloud-init, move containers and VMs between servers (even live sometimes).
provides a unified experience to deal with both containers and VMs, no need to learn two different tools / APIs as the same commands and options will be used to manage both. Even profiles defining storage, network resources and other policies can be shared and applied across both containers and VMs.
I do regular snapshots of my containers live and sometimes restore them, no issues there. De-duplication and incremental features are (mostly) provided by the storage backend, if you use BTRFS or ZFS for your storage pool every container will be a volume that you can snapshot, rollback, export at any time. LXD also provides tools to make those operations: https://documentation.ubuntu.com/lxd/en/latest/howto/instances_backup/
Your comment is wrong in a few ways and suggests using a LXC which is way slower than docker or podman and lacks the easy setup.
Proxmox is good because it makes it easy to create VMs and setup least access. It also has as new of kernel as stable Debian so no, its not terribly out of date.
If you want to suggest that someone install Debian + Docker compose that would make more sense. This isn’t a good setup for more advanced setups and it doesn’t allow for a not of flexibility.
This was a discussion about management solutions such as Proxmox and LXD and NOT about containerization technologies like Docker or LXC. Also Proxmox uses the Proxmox VE Kernel that is derived from Ubuntu.
Your comment makes no sense whatsoever. I’m not even sure you know the difference between LXD and LXC…
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
It’s 2024, avoid Proxmox and safe yourself a LOT of headaches down the line.
You most likely don’t need Proxmox and its pseudo-open-source bullshit. My suggestion is to simply with with Debian 12 + LXD/LXC, it runs VMs and containers very well. Proxmox ships with an old kernel that is so mangled and twisted that they shouldn’t even be calling it a Linux kernel. Also their management daemons and other internal shenanigans will delay your boot and crash your systems under certain circumstances.
What I would suggest you to use use instead is LXD/Incus.
LXD/Incus provides a management and automation layer that really makes things work smoothly - essentially what Proxmox does but properly done. With Incus you can create clusters, download, manage and create OS images, run backups and restores, bootstrap things with cloud-init, move containers and VMs between servers (even live sometimes).
Another big advantage is the fact that it provides a unified experience to deal with both containers and VMs, no need to learn two different tools / APIs as the same commands and options will be used to manage both. Even profiles defining storage, network resources and other policies can be shared and applied across both containers and VMs.
I draw your attention to containers (not docker), LXC containers because for most people full virtualization isn’t even required. In a small homelab if you can have containers that behave like full operating systems (minus the kernel) including persistence, VMs might not be required. Either way LXD/Incus will allow for both and you can easily mix and match and use what you require for each use case.
For eg. I virtualize the official HomeAssistant image with LXD because we all know how hard is to get that thing running, however my NAS / Samba shares are just a LXD Debian 12 container with Samba4, Nginx and FileBrowser. Sames goes for torrent client that has its own container. Some other service I’ve exposed to the internet also runs a full VM for isolation.
Like Proxmox, LXD/Incus isn’t about replacing existing virtualization techniques such as QEMU, KVM and libvirt, it is about augmenting them so they become easier to manage at scale and overall more efficient. I can guarantee you that most people running Proxmox today it today will eventually move to Incus and never look back. It woks way better, true open-source, no bugs, no BS licenses and way less overhead.
Yes, there’s a WebUI for LXD as well!
Since you didn’t include a link to the source for your recommendation:
https://github.com/canonical/lxd
I’ve been on Proxmox for 6 or so months with very few issues and have found it to work well in my instance, I do appreciate seeing another alternative and learning about it too! I very specifically like Proxmox as it gives me an actual IP on my router’s subnet for my machines such as Home Assistant. So instead of the 192.168.122.1 it rolls a nice 192.168.1.X/24 IP which fits my range which makes it easier for me to direct my outside traffic to it. Does this also do this? Based on your screenshots, maybe not, IDK.
Thanks for the link! I’ve been running Proxmox for years now without any of the issues like the previous commenter mentioned. Not that they don’t exist, just that I haven’t hit them. I really like Proxmox but love hearing about alternatives. One day I might get bored and want to set things up new with a different stack and anything that’s more free/open is better in my book.
Yes you configure LXD/Incus’ networking to use a bridge and it will simply delegate the task to your router instead of proving IPs itself. One of my nodes actually runs the two setups at the same time, I’ve a bunch of containers on an internal range and then my Home Assistant VM getting an IP from my router.
I’m going to experiment with this! I would love to get rid of Proxmox, it has so many problems and I only run containers anyway.
Is there an easy way to migrate containers? I’m not well versed in LXC despite using it for years.
I’ve no ideia if there’s a reasonable migration path but after running Proxmox for years I wouldn’t even want stuff that was tainted by it ever running on my pristine LXD nodes.
It’d be a pain in the rear to rebuild everything. This proxmox machine is the center of everything, even housing the disk all the config backups are on. I should probably not be doing that…
If you’re on a recent proxmox setup that uses LXC containers you might be able to export those containers using
lxc-snapshotor some other method and move them to the LXD node… May work just fine, may require other adjustments.Personally the move was worth it. I’m not gonna lie, a ton of my most complex solutions are setup using cloud-init and Ansible so moving from one solution to another was mostly running those again and watch the machines get re-created.
I’ll look at lxc snapshots after the hardware upgrade I got lined up, thanks!
Can someone explain the benefits of LXD without the opinionated crap?
What else do you need.
How well does it handle backups, and are they deduplicated incremental ones like proxmox backup server makes?
I do regular snapshots of my containers live and sometimes restore them, no issues there. De-duplication and incremental features are (mostly) provided by the storage backend, if you use BTRFS or ZFS for your storage pool every container will be a volume that you can snapshot, rollback, export at any time. LXD also provides tools to make those operations: https://documentation.ubuntu.com/lxd/en/latest/howto/instances_backup/
That makes sense, but no remote backups over the network? Local snapshots I don’t really count as backups.
Your comment is wrong in a few ways and suggests using a LXC which is way slower than docker or podman and lacks the easy setup.
Proxmox is good because it makes it easy to create VMs and setup least access. It also has as new of kernel as stable Debian so no, its not terribly out of date.
If you want to suggest that someone install Debian + Docker compose that would make more sense. This isn’t a good setup for more advanced setups and it doesn’t allow for a not of flexibility.
This was a discussion about management solutions such as Proxmox and LXD and NOT about containerization technologies like Docker or LXC. Also Proxmox uses the Proxmox VE Kernel that is derived from Ubuntu.
Your comment makes no sense whatsoever. I’m not even sure you know the difference between LXD and LXC…