I have a server configuration to what i though would be best, and that is running a Debian, then installing a service i most frequently use, and use containers for other services.
But, now i think that’s not a good solution and i’m looking for advice.
I thought of something like this:
Proxmox install
Spin up a VM for this service that is currently running on Debian (can’t be in a container)
Spin up a second VM, install Debian and Docker and install all other services as containers.
That would enable me to:
a) backup the 1st VM to be able to deploy it if needed (backups)
b) backup containers in the second VM so i can have them ready to be restored if needed
However, i’m not sure about setting it up like this.
I’m worried if Jellyfin will work good as a container on a VM.
Also, i’m worried about setting up nginx in a container on a VM, like, will it work as if installed on bare metal.
Other services i’m planning to run in containers on that 2nd VM are BookStack, Joplin, Mosquitto broker, Grafana, MariaDB, Influx DB, Studio Code, JellyFin, NectCloud etc.
The machine is a i3 1315U, 16GB RAM, 500GB SSD.
For the first VM i would allocate 2 cores and 4GB of RAM (that’s enough for what it does) and for the second VM (with all the containers) i would allocate the rest of the CPU and RAM.
Any advice is very welcome!
Is proxmox still the best choice?
Are there any other (better) choices?
Is something obviously wrong with this setup?
Docker in a lxc container is also used quite a lot with proxmox and would allow you to keep some resources without allocating everything for a docker VM.
In your scenario I don’t think I’d use Proxmox as you’re going to run into issues with lack of RAM. I think you’re going to have issues running out of memory either way though. Running the whole machine as a Docker node would probably be more memory-efficient than having the overhead of running separate VMs under Proxmox.
NGINX should run fine as a container. There’s even an official build available on Docker Hub.
I totally missed that you have an uncontainerized service. Can you run the service directly on the hardware host (safely)? If so, here’s how I would probably run it considering your memory constraints:
Install Debian
Set up the uncontainerized service directly on the host.
Install Docker directly on the host as well.
Spin up the containers for everything else.
Not the cleanest/most separated answer but it would reduce the memory load of additional layers of host/VM/containers. If this isn’t storing any sensitive data or being directly exposed to the internet that should be fine.
If you are dealing with sensitive data or exposing to the internet, I would consider your original plan of Proxmox VMs to separate everything but see if you can add additional RAM to help. Also consider installing something like fail2ban on every host and VM.
That is also a fine approach, didn’t think of that.
I never worked with Proxmox, and had no idea i can run containers directly on it.
How would restoring a container work in that case?
Example, i purchase another machine, install fresh proxmox on it, can i simply restore the containers from the original machine without any additional configuration?
The containers in Proxmox (LXC Containers) are a little different from Docker containers. You can’t deploy Docker containers directly as LXC containers. You can, however, run an LXC container and install Docker on it, then run Docker containers there.
Yeah, that will work fine! I’ve a similar setup and it works fine. 2 VM’s for stuff that needs a VM and a bunch of docker containers in a separate VM.
And your Nginx will work fine in Docker. Set it up on a random port and route from the modem/router to that random port and from there to your VM, so something like 443 on modem goes to port 8443 on the ip of the VM running docker.
It also gives you the possibility to later on add a second server with Proxmox, put them both in a cluster so you can easily move one of your VM’s to a second node.
Final advice is that Tuxis is offering 150GB of free Proxmox backup service. So you can use that for some important VM’s to be stored off site for free (encrypted of course) with full support within your Proxmox environment to create or restore backups (or even restore some files from inside the VM). See https://www.tuxis.nl/en/ordering/?case=PBS and https://www.proxmox.com/en/proxmox-backup-server/overview
Thanks for the input.
Do you think i will be running out of RAM in that configuration, like other commenters noted?
The reason I’d like to have two separate VMs is easier backups/restores, that way i don’t have to care about the phisycal machine, if i want to move to something else i only have to restore the VM.
As for the backups, i have one local backup on a separate machine (NAS) that gets backed up to an external drive, then another dedicated backup NAS that backups the first NAS and is otherwise disconnected from the internet, local network and power (turns on only once a week to backup), then another backup that backups the backup NAS to an off-site NAS, that also has an external drive making daily backups. Is that ok?
You would have 12 GB ram shared over all the dockers. I think you will be fine. Unless everything will be used intensively continuously. But that’s my opinion. Just give it a shot, nothing to loose. Promox itself does not take so much. So if it it does not run in this setup you need different hardware anyhow.
I don’t like the solution of running docker next to proxmox, not in a vm, you want proxmox to respond even if the docker vm is busy/overloaded.
In terms of backup you should be good. I would skip that weekly local backup construction, not sure what that adds if the off site backup is working reliable. I’ld format that one and add proxmox to it and make proper use of it (like a second docker vm)
I’m using 2GB RAM at this moment, not accounting for Jellyfin and Nextcloud, and i don’t have info about their load because they’re on a windows server.
That’s all running bare metal.
The offsite NAS is at my office, and is serving my office needs daily, i just added a backup of my home server to it.
Do you have any idea how much cores/ram should i leave to Proxmox?
None. Just use everything in the VM’s. Yes that’s over committing, but who cares if the lead normally is reasonable and you can watch the ’summary’ in pm to see how the system is doing. Stop worrying, grab the proxmox iso and have fun
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
3 acronyms in this thread; the most compressed thread commented on today has 11 acronyms.
[Thread #95 for this sub, first seen 31st Aug 2023, 10:55] [FAQ] [Full list] [Contact] [Source code]
To setup proxmox, you could install it on top of your current debian install : https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullseye
Docker in a lxc container is also used quite a lot with proxmox and would allow you to keep some resources without allocating everything for a docker VM.
Something like this?
https://benheater.com/proxmox-run-docker-on-linux-containers-lxc/
I do exactly that, enable nesting in the lxc container, install docker and go from there. It works great
Alright, i’ll check it out, thanks!
In your scenario I don’t think I’d use Proxmox as you’re going to run into issues with lack of RAM. I think you’re going to have issues running out of memory either way though. Running the whole machine as a Docker node would probably be more memory-efficient than having the overhead of running separate VMs under Proxmox.
NGINX should run fine as a container. There’s even an official build available on Docker Hub.
Am i getting this right:
Install Debian, setup a VM for the service (2gb, 2 cores) Install Docker on the same Debian OS, without virtualization, deploy containers?
I totally missed that you have an uncontainerized service. Can you run the service directly on the hardware host (safely)? If so, here’s how I would probably run it considering your memory constraints:
Not the cleanest/most separated answer but it would reduce the memory load of additional layers of host/VM/containers. If this isn’t storing any sensitive data or being directly exposed to the internet that should be fine.
If you are dealing with sensitive data or exposing to the internet, I would consider your original plan of Proxmox VMs to separate everything but see if you can add additional RAM to help. Also consider installing something like fail2ban on every host and VM.
I think you should be able to run containers directly on Proxmox, and have enough RAM to run one VM for the thing that can’t be containerized.
That is also a fine approach, didn’t think of that. I never worked with Proxmox, and had no idea i can run containers directly on it. How would restoring a container work in that case?
Example, i purchase another machine, install fresh proxmox on it, can i simply restore the containers from the original machine without any additional configuration?
The containers in Proxmox (LXC Containers) are a little different from Docker containers. You can’t deploy Docker containers directly as LXC containers. You can, however, run an LXC container and install Docker on it, then run Docker containers there.
Yeah, that will work fine! I’ve a similar setup and it works fine. 2 VM’s for stuff that needs a VM and a bunch of docker containers in a separate VM.
And your Nginx will work fine in Docker. Set it up on a random port and route from the modem/router to that random port and from there to your VM, so something like 443 on modem goes to port 8443 on the ip of the VM running docker.
It also gives you the possibility to later on add a second server with Proxmox, put them both in a cluster so you can easily move one of your VM’s to a second node.
Final advice is that Tuxis is offering 150GB of free Proxmox backup service. So you can use that for some important VM’s to be stored off site for free (encrypted of course) with full support within your Proxmox environment to create or restore backups (or even restore some files from inside the VM). See https://www.tuxis.nl/en/ordering/?case=PBS and https://www.proxmox.com/en/proxmox-backup-server/overview
Thanks for the input. Do you think i will be running out of RAM in that configuration, like other commenters noted?
The reason I’d like to have two separate VMs is easier backups/restores, that way i don’t have to care about the phisycal machine, if i want to move to something else i only have to restore the VM.
As for the backups, i have one local backup on a separate machine (NAS) that gets backed up to an external drive, then another dedicated backup NAS that backups the first NAS and is otherwise disconnected from the internet, local network and power (turns on only once a week to backup), then another backup that backups the backup NAS to an off-site NAS, that also has an external drive making daily backups. Is that ok?
You would have 12 GB ram shared over all the dockers. I think you will be fine. Unless everything will be used intensively continuously. But that’s my opinion. Just give it a shot, nothing to loose. Promox itself does not take so much. So if it it does not run in this setup you need different hardware anyhow.
I don’t like the solution of running docker next to proxmox, not in a vm, you want proxmox to respond even if the docker vm is busy/overloaded.
In terms of backup you should be good. I would skip that weekly local backup construction, not sure what that adds if the off site backup is working reliable. I’ld format that one and add proxmox to it and make proper use of it (like a second docker vm)
I’m using 2GB RAM at this moment, not accounting for Jellyfin and Nextcloud, and i don’t have info about their load because they’re on a windows server. That’s all running bare metal.
The offsite NAS is at my office, and is serving my office needs daily, i just added a backup of my home server to it.
Do you have any idea how much cores/ram should i leave to Proxmox?
None. Just use everything in the VM’s. Yes that’s over committing, but who cares if the lead normally is reasonable and you can watch the ’summary’ in pm to see how the system is doing. Stop worrying, grab the proxmox iso and have fun
Alright, yeah, i tend to overthink stuff to the point of not actually doing the thing i wanted. Thanks for the push!