I set up a new home server recently using containerized services, and I wanted to share what I learned. Nothing here is revolutionary, but this is the type of resource I wish I had when I started.
I’m open to feedback on what I could have done better!
Do you do some sort of versioning/snapshotting of your services? I’m on the compose route as well, and have one btrfs subvolume per service that holds the compose.yml and all bind-mounted folders for perstistent data. That again gets regularly snapshotted by snapper.
What leaves me a bit astounded is, that nobody seems to version the containers they are running. But without that, rolling back if something breaks might become a game of guessing the correct container version. I started building a tool that snapshots a service, then rewrites the image: in compose.yml to reflect what ever the current :latest tag resolves to. Surprisingly, there doesn’t seem to be an off-the-shelf solution for that…
I don’t do a great job of this, but take Immich for example. There, I specify the version in the compose.yml (technically, the version is in the .env file and substituted into the compose.yml). At that point, updating Immich is a matter of updating the version number and restarting the service.
These configuration files are all managed with git, so when I do these updates, I create a new commit. I just checked, and I have Forgejo pinned to a specific version in its compose.yml as well. But unfortunately, the other services are referencing :latest. I’m going to go back and pin them all :)
I built a small tool that does that for me now and published it: https://feddit.de/post/2909288 maybe you’ll find it useful, no guarantee that it doesn’t break something though :D
How do you do that? I’m building a similar system now that automatically updates my containers. I’ve played around with the API and I can see which versions are attached to the latest sha265, but I can’t find a way to automatically tell which version it is. Especially when the same sha is linked to multiple versions
I am familiar with tunnels in general, though I haven’t seen Cloudfare’s offering. I’m with the sibling comment that it’s another external dependency when my goal is to self host more. But that’s a tool I’ll keep in mind!
I’ll have to check out Caddy plugins! One I saw called the docker command to discover exposed ports, meaning it depends on Docker explicitly. I’ll see what I can dig up wrt Podman. This sounds very interesting. Thanks!
Not OP, i’ve heard nothing but good things about cloudflare tunnels but for me they have two major drawbacks. The first is you can’t use them for a self hosted media server such as jellyfin as it violates their terms of service. The second is you have to trust them with all your traffic. Now i have no reason to think they would do anything nefarious but i’m at the point in my threat model journey that the less i trust in any corporations hands the better. Just my two cents.
I find it disappointing that everyone’s first suggestion in the selfhosted community is Cloudflare. It seems to run counter to the spirit of selfhosting to hand off the last part to the giant corporation that controls 90% of the Internet.
Most of what Cloudflare does—if it’s necessary at all—can be replicated with a cheap (or free) VPS sitting in front of your network on a VPN, and the remainder doesn’t matter for most selfhosted use cases.
That’s a great point about Ansible. Compose automates most of the setup, but automating all of it would be amazing. I’ll try it with the next service I set up, and if it goes well, I’ll document it. Thanks for the suggestion!
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
4 acronyms in this thread; the most compressed thread commented on today has 11 acronyms.
[Thread #76 for this sub, first seen 23rd Aug 2023, 18:55] [FAQ] [Full list] [Contact] [Source code]
Do you do some sort of versioning/snapshotting of your services? I’m on the compose route as well, and have one btrfs subvolume per service that holds the compose.yml and all bind-mounted folders for perstistent data. That again gets regularly snapshotted by snapper.
What leaves me a bit astounded is, that nobody seems to version the containers they are running. But without that, rolling back if something breaks might become a game of guessing the correct container version. I started building a tool that snapshots a service, then rewrites the
image:in compose.yml to reflect what ever the current:latesttag resolves to. Surprisingly, there doesn’t seem to be an off-the-shelf solution for that…I don’t do a great job of this, but take Immich for example. There, I specify the version in the compose.yml (technically, the version is in the .env file and substituted into the compose.yml). At that point, updating Immich is a matter of updating the version number and restarting the service.
These configuration files are all managed with git, so when I do these updates, I create a new commit. I just checked, and I have Forgejo pinned to a specific version in its compose.yml as well. But unfortunately, the other services are referencing
:latest. I’m going to go back and pin them all :)I built a small tool that does that for me now and published it: https://feddit.de/post/2909288 maybe you’ll find it useful, no guarantee that it doesn’t break something though :D
How do you do that? I’m building a similar system now that automatically updates my containers. I’ve played around with the API and I can see which versions are attached to the latest sha265, but I can’t find a way to automatically tell which version it is. Especially when the same sha is linked to multiple versions
I only keep track of the sha256, compose happily uses those. I published the tool https://feddit.de/post/2909288 :)
Ah ok. I thought you meant the numbered version. I’m doing the same with sha256 too.
deleted by creator
I am familiar with tunnels in general, though I haven’t seen Cloudfare’s offering. I’m with the sibling comment that it’s another external dependency when my goal is to self host more. But that’s a tool I’ll keep in mind!
I’ll have to check out Caddy plugins! One I saw called the docker command to discover exposed ports, meaning it depends on Docker explicitly. I’ll see what I can dig up wrt Podman. This sounds very interesting. Thanks!
Not OP, i’ve heard nothing but good things about cloudflare tunnels but for me they have two major drawbacks. The first is you can’t use them for a self hosted media server such as jellyfin as it violates their terms of service. The second is you have to trust them with all your traffic. Now i have no reason to think they would do anything nefarious but i’m at the point in my threat model journey that the less i trust in any corporations hands the better. Just my two cents.
I find it disappointing that everyone’s first suggestion in the selfhosted community is Cloudflare. It seems to run counter to the spirit of selfhosting to hand off the last part to the giant corporation that controls 90% of the Internet.
Most of what Cloudflare does—if it’s necessary at all—can be replicated with a cheap (or free) VPS sitting in front of your network on a VPN, and the remainder doesn’t matter for most selfhosted use cases.
Great post, thanks for sharing 👍
I would suggest to give Ansible a try, it would make it really easy to deploy a new service with all required users and config.
That’s a great point about Ansible. Compose automates most of the setup, but automating all of it would be amazing. I’ll try it with the next service I set up, and if it goes well, I’ll document it. Thanks for the suggestion!
I am saving this post for the future, informative! Thanks!
Getting address not found.
I would recommend using Ansible to manage your containers and infrastructure in general. It has quite a steep learning curve, but it’s worth it!
Great read, thanks for sharing