Today I decided to get an inexpensive custom domain from Namecheap and try self-hosting Lemmy. A few bucks later I was thinking, “Hey, this is going to be cake.”
I’d read some of the warnings about Oracle Cloud free tier, but figured I’d still give it a shot for hosting. I found a simple how-to for quickly getting an Ubuntu instance spun up with Docker and Portainer. A few minutes later I’m thinking, “This is so easy!”
Then I try to access Portainer using HTTPS and see my first “Your connection is not private,” warning. “No worries,” I think. “Advanced>Proceed. I’m in.”
So I run Lemmy Easy Deploy. “The lights are green, the trap is clean! Boom. Here we go!”
Nothing.
Ports seem to be open on Oracle, but no Lemmy at either 80 or 443.
“Maybe Lemmy is more particular about SSL certificates and such?” I think, for the first time getting worried.
"Err, I think that if I change my nameserver to Cloudflare I can destroy my Lemmy containers, re-run Lemmy Easy Deploy with a Cloudflare API token, and maybe fix it?
Four hours later, after repeatedly starting over, clearing my browser cache every 5 minutes, switching back and forth between nameservers, even deleting the whole Oracle Cloud VM and starting from scratch, I realize that an HTTP connection to port 443 is returning “Client sent an HTTP request to an HTTPS server.”
“Were you there before, message?” I wonder.
Lemmy friends, can you help me? Or am I better off just deleting the VM and giving up the whole idea?
This series of commands installed an official docker repository in your system so docker itself is always up-to-date, Ubuntu gets updates like this kinda slowly.
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
apt update
apt install caddy
Edit the file /etc/caddy/Caddyfile (for example using nano /etc/caddy/Caddyfile)
Paste this content (replace domain.tld with your Lemmy instance domain and the 1236 with the port you have configured in the docker-compose.yml file for {{ lemmy_port }}):
No, most browsers will do what they’re told. At least with nginx, you have to set up a server to listen on port 80 and send a 301 permanently redirected message with the https URL.
Ah, maybe I wasn’t clear enough, caddy doesn’t need a configuration to redirect traffic from port 80 to port 443, it does it automatically: https://caddyserver.com/docs/automatic-https
If you want to configure an http only site you need to specifically configure it in your Caddyfile.
Here’s an example for a reverse proxy
# We don't specify the protocol so Caddy assumes HTTPS and redirects any HTTP to secure.
secure.example.com { reverse_proxy :<port> }
# HTTP is specified so Caddy won't redirect to port 443 nor generate a certificate.
http://insecure.example.com { reverse_proxy :<port> }
I just went to copy the error message I saw before and… it’s working.
Maybe because I switched back to Namecheap’s nameserver? Or maybe because I cleared my cache again? Or maybe because I game it some more propagation time?
That was my exact experience too when I first started out.
Things that should have worked didn’t, and things that shouldn’t have worked did
And then eventually the things that should have worked started working, with no clear indication of why they weren’t working, or why they suddenly started working.
Edit: **this will make your oci instance less secure **and will break integrations with other oci services. Do not use this in production, but ONLY for testing if the host fw rules affect your app.
I’m currently using oraclecloud for my bots. I work in the space (cloud/systems engg) and the first thing that got me was that the oracle ubuntu instances have custom iptables in place for security.
I’m not sure if it still has, but last i checked a year ago I had to flush iptables before I was able to use other ports. I didint really want to deal with another layer of security to manage as I was just using the arm servers for my hobby.
It might be something worth checking, it isn’t specific to lemmy though.
I found it unintuitive because other major cloud providers do not have any host firewall/security in place (making it easier to manage security using SG/NACL, through the console).
That’s not really the right approach on OCI, unfortunately: if you just flush the rules you also break a lot of their management plane.
You’d want to modify the /etc/iptables/rules.v4 and rules.v6 files to add any rules you want to load on boot (and, of course, if you just flush the rules without saving them, then it won’t persist and a reboot will break things, again).
It’s an arguable benefit: I’m a fan of having the security policies AND iptables sitting between me and doing something stupid, but I also spent most of the last decade dealing with literally thousands and thousands of compromised hosts that just whoopsie oopsed redis/jenkins/their database/a ftp service in a publicly accessible state, got hacked, then had the customer come crying to us asking why we didn’t keep them from blowing their foot off - which, basically, is what the OCI defaults do.
I agree with this, what I suggested is not a best practice, I should preface my post with that.
And I feel your pain! I get calls that are extremes, like people putting too much security where the ticket is “P1 everything is down, fly every engineer here” for an nACL/SG they created.
The other extreme is that deliberate exposure of services to the public internet (other service providers send us an email and ask us to do something about it, but not our monkeys, shared responsibility, etc).
Yeah, I just mentioned it because OCI is kinda wonky and requires some static routing stuff in the iptables on the host to have the platform work as intended (which, as far as I’m aware, no other hyperscaler does), which strikes me as really really lazy engineering, but I’m just a simple computer janitor so maybe I’m wrong there.
The most infuriating thing at my last job was people sending in a ticket freaked out that their database was stolen and ransomed, and us going ‘Well, we sent you 15 emails over the last 3 months telling you that you had the database open and improperly secured, so what exactly are you wanting us to do now?’
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
First post – It worked!
What tutorial have you followed? I followed the official one with docker and it worked flawlessly.
Lemmy Easy Deploy. I didn’t know where to find any tutorials for using an Oracle Cloud VM. Did the official have that?
I assume the VM is some kind of Linux server? Ot is there something specific to Oracle Cloud?
Yes, the VM is Ubuntu 22.04.
Edit: Replied out of context. Fixed.
Then it should work with this: https://join-lemmy.org/docs/administration/install_docker.html
Here are the commands, basically:
Installing docker
From this doc: https://docs.docker.com/engine/install/ubuntu/
for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; doneapt-get updateapt-get install ca-certificates curl gnupginstall -m 0755 -d /etc/apt/keyringscurl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpgchmod a+r /etc/apt/keyrings/docker.gpgecho "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | tee /etc/apt/sources.list.d/docker.list > /dev/nullapt-get updateapt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-composeThis series of commands installed an official docker repository in your system so docker itself is always up-to-date, Ubuntu gets updates like this kinda slowly.
Installing Lemmy
From this doc: https://join-lemmy.org/docs/administration/install_docker.html
mkdir /opt/lemmycd /opt/lemmywget https://raw.githubusercontent.com/LemmyNet/lemmy-ansible/main/templates/docker-compose.ymlwget https://raw.githubusercontent.com/LemmyNet/lemmy-ansible/main/examples/config.hjson -O lemmy.hjsonwget https://raw.githubusercontent.com/LemmyNet/lemmy-ansible/main/templates/nginx_internal.confwget https://raw.githubusercontent.com/LemmyNet/lemmy-ansible/main/examples/customPostgresql.confnano) and change everything that starts with{{and ends with}}mkdir -p volumes/pictrschown -R 991:991 volumes/pictrsdocker-compose up -dInstalling webserver
I chose Caddy, you can choose a different one but then you’ll have to check on your own.
From this guide: https://caddyserver.com/docs/install#debian-ubuntu-raspbian
apt install -y debian-keyring debian-archive-keyring apt-transport-httpscurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpgcurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.listapt updateapt install caddy/etc/caddy/Caddyfile(for example usingnano /etc/caddy/Caddyfile)domain.tldwith your Lemmy instance domain and the1236with the port you have configured in the docker-compose.yml file for{{ lemmy_port }}):Hope I haven’t forgotten anything, feel free to ask.
Edit: After all the commands, restart caddy with
service caddy restart.Trying do https://YOUR-DOMAIN-HERE
yea it might just be missing a proper http -> https redirect, not sure how to do that in Caddy cause I only know nginx.
Caddy auto redirects to https as long as you don’t add “http://” to the domain.
That sounds more like your browser, not caddy.
No, most browsers will do what they’re told. At least with nginx, you have to set up a server to listen on port 80 and send a 301 permanently redirected message with the https URL.
Ah, maybe I wasn’t clear enough, caddy doesn’t need a configuration to redirect traffic from port 80 to port 443, it does it automatically: https://caddyserver.com/docs/automatic-https
If you want to configure an http only site you need to specifically configure it in your Caddyfile.
Here’s an example for a reverse proxy
Yeah, I figured that’s what you meant after rereading it. Chrome at least does automatically switch to HTTPS in some cases, like where it’s seen HSTS.
Now I feel dumb.
That didn’t work earlier.
I just went to copy the error message I saw before and… it’s working.
Maybe because I switched back to Namecheap’s nameserver? Or maybe because I cleared my cache again? Or maybe because I game it some more propagation time?
Or maybe magic?
Each potential reason seems equally likely to me.
Thanks
DNS can take 4 to 24 hours to replicate out. Next time, you can adjust you hosts file temporarily to test while it does it’s thing.
Welcome to the world of web administration
That was my exact experience too when I first started out.
Things that should have worked didn’t, and things that shouldn’t have worked did
And then eventually the things that should have worked started working, with no clear indication of why they weren’t working, or why they suddenly started working.
All I can say is that it gets easier with time
Edit: **this will make your oci instance less secure **and will break integrations with other oci services. Do not use this in production, but ONLY for testing if the host fw rules affect your app.
I’m currently using oraclecloud for my bots. I work in the space (cloud/systems engg) and the first thing that got me was that the oracle ubuntu instances have custom iptables in place for security.
I’m not sure if it still has, but last i checked a year ago I had to flush iptables before I was able to use other ports. I didint really want to deal with another layer of security to manage as I was just using the arm servers for my hobby.
It might be something worth checking, it isn’t specific to lemmy though.
I found it unintuitive because other major cloud providers do not have any host firewall/security in place (making it easier to manage security using SG/NACL, through the console).
That’s not really the right approach on OCI, unfortunately: if you just flush the rules you also break a lot of their management plane.
You’d want to modify the /etc/iptables/rules.v4 and rules.v6 files to add any rules you want to load on boot (and, of course, if you just flush the rules without saving them, then it won’t persist and a reboot will break things, again).
It’s an arguable benefit: I’m a fan of having the security policies AND iptables sitting between me and doing something stupid, but I also spent most of the last decade dealing with literally thousands and thousands of compromised hosts that just whoopsie oopsed redis/jenkins/their database/a ftp service in a publicly accessible state, got hacked, then had the customer come crying to us asking why we didn’t keep them from blowing their foot off - which, basically, is what the OCI defaults do.
I agree with this, what I suggested is not a best practice, I should preface my post with that.
And I feel your pain! I get calls that are extremes, like people putting too much security where the ticket is “P1 everything is down, fly every engineer here” for an nACL/SG they created.
The other extreme is that deliberate exposure of services to the public internet (other service providers send us an email and ask us to do something about it, but not our monkeys, shared responsibility, etc).
Yeah, I just mentioned it because OCI is kinda wonky and requires some static routing stuff in the iptables on the host to have the platform work as intended (which, as far as I’m aware, no other hyperscaler does), which strikes me as really really lazy engineering, but I’m just a simple computer janitor so maybe I’m wrong there.
The most infuriating thing at my last job was people sending in a ticket freaked out that their database was stolen and ransomed, and us going ‘Well, we sent you 15 emails over the last 3 months telling you that you had the database open and improperly secured, so what exactly are you wanting us to do now?’
Good tip. Thanks!