I don’t know when it started, I usually check my instance through the Voyager app and it never showed a problem, but I’m suddenly getting “server error” and 502 bad gateway errors on my instance on desktop. I haven’t made any major changes (read, any at all) since I last confirmed it worked on desktop.
Checking the logs I don’t see any obvious proxy issues but I am getting weird connection issues in lemmy-ui:
TypeError: fetch failed
lemmy-lemmy-ui-1 | at Object.fetch (node:internal/deps/undici/undici:11730:11)
lemmy-lemmy-ui-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
lemmy-lemmy-ui-1 | cause: ConnectTimeoutError: Connect Timeout Error
lemmy-lemmy-ui-1 | at onConnectTimeout (node:internal/deps/undici/undici:6869:28)
lemmy-lemmy-ui-1 | at node:internal/deps/undici/undici:6825:50
lemmy-lemmy-ui-1 | at Immediate._onImmediate (node:internal/deps/undici/undici:6857:13)
lemmy-lemmy-ui-1 | at process.processImmediate (node:internal/timers:478:21) {
lemmy-lemmy-ui-1 | code: 'UND_ERR_CONNECT_TIMEOUT'
lemmy-lemmy-ui-1 | }
lemmy-lemmy-ui-1 | }
I’ve had some similar issues in recent times and it largely seems to come down to timeouts when the containers are starting up amongst the other few dozen on the box. If I manually restart the lemmy containers after everything else is settled in it comes up fine.
So…it’s working now? I haven’t touched anything yet, but I just checked my instance again and it works perfectly fine on desktop now. It always worked through Voyager, so I was able to let people know there was an issue. If it comes back I’ll try some of these suggestions to find a more permanent fix.
This is an issue that has been present for a long time but recently came back worse with the 0.19 update. It is caused by lemmy-ui failing to load the site icon and it typically happens whenever the the docker container is started/restarted while you have a site icon set. The only fix at the moment is to manually delete the database entry which contains a link to the site image.
Once I get home from work this afternoon I will post my docker-compose.yml which has an extra container that automatically deletes the database entry then resets it.
EDIT: Here is the extra container from my docker-compose.yml
icon-fix:
image: iconfix:latest
command:
- /bin/sh
- -c
- |
sleep 10
# Remove Site Icon - be sure to set your postgres password
PGPASSWORD=<POSTGRESS PASSWORD HERE> psql -U lemmy -h postgres -d lemmy -c "UPDATE site SET icon = NULL WHERE id = 1;"
# Refresh Site - replace example.com with your sites domain
curl -f -sS -H "Host: example.com" http://lemmy-ui:1234 > temp.html
# Reset Site Icon - Set your postgres password and replace the URL with one that points to your icon
PGPASSWORD=<POSTGRESS PASSWORD HERE> psql -U lemmy -h postgres -d lemmy -c "UPDATE site SET icon = 'https://example.com/pictrs/image/2cc85182-5739-4c86-b982-94fc913e80d3.webp' WHERE id = 1;"
depends_on:
- postgres
And here is the Dockerfile for building the iconfix image
FROM docker.io/postgres:latest
RUN apt update
RUN apt install -y curl
I would start at the github repo and check if that issue has been documented.
If yes, follow the instructions. If not, check your dns since 502 often came from dns in my case.
If both doesnt reveal anything, you could open an issue in the repo and post your (sanitized) logs and wait for answers.
The error suggests a problem with the lemmy-lemmy-ui-1 container. Maybe it needs an update or has pulled a wrong update. When did you update last? Did you try restarting the stack?
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
I’ve had some similar issues in recent times and it largely seems to come down to timeouts when the containers are starting up amongst the other few dozen on the box. If I manually restart the lemmy containers after everything else is settled in it comes up fine.
So…it’s working now? I haven’t touched anything yet, but I just checked my instance again and it works perfectly fine on desktop now. It always worked through Voyager, so I was able to let people know there was an issue. If it comes back I’ll try some of these suggestions to find a more permanent fix.
deleted by creator
This is an issue that has been present for a long time but recently came back worse with the 0.19 update. It is caused by lemmy-ui failing to load the site icon and it typically happens whenever the the docker container is started/restarted while you have a site icon set. The only fix at the moment is to manually delete the database entry which contains a link to the site image.
Check this GitHub link for instructions. https://github.com/LemmyNet/lemmy-ui/issues/1530#issuecomment-1605781461
Once I get home from work this afternoon I will post my docker-compose.yml which has an extra container that automatically deletes the database entry then resets it.
EDIT: Here is the extra container from my docker-compose.yml
And here is the Dockerfile for building the
iconfiximageI would start at the github repo and check if that issue has been documented.
If yes, follow the instructions. If not, check your dns since 502 often came from dns in my case.
If both doesnt reveal anything, you could open an issue in the repo and post your (sanitized) logs and wait for answers.
The error suggests a problem with the lemmy-lemmy-ui-1 container. Maybe it needs an update or has pulled a wrong update. When did you update last? Did you try restarting the stack?
Good luck.
If the error is with the UI, then trying a mobile app is a good place to start (since they should connect to the API directly).