Basically have watchtower monitor and update containers whenever new images are released. I’ve recently noticed that with searxng (using redis as db), hosted through nginx proxy manager, will have a steady downtime of about 15mins post update then come back online.
This is extremely frequent for searxng’s case as I have watchtower run every day and my preferred way of running most of my containers is with the latest tag. The way out of this downtime in my experience is a restart of NPM which brings back the searxng service.
I’m looking for a way to automate a restart of the NPM container after a successful update of searxng’s container.
I have checked the docs for watchtower, and the lifecycle hooks (a way to run sh scripts pre/post update) are able to run only from the applied container and not from the host system.
Sounds like trying to use a rolled up newspaper to kill a fly. You can do it with janky restarts of your reverse proxy container, which results in down time of all apps, or you’d wait some long period of time for it to detect the change and rewire itself.
I’d recommend reaching for an electric bug slapper instead. Use something like traefik, where you can allow it to connect to your docker socket (just like watchtower), and automatically wire up the new container’s reverse proxy when it comes back online.
Sounds like trying to use a rolled up newspaper to kill a fly.
Isn’t this what tech/sysadmin solutions are all about now? 😆
You can do it with janky restarts of your reverse proxy container, which results in down time of all apps
Yet to see if this solution breaks up my other services but it currently fixes the downtime I’ve recently been having with the searxng docker + NPM setup.
Use something like traefik, where you can allow it to connect to your docker socket (just like watchtower), and automatically wire up the new container’s reverse proxy when it comes back online.
NPM covers most of my needs plus traefik doesn’t work for me (due to other reasons unrelated to the tech).
Yeah. We reach for what works, and sometimes they’re great, other times there may be better solutions. When a reverse proxy container restarts, there will be a brief disconnect/outage of everything it is proxying until it comes back. I think I’m this case there are other solutions available, but I trust you’ve also weighted the pros and cons of your decision. Hope you find the right tool that meets your objectives!
Either use depends_on or think of a health check and use Will Farrell’s simple Docker Autoheal container that restarts containers when they become unhealthy.
https://github.com/willfarrell/docker-autoheal
This will do exactly what you want: You have to configure a healthcheck for searnxg that detects when it’s down. Maybe something with curl or whatever.
As soon as it’s down autoheal will restart the container. Doesn’t matter why it is down (update, dependency not running, …) autoheal will just restart the container.
Ahhhh… alright, I misunderstood. So either depends_on is your friend or you could implement a rather dirty solution: Write a little script for the NPM healthcheck that also checks if searxng is online. Then use autoheal.
But that would be my last solution and only if the searxng is very closely depending on the npm container.
No worries. I ended up writing a systemd service that restarts the npm container a minute after the watchtower runs. I don’t know why I didn’t think of that earlier but I guess it’s a much simpler solution than using depends_on with npm which hosts a lot of my other services.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
Not sure if this will work in your use case but I use OliveTin for automating docket commands. Link here https://github.com/OliveTin/OliveTin
Doesn’t. Not automated.
Sounds like trying to use a rolled up newspaper to kill a fly. You can do it with janky restarts of your reverse proxy container, which results in down time of all apps, or you’d wait some long period of time for it to detect the change and rewire itself.
I’d recommend reaching for an electric bug slapper instead. Use something like traefik, where you can allow it to connect to your docker socket (just like watchtower), and automatically wire up the new container’s reverse proxy when it comes back online.
Isn’t this what tech/sysadmin solutions are all about now? 😆
Yet to see if this solution breaks up my other services but it currently fixes the downtime I’ve recently been having with the searxng docker + NPM setup.
NPM covers most of my needs plus traefik doesn’t work for me (due to other reasons unrelated to the tech).
Yeah. We reach for what works, and sometimes they’re great, other times there may be better solutions. When a reverse proxy container restarts, there will be a brief disconnect/outage of everything it is proxying until it comes back. I think I’m this case there are other solutions available, but I trust you’ve also weighted the pros and cons of your decision. Hope you find the right tool that meets your objectives!
Either use depends_on or think of a health check and use Will Farrell’s simple Docker Autoheal container that restarts containers when they become unhealthy. https://github.com/willfarrell/docker-autoheal
Doesn’t seem like this will do what I want.
This will do exactly what you want: You have to configure a healthcheck for searnxg that detects when it’s down. Maybe something with curl or whatever.
As soon as it’s down autoheal will restart the container. Doesn’t matter why it is down (update, dependency not running, …) autoheal will just restart the container.
I want to restart the NPM container (when the searxng service goes down) instead of searxng.
Ahhhh… alright, I misunderstood. So either depends_on is your friend or you could implement a rather dirty solution: Write a little script for the NPM healthcheck that also checks if searxng is online. Then use autoheal.
But that would be my last solution and only if the searxng is very closely depending on the npm container.
No worries. I ended up writing a systemd service that restarts the npm container a minute after the watchtower runs. I don’t know why I didn’t think of that earlier but I guess it’s a much simpler solution than using
depends_onwith npm which hosts a lot of my other services.