I’m looking into setting up some monitoring combined with simple automation for my selfhosting. Currently I was thinking about using Zabbix.
I want to:
Track bandwidth usage on a router/fw and on a managed switch and track cpu/ram/disk usage on my vms.
Simple monitoring (up/down/maintenance) on the router, switch, my vms as well as on linux services (jellyfin/forgejo/etc) and windows services (lab for studying work-related tools).
I’m also interested in doing simple https checks on my webuis (i’ve had a service running but the website returning both 403 and 404 before) and testing nslookup on my internal dns (if the service is up but the lookups timeout I still want to try restarting the service).
Is there any FOSS/FLOSS alternatives that I should look into before diving into Zabbix?
I used to use MQTT, static_status and Healthchecks.io, and have that data passed through to Home Assistant, but it started to get pretty cumbersome as the amount of machines I had grew.
I now use just Zabbix and HealthchecksIO. I did need to spend some time writing new templates for some additional data I wanted to collect (like SMART data for SSDs that provide health metrics in non-standard attributes, and HealthchecksIO so I could see the status of various checks on my zabbix dashboard)
Zabbix also has some additional features I found appealing, like proxies that can continue recording data when the main server is down, and built in encryption. Some checks like open ports/icmp responses etc can be checked using either the local agent, the remote server, or both, which helps quickly diagnose things like firewall config issues.
I did look at some other solutions, but I wanted something integrated to hit the ground running. Mobile apps are very limited, and there is no official one to my knowledge. I use Moobix which I don’t believe is FOSS - but I could be wrong there
Try each solution out and see what works best for you!
You know you can basically implement Healthchecks.io completely in Zabbix using zabbix-sender or any compatible implementation of it? (Or find a better way, e.g. querying the timestamp of a logfile or even check the logfile for “OK” or “ERROR” lines… lots of ways possible.)
For me it’s the other way around. In Check_MK I was constantly writing new custom checks and it was all manual code and overall felt like Nagios on steroids (what it was back then) - just not in a good way.
In Zabbix you can do everything in the UI without messing around in the file system. And things like translating SNMP results to readable text works throughout the system without having to include a Python file and then call it from within your various other checks. All the alerting logic can be clicked together and easily amended in the UI. It’s so much more comfortable once you’ve figured it out.
I use netdata (the FOSS agent only, not the cloud offering) on all my servers (physical, VMs…) and stream all metrics to a parent netdata instance. It works extremely well for me.
Other solutions are too cumbersome and heavy on maintenance for me. You can query netdata from prometheus/grafana [1] if you really need custom dashboards.
I guess you wouldn’t be able to install it on the router/switch but there is a SNMP collector which should be able to query bandwidth info from the network appliances.
Gonna check it out!
Is it easy to setup automatic responses to the alerts, f.e. restarting a service if it isn’t answering requests in a timely manner?
Have you used it together with Windows Servers too?
Prometheus/VictoriaMetrics/Grafana are pretty good, had no issues with it and there’s an exporter for damn near anything. They’re pretty easy to custom write too.
But these 3 are all about metrics, right? While they’re great to monitor and analyse numbers (ping times, disk space, memory, etc.), they aren’t that great with e.g. plaintext error messages in log files. That’s how I remember it from a few years ago, at least.
May have to explore this, I still run influxdb and telegraf for a push metrics operation instead of pull like prom. Things have been smooth for a while but a couple months ago disk temps and metrics stopped working with no errors or missing plugins
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
deleted by creator
I used to use MQTT, static_status and Healthchecks.io, and have that data passed through to Home Assistant, but it started to get pretty cumbersome as the amount of machines I had grew.
I now use just Zabbix and HealthchecksIO. I did need to spend some time writing new templates for some additional data I wanted to collect (like SMART data for SSDs that provide health metrics in non-standard attributes, and HealthchecksIO so I could see the status of various checks on my zabbix dashboard)
Zabbix also has some additional features I found appealing, like proxies that can continue recording data when the main server is down, and built in encryption. Some checks like open ports/icmp responses etc can be checked using either the local agent, the remote server, or both, which helps quickly diagnose things like firewall config issues.
I did look at some other solutions, but I wanted something integrated to hit the ground running. Mobile apps are very limited, and there is no official one to my knowledge. I use Moobix which I don’t believe is FOSS - but I could be wrong there
Try each solution out and see what works best for you!
You know you can basically implement Healthchecks.io completely in Zabbix using
zabbix-senderor any compatible implementation of it? (Or find a better way, e.g. querying the timestamp of a logfile or even check the logfile for “OK” or “ERROR” lines… lots of ways possible.)Uptime Kuma is great for simple up/down and web checks. Librenms is worth looking at too for other metrics.
I’ll have a look! Cheers!
I’ve used Zabbix for that before. I hope you like SNMP, though!
I’ve used SNMP a lot together with nagios so I should be able to handle it. :D
I’m using CheckMk for pretty much all of that. Personally I found zabbix to have too much overhead.
For me it’s the other way around. In Check_MK I was constantly writing new custom checks and it was all manual code and overall felt like Nagios on steroids (what it was back then) - just not in a good way.
In Zabbix you can do everything in the UI without messing around in the file system. And things like translating SNMP results to readable text works throughout the system without having to include a Python file and then call it from within your various other checks. All the alerting logic can be clicked together and easily amended in the UI. It’s so much more comfortable once you’ve figured it out.
Also uptime kuma for fast and easy up/down, web services, etc.
I’ve been using Zabbix for ages now. It has issues but I got used to it.
Zabbix still remains a good choice imo. It works fine with Grafana et now the Zabbix-grafana plugin is officially supported by Grafana.
Zabbix without Grafana is pretty weak in term of visualization.
I use netdata (the FOSS agent only, not the cloud offering) on all my servers (physical, VMs…) and stream all metrics to a parent netdata instance. It works extremely well for me.
Other solutions are too cumbersome and heavy on maintenance for me. You can query netdata from prometheus/grafana [1] if you really need custom dashboards.
I guess you wouldn’t be able to install it on the router/switch but there is a SNMP collector which should be able to query bandwidth info from the network appliances.
Gonna check it out!
Is it easy to setup automatic responses to the alerts, f.e. restarting a service if it isn’t answering requests in a timely manner?
Have you used it together with Windows Servers too?
No
It should be possible using
script to execute on alarm = /your/custom/remediation-scripthttps://learn.netdata.cloud/docs/alerts-&-notifications/notifications/agent-dispatched-notifications/agent-notifications-reference. I have not experimented with this yet, but soon will (implementing a custom notification channel for specific alarms)I’d rather find the root cause of the downtime/malfunction instead of blindly restarting the service, just my 2 cents.
Prometheus/VictoriaMetrics/Grafana are pretty good, had no issues with it and there’s an exporter for damn near anything. They’re pretty easy to custom write too.
But these 3 are all about metrics, right? While they’re great to monitor and analyse numbers (ping times, disk space, memory, etc.), they aren’t that great with e.g. plaintext error messages in log files. That’s how I remember it from a few years ago, at least.
Grafana/Loki does logs. Still early days for me but it’s solid so far.
Cheers! I’ve heard of Prometheus/Grafana but VictoriaMetrics was a new one. Gonna look into it!
Yeah VictoriaMetrics is the new favorite since Influx keeps reinventing their wheels and trying to move everyone to the cloud.
May have to explore this, I still run influxdb and telegraf for a push metrics operation instead of pull like prom. Things have been smooth for a while but a couple months ago disk temps and metrics stopped working with no errors or missing plugins