Hi everyone,
I host a server with few services running for my family and friends. The server freezes sometimes. Where the services stop responding. If I try to ssh into the server, it takes a lot of time (3 to 5 mins) for it to connect. After being connected, every letter I type takes 2 to 3 mins to appear on screen.
Seems like the server is overloaded with tasks. But I am not sure what load it is running. I have netdata installed. I could pull up following screenshots for insights.
Can someone please help me in troubleshooting the issue?
I have tried testing stress testing my RAM and CPU, and they were fine. But I would start troubleshooting from scratch if you have recommendations for testings softwares.
Please also let me know if there is anything I can pull from netdata to help in trouble shooting.
It depends whether the problem is that you don’t have enough RAM, or something is using more RAM than it should. In my experience it’s almost always the latter.
Also, does High IO wait time indicate issue with the boot drive (which is an SSD that is 4 years old)
No, it means the CPU is waiting for disk I/O to complete before it can work on tasks. When available RAM is low, pages get swapped out to disk and need to be swapped back in before the CPU can use them. It could also be an application that’s reading and writing a huge amount from/to disk or the network, but given the high memory usage I’d start looking there.
Okay. Thanks for the insight.
Planning to do this:
You need to monitor which process is using all the memory. The easiest way is probably to keep htop running in a screen or tmux session, periodically connect, and look at which processes have the highest used memory.
Will report back with findings when it crashes again. Thanks for all the help.
It’ll probably be obvious before it crashes, you can see in the graphs that the “used” memory is increasing steadily after a reboot. Take a look now and see which process is causing that.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
It depends whether the problem is that you don’t have enough RAM, or something is using more RAM than it should. In my experience it’s almost always the latter.
No, it means the CPU is waiting for disk I/O to complete before it can work on tasks. When available RAM is low, pages get swapped out to disk and need to be swapped back in before the CPU can use them. It could also be an application that’s reading and writing a huge amount from/to disk or the network, but given the high memory usage I’d start looking there.
Okay. Thanks for the insight. Planning to do this:
Will report back with findings when it crashes again. Thanks for all the help.
It’ll probably be obvious before it crashes, you can see in the graphs that the “used” memory is increasing steadily after a reboot. Take a look now and see which process is causing that.