Issue Description: I have been having this issue with my raspberry pi running dietPi where it seems to lock up and I cannot SSH / access any of the services on it. The interesting part is that the interface seems to be up and I can still ping it on the local network but shows no video output. usually I get about 3-6 days before the issue appears again.
I am not really sure where to start to diagnose this issue. Any help would be appreciated!
Things Tried:
Reduced operating temp by getting a fan. I want to say this improved the length in between this issue appearing but don’t really have any hard evidence.
uninstalled unused services
Limited active torrents in QbitTorrent
EDIT: Small thing to mention is that the CPU load is usually really high - like not uncommon for the load to be between 8-10 but I have seen it as high as 24.
Temporary fix:
Power cycle - everything comes up again in less than a minute.
@AverageGoob I have this issue with one of my hosts as well. It appears to be a problem with the micro SD card. Same card, different pi = same problem. I’m currently working around it with a watchdog but will need to replace the card soon.
Are you running your OS from USB or from a micro SD card?
I upgraded to the Pi4 but I use this case. It has a daughter board that lets me use an m.2 SATA SSD over USB. But any USB to SATA adapter should work fine
You should get a scsi enabled adapter though, otherwise you may have to disable it in the kernel boot settings. And if you forget that it will run at like kbit/s.
@a_fancy_kiwi I agree, same here. This is the last pi that’s running off an SD card with services that do “significant” disk I/O. I have a few zeros that only really write to the card for OS updates. Their job is to collect data and send it via the network. I haven’t had issues with that kind of workload using micro SD cards.
Edit: For Pis with write workloads I’m using basic USB3 SSDs. Didn’t have good results with USB sticks though.
@AverageGoob The watchdog saves me from rebooting the host manually, but at the risk of data loss (though not more than a locked up SD card). I configured a custom script that writes to a file, when the card has problems, the watchdog kicks in. To keep the script from stressing the card even more, the script only writes to the file every few minutes.
As you said it’s only a workaround. I’ll move the stuff on the problematic host to a VM with SSD shortly.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
@AverageGoob I have this issue with one of my hosts as well. It appears to be a problem with the micro SD card. Same card, different pi = same problem. I’m currently working around it with a watchdog but will need to replace the card soon.
Are you running your OS from USB or from a micro SD card?
I’d bet $1 it’s the SD card. My 3B+ used to have the same problem. Been running pis off some sort of SSD ever since, no issues.
Id be willing to try this. How do you have it connected? Just using an external USB attached one?
I upgraded to the Pi4 but I use this case. It has a daughter board that lets me use an m.2 SATA SSD over USB. But any USB to SATA adapter should work fine
You should get a scsi enabled adapter though, otherwise you may have to disable it in the kernel boot settings. And if you forget that it will run at like kbit/s.
Pi 3B has dedicated bus for SD card but ethernet and usb share bandwidth. Enable zram, disable all swap and keep using sd card.
@a_fancy_kiwi I agree, same here. This is the last pi that’s running off an SD card with services that do “significant” disk I/O. I have a few zeros that only really write to the card for OS updates. Their job is to collect data and send it via the network. I haven’t had issues with that kind of workload using micro SD cards.
Edit: For Pis with write workloads I’m using basic USB3 SSDs. Didn’t have good results with USB sticks though.
I am running it from an SD card. Did setting up the watchdog ultimately work for you? I did come across a watchdog as a possible workaround.
@AverageGoob The watchdog saves me from rebooting the host manually, but at the risk of data loss (though not more than a locked up SD card). I configured a custom script that writes to a file, when the card has problems, the watchdog kicks in. To keep the script from stressing the card even more, the script only writes to the file every few minutes.
As you said it’s only a workaround. I’ll move the stuff on the problematic host to a VM with SSD shortly.