I’ve been running my most recent Server built for quite some time now. I think Uptime was somewhere around 5 Months. Absolutely flawless. A few Days ago i started to have issues. Hard-Locks, Freezing…but absolutely zero log entries. Nothing.
The Server was built with “off the shelf” Hardware and no ECC (even though the Ryzen CPU technically supports it, at the time ECC 3200 MHz Memory was still a lot more expensive than it is now) and is running a ZFS. Risky business, but it’s “just” a home server. Would never built a server running mission critical stuff like that (and I’ve been doing that for over 10 years now as my main job).
Over the last few weeks, i’ve been trying some stuff and had a pretty high memory load.
In any case, i also like Astrophysics and have some newsletters about Auroras and so on. They are extremely rare, here in southern Germany to occur.
Yesterday we had one of the biggest and brightest I’ve ever seen.
But it got me thinking about my hard locks and crashes and i remembered, i had an account for ESA’s SSCC (SSA Space Weather Coordination Centre). They have something called “Post-Event Analysis”, where you can correlate certain timestamps to real time data, for example from DSCOVR (“THE” Space Weather Satellite).
For Auroras to occur, the so called “Bz-Value” is important. Basically, it tells the direction of the interplanetary magnetic field. If it’s direction is towards the sun and towards the charged particles the sun throws at us, they get deflected. If it’s with the direction of the solar wind, the particles “come in” and produce auroras…because the charged particles charge other parts - they generally charge oxygen, which results in green auroras - they also can do all sorts of stuff (and that’s why spaceships, sats and other stuff floating around in space need shielding). The Value is measured in nanoTesla(nT).
It certainly could. A bit-flip in a core part of the kernel could easily cause it to lock up, if an address is corrupted and it starts writing garbage over its code, or execution jumps to somewhere unexpected, or an instruction is changed from something reasonable to a halt.
Yes, most of those should trigger a blue screen or kernel panic, but that’s not guaranteed when you’re making completely random changes.
Sure - i should have mentioned, that the system itself runs not on the ZFS but from it’s own SSD. So a “ZFS Cache in Memory Bit-Flip” should (theoretically…) not cause a hard-lock/freeze. It would probably trigger a complete garbage collection though.
And yes - that’s what was so confusing to me, no kernel panic, no log entry…nothing, just a sudden, random freeze.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
In memory?
Oh! I thought OP was referencing OS files from the drive.
It also wouldn’t cause Hard-Locks and Freezes without any errors
It certainly could. A bit-flip in a core part of the kernel could easily cause it to lock up, if an address is corrupted and it starts writing garbage over its code, or execution jumps to somewhere unexpected, or an instruction is changed from something reasonable to a halt.
Yes, most of those should trigger a blue screen or kernel panic, but that’s not guaranteed when you’re making completely random changes.
Sure - i should have mentioned, that the system itself runs not on the ZFS but from it’s own SSD. So a “ZFS Cache in Memory Bit-Flip” should (theoretically…) not cause a hard-lock/freeze. It would probably trigger a complete garbage collection though.
And yes - that’s what was so confusing to me, no kernel panic, no log entry…nothing, just a sudden, random freeze.
Right, a bit flip in ZFS cache shouldn’t cause that. But a bit flip in active memory could.
Absolutely! And I think that’s actually what happened :)