I’ve had the idea for a while to use an LLM to gather metadata about books for me as well as generate tag lists for themes, plot, writing style, etc for everything in my ebook library. You could also generate non spoiler plot summaries and produce recommendations for similar books.

Interesting that the one has such large capacitors in it. I imagine that is as last-ditch effort to keep the board powered long enough to finish flushing all of its caches in the event of a power failure.

That’s exactly the point of power loss protection (aka PLP.) As a side effect of not needing to wait for a flush after a write synchronous write workloads are dramatically faster on enterprise drives with PLP.

Edit: To add a bit of detail - you don’t need to wait for a flush after a synchronous write with PLP because the drive firmware can lie and immediately return from a flush call because there’s enough backup power to complete that flush if the power were cut.

You can access Gmail over IMAP and pull down messages locally. If you do this; Back. Up. Your. Mbox.

Also, fun fact, you can move messages from a local mbox to Gmail while preserving read status and original dates if you want to add old email to Gmail for some reason.

+1, your list of browser extensions, list of plugins and list of available fonts are also available to anyone trying to fingerprint you. This idea that NAT will somehow obscure you enough to be anonymous online is security voodoo.

Yes, the machine that stays off 363 days of the year is such a security risk to my home network 🙄

I mean, the horror of having to tick a box to use rotating v6 addresses. These are all solved problems, they’re not a flaw worth ignoring the entire ipv6 protocol over. Most major operating systems have moved to stable privacy preserving addresses by default, that’s true, but it’s not all that difficult to turn on address randomization and rotation either. And, hell, if you’re that married to NAT as security just use NAT66 and call it a day, nothing about NAT is exclusive to ipv4.

Your firewall should take care of that, it’s pretty rare to be connected directly without one and by default any decent routing package will filter incoming traffic that’s not in the state tracking table. NAT isn’t designed for security, any security benefit it provides is a side effect rather than the intended purpose.

Edit: check out ipv6 privacy extensions too, there are solutions there that can reduce info disclosure if that’s a concern. You can accomplish many of the same benefits of NAT with v6 features without the downsides that NAT brings.

Ipv6 is fantastic, it has less overhead than v4 and removes the need for NAT or other translation. Support can be spotty in cheaper and older devices but there’s no reason not to learn and adopt it where possible.

The only windows machine on my home network is the backup Windows laptop that I only boot when I need to run something like Odin to flash a tablet or some niche Nintendo switch management software.

Most enterprise drives are TLC these days, MLC just doesn’t provide the storage density that enterprises require anymore. I only mentioned MLC because you’ll occasionally find mSATA drives in the <=256GB range that use MLC. You have to check the datasheet for each model, look for endurance rated at 5DWPD or higher, those will typically be MLC or heavily over provisioned TLC. If you want enterprise drives with greater endurance than the usual 0.5 or 1 DWPD look for the over provisioned models with capacities like 400GB, 800GB, 1.6T or 3.2T. those are 512GB, 1TB, 2TB and 4TB raw capacity drives with a bunch of flash set aside for wear leveling purposes. You don’t often see 300GB, 600GB, 1.2T or 2.4T drives anymore but those are often very high endurance (write intensive, 10 DWPD or so) models.

Check the datasheets for drives when you’re shopping and you can get a pretty good idea of what their durability is like, I usually buy 1 DWPD drives for write occasional bulk storage and 3+ DWPD for anything with a serious write workload. You can also help the drive controller a bit by running blkdiscard against the entire device before partitioning, then only partition and use ~80% of available space. The drive controller will typically grab free unused blocks and use them for wear leveling but only if they’ve been marked free (TRIMmed) and never allocated after. If you can’t find or can’t afford high endurance drives you can usually buy a larger lower endurance drive and over provision it in this way to extend its lifespan.

(The last time MLC flash was really common was back in maybe 2014-2015, some of the older Samsung pro drives like the 850/860 pro were built using MLC. Those had legendary real world endurance, I think they’d get up to 10+PB written before actually failing. It’s a shame they didn’t have PLP because they would have made good budget array storage if they did.)

My approach to this has always been to buy one enclosure and validate it, then go buy like 8 more after thorough testing. Obviously don’t place an order for 10 units of an unknown tech item from AliExpress or you’re looking at a bad time. Look for enclosures that use known good chipsets and there’s not as much risk as you’re expecting. I have something like 8 msata enclosures here that work flawlessly and another half dozen sata+nvme rtl9210b enclosures that also work well.

Buy used Samsung mSata or m.2 2230 drives on fleaBay. Stick with Samsung and other well known brands with decent spec sheets and warranties, that’s the cheapest way to handle durable storage on a pi. USB enclosures are like $5-7 on AliExpress or fleaBay.

Buy MLC drives if you need higher endurance (check the model no and look up the datasheet.) TLC will usually be fine for a few years, MLC will last a bit longer. If you’re killing drives faster than you expect buy larger (512 instead of 256GB), blkdiscard the entire device once it’s installed and then only partition 60-80% of it. Never touch the rest of the freed storage and the drive controller should be able to use those blocks for wear levelling to reduce the NAND wear rate.

Edit: One heads up, I usually buy used drives from eBay because their buyer protection is top tier, if there’s anything wrong with the drive when it’s delivered or when I test it it goes right back for a refund. This makes buying blind viable thanks to an easy return policy.

If you’re sourcing used drives somewhere else insist on seeing SMART data before purchasing and don’t buy heavily worn drives. Look at the drive model datasheet, find the warranted endurance of the drive (if it’s a 512GB drive rated for 1 DWPD over 3y that means the rated endurance is ~ 0.5T * 365 * 3 or roughly ~550TB written over 3y. Pass on buying drives approaching their rated endurance, try to buy lightly used drives wherever possible and you shouldn’t have problems with reliability.

Buy external drives. Don’t run them in RAID, use one to store backups and plug it in once or twice a week to copy data to it.

The secret to RAID is that it doesn’t buy you data protection, it buys you uptime to access data while a device in the array is failed. This is most valuable to businesses that can’t afford the downtime that recovery from a backup incurs. The most paranoid RAID will still fail sooner or later, due to hardware or software failure, and as a home user with a limited budget you’re far better off having one offline backup that you can use to recover data from once that happens.

Backup only data you can’t afford to lose (eg: don’t backup downloaded data that can be replaced easily, like a game or movie collection) and your backups will be much more manageably sized and you won’t need to spend as much on your backup drive. If a backup disk is too much for your budget you can always exploit cloud backup plans, backblaze PC backup has no limit on the size of your backups and only charges something like ~$60/yr.

Edit: It’s also worth thinking about what kind of data you’re storing and splitting that data across multiple devices if possible. If you’re storing bulk data where performance isn’t critical, like backups from other machines or a movie collection, you can pay a much lower price by buying a hard drive instead of flash. Even if only some of your data requires fast flash you can still use a cheaper HDD to store bulk data and buy a smaller flash drive for performance sensitive tasks. When I build NAS I split my data two pools, one bulk pool of HDDs and one much smaller fast pool comprised of flash storage. Put performance critical data on flash, put bulk storage on HDDs, this will allow you to spend less on bulk and still have fast storage performance for tasks that require it. A 512GB or 1TB SSD alongside a 4TB, 6TB or 8TB HDD is significantly cheaper than spending on a 4TB or 8TB SSD.

Shop eBay for refurbished storage, it’ll be significantly cheaper than spending on brand new drives.

I don’t encourage people to buy anything older than ~2016 or Skylake era. Older chips tend to eat enough power that they’re more expensive over time (usually less than a year after purchase) than newer more power efficient parts. Run the math on power consumption with the chip’s TDP for a year as an estimate and you’ll often be surprised by just how expensive chips from <2016 end up being to run. Cpubenchmark.net will do that for you if you use the comparator, just remember to set your average kWh cost.

+1, this is my first stop too

An Optiplex 5040 should be well and thoroughly supported for 6+y now

Make sure the microcode package is up to date as well

Looks like someone fucked up package dependencies somewhere.

I’m surprised they don’t have some basic automated testing running in a VM after new package releases but I suppose they don’t need it if they can farm that duty out to their free userbase.

Hardware accelerated JSON Markov chain operations when?

Time to train an LLM to format XML and hope for the best

The best I can do is an ML model running on an NPU that parses JSON in subtly wrong and impossible to debug ways

Ad money machine didn’t go brrrrrr

For RSS I like ReadYou, for feeds I like Mastodon with a variety of interests followed. There are a surprising number of orgs on Mastodon these days.

Depends on the SSD, the one I linked is fine for casual home server use. You’re unlikely to see enough of a write workload that endurance will be an issue. That’s an enterprise drive btw, it certainly wasn’t cheap when it was brand new and I doubt running a couple of VMs will wear it quickly. (I’ve had a few of those in service at home for 3-4y, no problems.)

Consumer drives have more issues, their write endurance is considerably lower than most enterprise parts. You can blow through a cheap consumer SSD’s endurance in mere months with a hypervisor workload so I’d strongly recommend using enterprise drives where possible.

It’s always worth taking a look at drive datasheets when you’re considering them and comparing the warranty lifespan to your expected usage too. The drive linked above has an expected endurance of like 2PB (~3 DWPD, OR 2TB/day, over 3y) so you shouldn’t have any problems there. See https://www.sandisk.com/content/dam/sandisk-main/en_us/assets/resources/enterprise/data-sheets/cloudspeed-eco-genII-sata-ssd-datasheet.pdf

Older gen retired or old stock parts are basically the only way I buy home server storage now, the value for your money is tremendous and most drives are lightly used at most.

Edit: some select consumer SSDs can work fairly well with ZFS too, but they tend to be higher endurance parts with more baked in over provisioning. It was popular to use Samsung 850 or 860 Pros for a while due to their tremendous endurance (the 512GB 850s often had an endurance lifespan of like 10PB+ before failure thanks to good old high endurance MLC flash) but it’s a lot safer to just buy retired enterprise parts now that they’re available cheaply. There are some gotchas that come along with using high endurance consumer drives, like poor sync write performance due to lack of PLP, but you’ll still see far better performance than an HDD.

+1 automate your backup rolling, setup your monitoring and alerting and then ignore everything until something actually goes wrong. I touch my lab a handful of times a year when it’s time for major updates, otherwise it basically runs itself.

That’s what I’d do here, used enterprise SSDs are dirt cheap on fleaBay

If I had to guess there was a code change in the PVE kernel or in their integrated ZFS module that led to a performance regression for your use case. I don’t really have any feedback there, PVE ships a modified version of an older kernel (6.2?) so something could have been backported into that tree that led to the regression. Same deal with ZFS, whichever version the PVE folks are shipping could have introduced a regression as well.

Your best bet is to raise an issue with the PVE folks after identifying which kernel version introduced the regression, you’ll want to do a binary search between now and the last known good time that this wasn’t occurring to determine exactly when the issue started - then you can open an issue describing the regression.

Or just throw a cheap SSD at the problem and move on, that’s what I’d do here. Something like this should outlast the machine you put it in.

Edit: the Samsung 863a also pops up cheaply from time to time, it has good endurance and PLP. Basically just search fleaBay for SATA drives with capacities of 400/480gb, 800/960gb, 1.6T/1.92T or 3.2T/3.84T and check their datasheets for endurance info and PLP capability. Anything in the 400/800/1600/3200Gb sequence is a model with more overprovisioning and higher endurance (usually refered to as mixed use) model. Those often have 3 DWPD or 5 DWPD ratings and are a safe bet if you have a write heavy workload.

iowait is indicative of storage not being able to keep up with the performance of the rest of the system. What hardware are you using for storage here?

But can they run Crysis?

Yeah, you’ll be fairly limited as far as GPU solutions go. I have a handful of hh AMD cards kicking around that were originally shipped in t740s and similar but they’re really only good for hardware transcoding or hanging extra monitors off the machine - it’s difficult to find a hh board with a useful amount of vram for ml/ai tasks.

Distcc, maybe gluster. Run a docker swarm setup on pve or something.

Models like those are a little hard to exploit well because of limited network bandwidth between them. Other mini PC models that have a pcie slot are fun because you can jam high speed networking into them along with NVMe then do rapid fail over between machines with very little impact when one goes offline.

If you do want to bump your bandwidth per machine you might be able to repurpose the wlan m2 slot for a 2.5gbe port, but you’ll likely have to hang the module out the back through a serial port or something. Aquantia USB modules work well too, those can provide 5gbe fairly stably.

Edit: Oh, you’re talking about the larger desktop elitedesk g1, not the USFF tiny machines. Yeah, you can jam whatever hh cards into these you want - go wild.

Bus issues usually. Having a disk (or 4) drop out of a ZFS filesystem regularly isn’t a good time.

If you can find a combination of enclosure, driver/firmware and USB port that provides you with a reliable connection to the drive then USB is just another storage bus. It’s generally not recommended because that combination (enclosure, chipset, firmware, driver, port) is so variable from situation to situation but if you know how to address the pitfalls it can usually work fine.

I only backup data that I either can’t replace or would have to spend significant effort to replace. Most of what’s on a media server doesn’t fall into that category.

You don’t need any kind of special computer to learn programming. Find an introduction to programming course in Python that works for you and just go for it. Eventually a nice laptop will be useful but you’re not writing the kind of code that’s going to take hours to compile for the first couple of years anyway.

I’m pretty tempted to just buy a steam deck and run yuzu to play my switch games. Between vita3k, yuzu and whatever emulates a 3ds I’m thinking I could consolidate all of my handhelds pretty effectively at this point.

I’m not sure what you’re shopping for with AES-NI but I can strongly recommend the HP T730 and T740 thin clients if you’re trying to build a budget home firewall machine. Both support AES-NI (but obviously not Xeon QAT) and the t730 is cheap on eBay. Drop whatever NIC and an SSD in and you’re off to the races with OPNSense. The T740 is performant enough to run OPNSense on Proxmox if that’s your thing, you’ll have plenty of spare processing time to do something else on the machine beyond routing/firewalling a 1-2Gb home connection.

Go read the switch custom firmware guide, everything is explained in detail

Read the data sheets.

You’re mostly going to be concerned with IOPS and endurance for VM hosting.

I use mostly Samsung, SK Hynix, Micron and SanDisk. For bulk storage it doesn’t really matter which of those you pick but for fast storage you’ll want to be sure the drive offers PLP.

Go hit up fleaBay and see what’s available in the way of enterprise drives in the size you need then google the model numbers and check out the datasheets. Once you know what each drive is capable of you can decide which to buy. I usually try to buy 3 dwpd models for VM storage and 1.3 dwpd for bulk, you might prefer to focus on IOPS over endurance it’ll depend on your application.

Edit: for a VM host pool you’re primarily going to be concerned with IOPS, endurance and having PLP for better ZFS performance. For bulk storage you can skimp on specs to some extent. I prefer to use cheaper drives like the SanDisk cloudspeed eco line for a bulk storage pool and whatever high IOPS+endurance drives I can find cheap for my VM host pool. When you split your pools you can do things like use mirror zdevs for performance for VMs and raid z whatever for bulk storage.

How many drives are you looking to use, what are they for, what interfaces do you have available on the machine (SAS backplane, SATA, any number of available NVMe hookups of some flavor, etc), what pool topology are you trying to use and what is the intended workload you want to jenga tower off of all of the above? With more info people can give you more specific recommendations. (E: and what sort of machine are you running things on while I’m at it, processors and amount of RAM would be useful)

Look for something built on an Intel i225 or i226, there should be a 4 port model