The saga .... continues!
Dun! Dun! Dunnnnnn!!
After much research, I've pretty much settled on unRAID.
The license isn't free, but you can start out cheaper and increase later (for an additional $9 upgrade "fee"). The license is based on how many disk devices are attached to the server.
Up to 6 devices: $49
Upgrade from 6 devices to 12 for another $39
Upgrade from 6 devices to unlimited for another $79
Up to 12 devices: $89
Upgrade from 12 devices to unlimited for another $49
Unlimited devices: $149https://unraid.net/pricing
You can use the 30-day unlimited license to test your hardware, but then you need a paid license for the number of devices you'll make part of the array, part of the cache pool, unattached (non-array) storage, etc.
unRAID supports up to 30 devices in an array (28 data drives + 2 parity drives) plus another 24 devices to use as caching storage. Additional devices can be added as "unassigned", meaning not part of a storage array, which can store anything, including VM files that probably don't need to be on the protected array anyway. Performance matters more for VMs as long as you make a backup occasionally.
I guess the number one reason for choosing this over FreeNAS or the open source version of Synology's DSM is the way the filesystem manages the individual disks, I've managed RAID arrays for decades, and they all have the same problem: no matter how many spares you add to the array, there is always the chance of the array being lost if you have 2 drives fail.
unRAID uses 1-2 parity drives (2 gives more protection).
Let's say one data drive fails. You replace it with a new disk, and the system starts rebuilding the new disk to replace the bad one.
Now let's say another drive fails. The parity for the array is still being protected by 2 non-storage parity drives. Not a problem. Replace the bad drive #2, and it begins the rebuild along with bad disk #1. And so on, and so on. As long as one parity drive is present, the array can rebuild itself.
In a RAID 5 array, which has been the industry standard forever, that scenario would mean having to completely restore the array from the most recent backup (you do have a full backup, right?). That's because the parity information is contained on the data storage drives. If you lose a disk, there is sufficient parity information striped across the remaining drives to rebuild it onto a new disk. But, once you lose disk #2, assuming #1 is still in the process of rebuilding, it's GAME OVER, MAN!
RAID 6, which nobody I ever worked with used or recommended, creates another disk to act as a failsafe against 2 bad drives. That's okay for that array as long as 3 drives don't fail. These things can, and do, happen close together. My co-worker was demonstrating ON A LIVE PRODUCTION SERVER how he could remove a drive and not lose the array. Unfortunately, he didn't plan on the offline spare in the RAID chassis being bad! A little while after it started rebuilding, it went offline. OOPS!
To make things worse, the good drive he pulled was still not part of the array, because he didn't know how to tell the RAID controller that drive bay was no longer "dead". Each time he replaced the drive for a rebuild, it was marked "dead". So, at that point, his spare disk and parity protection were offline. He had zero data protection. Had another drive failed, he'd have been up the proverbial creek.
The next day, he was finally able to contact the external RAID manufacturers, and they explained how the reset the status of a dead drive. Super easy, but only if you know how. There's no option in the GUI. You change it, and it won't stick. Instead, you have to change SOMETHING ELSE (like the setting for write-back operation) and save. Then reverse the change to what it was and save again. The first save makes the RAID re-poll the drives, and the status is magically set to ONLINE. LOL!
He wound up spending the night in the facility doing a complete backup to tape -- twice, just in case -- and then reinitializing the RAID array and restoring the backup. I was in San Diego and Vegas that whole time. Poor guy.
With unRAID, every single storage disk in the array can be rebuilt, because there is no striping And, if the parity is lost, any surviving storage drives can be mounted individually and the data extracted, unlike a striped array where the files are written across multiple volumes.
That whole concept of storing an entire file on a single disk makes too much sense. You lose the performance gained from striping, but that can be compensated for by using SSD cache drives.
Right now, I'm going through all my old disks and running the unRAID "Pre-Clear" utility. it stresses the disks and does full disk read-zero-read passes. A 750GB drive on my PC took over 11 hours. 2TB drives took a day and a half. All up to the speed of the PC, number of simultaneous disks being cleared, and the speed of the disk: SATA 2 or 3, hardware cache, and RPMs. I'd doing 4 750GB disks right now. The tool gives more confidence that the drive is unlikely to fail soon.
unRAID boots from a USB flash drive. It has to be that way, because the license is tied to the drive.
I've ordered an adapter card and a set of SAS-to-SATA connector cables. The one card can support up to 8 SATA drives in HBA (Host Bus Adapter) mode as opposed to RAID mode, All RAID functions will be software-based. That, along with the 6 ports on the motherboard, puts me at 14 SATA devices. Add the M.2 slots, and I'm at 16. Looks like I'll be getting the unlimited license. I can fit 8 drives in the 3.5" bays, 2 in the 5.25" bays, and 2 2.5" SSDs. That's 12 storage devices that the case can handle plus the M.2 sticks. Works out to where I have plenty of controller ports for the maximum number of SATA devices I can squeeze in my case.
At one point, I was ready to just install Linux and PLEX, then call it a day. But, this unRAID product is interesting. If it helps shore up my 12+ year old NAS storage as well as do PLEX media, VMs and anything else I throw at it, then this looks like a good long term way to go.
This is a lot more than I intended to type tonight, but I do like to share!