How do you keep your digital music files? (Read 12184 times)

Jl808

Re: How do you keep your digital music files?
« Reply #20 on: April 26, 2020, 09:40:49 PM »
Thanks for sharing your research, Flapp!

Your last 2 posts have piqued my interest in building something, although it will likely have to take a backseat for when I have a bit more free time from work. No I haven’t heard of XPENOLOGY and will have to check it out including the other open source NAS options.  I’ve been pretty happy with the Synology and haven’t felt the need to look at other options, but it would be interesting to check our other options to see if they might be better. I’ll need to fire up a new VM on my server and check it out sometime.
I think, therefore I am armed.
NRA Life Patron member, HRA Life member, HiFiCo Life Member, HDF member

The United States Constitution © 1791. All Rights Reserved.

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #21 on: April 27, 2020, 03:14:30 AM »
Thanks for sharing your research, Flapp!

Your last 2 posts have piqued my interest in building something, although it will likely have to take a backseat for when I have a bit more free time from work. No I haven’t heard of XPENOLOGY and will have to check it out including the other open source NAS options.  I’ve been pretty happy with the Synology and haven’t felt the need to look at other options, but it would be interesting to check our other options to see if they might be better. I’ll need to fire up a new VM on my server and check it out sometime.

One main thing I saw about the Xpenology DSM is that you can't do updates of the software from Synology, even if you already have a Synology NAS.  The machine has to be registered to allow updates.  So, any updates have to be performed the old fashioned way -- download and install over the old. 

The other thing thing I noticed is the drivers in the DSM installation don't always work with all non-Synology-based hardware.  Xpenology tried to help with that, but they don't have all NICs, motherboard chipsets, etc. tested with their drivers.  Synology is working with a closed hardware, so their software has a smaller target to hit.

I was thinking of using whatever new server I build as a VM server, too.  I really don't need something that fancy, as I just use the VMware Workstation and Fusion versions on my PC and MacBook.  I rarely fire up more than one VM at a time.

I'm in the process now of finally cleaning up all my iTunes files.  First step is to convert all the AAC/m4a files to MP3.  That's been running for 2 hours so far.  Next, copy, rename and organize in artist/album named folders all the MP3 files.  I'm using a free tool to bulk-rename files that need dashes added or artist names removed from the song titles.

I feel like the hoarder who's finally putting everything I have a use for in well labeled boxes and file folders.  :thumbsup:
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #22 on: April 27, 2020, 03:22:36 AM »
BTW, here are a few tidbits I've picked up from PLEX Server.

1.  The FireTV Android app for PLEX Client doesn't work on movies or TV shows that are split into multiple files ("stacked").  You can either make them all have non-stacked names and watch them individually, or append them to one another and make a single file.  I did the latter.

Stacked media files worked fine on my PC, Mac and ReadyNAS players.

2.  No need to group collections of movies into folders for easy binge watching.  Just use the Tags feature.  Select the movies in the group, and add them to a collection under Tags.  Then you can filter by changing the "ALL" filter button to "Collections."  Nice feature.

3.  There are separate articles on the PLEX site for movies, music, TV shows, and so on.  Each has a short tutorial on the best folder (directory) structure and file naming conventions to use for each type of media.  Finally!  I have a standard!!   :thumbsup: :rofl:  I've been struggling with that every time I tried to better organize my media files.  Never seemed to be as robust or useful as I'd hoped.

Lots to learn!   :geekdanc:
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #23 on: April 28, 2020, 05:26:20 PM »
Well, I guess I'll be building a server.  Probably a good thing.  I watched a couple of movies on the ReadyNAS PLEX server yesterday, and about halfway into each movie it stuttered a bit.  Not too bad, but annoying.

Earlier I was wondering if I should get an AMD Ryzen for the core count or an Intel i5 for the Intel Quick Sync Video feature, which helps with transcoding to H.264/H.265.  Either choice would have cost around $300 -- just for the CPU.  Don't really want to get a less powerful CPU than my current i7-7700K workstation.

I checked Craig's List, and someone had an Intel CPU they'd just listed about 24 hours earlier.  I had looked at the i7-9700K on Newegg this week discounted for $397 + tax/shipping and decided against it based on price.

The i7-9700K I bought on CL was just $250.  I checked the Intel warranty page, and although there's no warranty available ("tray" CPU sold to OEM system builders), I at least verified it's genuine.  $150 less than a new i7-9700K and $50 less than the other options I considered.

This CPU is the next-to-top level CPU for the 300 series chipset motherboards -- the i9-9900K being the very top.  Performance-wise, they are almost identical until you need more than 8 cores.  It should be in demand for quite a while if I ever want to part with it.  My i7-7700K just turned 3 years old, and it's selling on ebay for $50 more than I paid new.  It's the best CPU available for its motherboard line and hard to find, so anyone looking to replace/upgrade their PC would be willing to pay a little more to avoid having to get an entirely new system. :thumbsup:

Now I need a motherboard and CPU water cooler. 

The board I am considering supports Intel HD video using this CPU, so no need for a discrete GPU card.  I have 32GB of 3200Ghz DDR4 DRAM I can split and use 1/2 in both PCs for now.  I also have 3 x 2.5" SSDs and 2 x NVME M.2 SSDs.  Plenty of parts to share.

Now if Newegg would just have the few parts I do need on sale this week!  My shopping cart comes to $300 right now. Add the case and CPU, and my cost is $650 -- about what I would have spent on a 4-5 Bay transcoding-capable NAS.  AND I can put some of the parts I have sitting in boxes to use.  Bonus!

Basically, the new CPU will become my workstation (with some RGB bling added  O0), and my old CPU will be "promoted" to server.   :geekdanc:
« Last Edit: April 29, 2020, 12:51:27 AM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #24 on: May 02, 2020, 12:44:58 AM »
Friday night (opps!  Saturday morning!) update ...

Received my motherboard today.  Installed the Craig's List CPU and 1/2 my DDR4 RAM borrowed from my workstation, plugged in an HDMI cable to the TV, attached a brand new 1000W power supply (Warranty replacement for an 850W PS from Corsair that died), and added a spare keyboard and trackball. This was my bench test config -- with the MB sitting on it's cardboard box.

The "system" POSTed with no problems.  Very first order of business was to update the BIOS.  Only one version behind, so not bad out of the box.  I love being able to boot into a BIOS update utility instead of breaking out an old DOS boot disk/CD/Thumbdrive -- or after having to make a new one because I can't find the one I know is around here somewhere!   :wacko:

I turned on the CPU Game Boost feature on the MSI BIOS, and the CPU clock speed went from 3.6 ghz base speed to 4.7 ghz.  Memory is running at 3200 ghz, just like in the other machine -- overclocked using the Intel XMP setting (Extreme Memory Profile).

So, I guess the CPU works.   :thumbsup: :thumbsup:

My AIO Water Cooler for the CPU arrives Monday.  I also ordered a 32 GB RAM kit (2 x 16) today.  I can put all 4 sticks of my old RAM in my new workstation, and I'll have room to add another 32GB in the server if ever I need to.  A 64GB kit for that MB would have cost $400.  The 32GB I ordered was on sale today for $119.  This way, if I need more, it'll hopefully be cheaper in a year or so.  I was toying with buying faster clock speed RAM, but the cost/additional-performance just isn't there over 3200ghz RAM unless you're building a high-end AMD machine.  Those CPUs seem to need faster RAM to get the expected performance out of their CPUs.

Basically, I only need the cooler to get the build finished for both my new i7-9700K workstation and i7-7700K server.  Then I can see how well Plex does on a REAL computer.   8)  Right now, I have an old & partially corroded heat sink stuck to the CPU running 2 120mm fans.  Seems to still be good after some 13+ years.  Haven't used it for at least 8.  I figured 2 fans was prudent, since this heat sink wasn't really made for this CPU.

I considered running it on my workstation, but then I run the risk of messing up the server features when doing anything new, like upgrading hardware or install O/S updates and application software. 

Workstation  <->  Server <->  NAS  <->  Backup NAS

Looks like a plan!   :geekdanc:
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #25 on: May 02, 2020, 12:59:15 AM »
BTW, I have one mystery to track down.  On my older MSI board, I was able to achieve more than the Intel-advertised Max Turbo Frequency of the i7-7700K by just turning on the Game Boost feature. 

i7-7700K
Base clock:  4.2 GHz
Max Trubo:  4.5 GHz
Game Boost Enabled:  4.8 GHz

However ..

i7-9700K
Base clock:  3.6 GHz
Max Trubo:  4.9 GHz
Game Boost Enabled:  4.7 GHz

I think the Game Boost should at least give the CPU setting for the Intel Max, or another 0.2 GHz.  I was expecting 5.0 off the bat, with MAYBE 4.9 as a minimum boost.

I know the new CPU is much faster than the old one.  But, there are those who see very stable settings running the 9700K at 5.0 GHz.

More fun!!   :geekdanc:
« Last Edit: May 02, 2020, 01:29:31 AM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Jl808

Re: How do you keep your digital music files?
« Reply #26 on: May 02, 2020, 06:10:45 AM »
Maybe you are having an apples-to-oranges comparison.

What other components are different besides the CPU?

Could the MB be making a difference?
I think, therefore I am armed.
NRA Life Patron member, HRA Life member, HiFiCo Life Member, HDF member

The United States Constitution © 1791. All Rights Reserved.

jerry_03

Re: How do you keep your digital music files?
« Reply #27 on: May 02, 2020, 09:35:27 AM »
BTW, I have one mystery to track down.  On my older MSI board, I was able to achieve more than the Intel-advertised Max Turbo Frequency of the i7-7700K by just turning on the Game Boost feature. 

i7-7700K
Base clock:  4.2 GHz
Max Trubo:  4.5 GHz
Game Boost Enabled:  4.8 GHz

However ..

i7-9700K
Base clock:  3.6 GHz
Max Trubo:  4.9 GHz
Game Boost Enabled:  4.7 GHz

I think the Game Boost should at least give the CPU setting for the Intel Max, or another 0.2 GHz.  I was expecting 5.0 off the bat, with MAYBE 4.9 as a minimum boost.

I know the new CPU is much faster than the old one.  But, there are those who see very stable settings running the 9700K at 5.0 GHz.

More fun!!   :geekdanc:

What Operating system do you plan on running on your server?

I have a server too, but its a prebuilt Dell server with Xenon CPU, 64gb ECC RAM, etc.

Its a VM server, running VMWare's ESXi. If you just install ESXi the software is free to use but is neutered (doesnt allow vMotion, Host Clustering, etc, etc) vs the paid vSphere license.

My main VM that I run is Windows 2016 Server that hosts my Plex Server. I also run a couple of other VMs: pfSense firewall VM, Windows 10 VM that hosts my thin client image, Ubuntu VM, Kali Linux VM, and a bunch of other VMs that i run every now and then. I have all of these VM's cause i work in IT and use it as a lab to practice/learn new configurations and such. Plus im just a nerd and its fun to me.

I may actually build another server with an i7 or i9 CPU and dedicated GPU and put Windows 2019 Server on it as the hypervisor and run Hyper-V. Hyper-V allows you to passthru the GPU the VM's, so basically you can game on the VM. If I do go through with this build and QoS is good enough with the gaming, I may just get rid of my dedicated gaming PC and just run all my games off the VM hosted on the Hyper-V server, and have my thin client on my desk to connect to the VM.
« Last Edit: May 02, 2020, 09:42:45 AM by jerry_03 »

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #28 on: May 02, 2020, 11:24:23 AM »
What Operating system do you plan on running on your server?

I'm going to start with Xen running with Ubuntu.  Seems to be a well supported and documented project, and it advertises as the only open source Type 1 (bare metal) hypervisor product out there.

Once I have that running, I'll definitely add a VM for Windows 10, a couple of Linux flavors (Ubuntu for sysadmin use, CentOS for dev/test),  and a FreeNAS VM of course (the whole reason for this "project").

The server's main purpose is to run PLEX Media Server.  It can run on Windows (but why?), Linux and even FreeNAS machines (FreeNAS is FreeBSD at its core). 

I'm trying the beef this machine up enough to run my main OS (hypervisor) and at least 2 VMs at once.  Loading PLEX and FreeNAS on one VM tackles two requirements in one instance. 

If I didn't want to run other OS flavors, too, I'd just load FreeNAS, with PLEX, and be done -- but I'm never done when it comes to computers.  Always something new to play around with.

I also have 2 licensed versions of VMware -- Workstation for the PC, and Fusion for the Mac.  I can use those machines for non-server type VM needs, which I do on the Mac daily.  I have a couple of Windows-only applications that I run in a VM on my older MacBook Air.  VMware had a very good promo for the holidays, so I finally upgraded both versions.  I'd been holding off because my kids no longer qualify for the student discounts  :(, and I didn't need to upgrade at full cost just to have the latest releases as long as all was working.  With Windows 10 and the latest Mac OS upgrade, I needed to upgrade VMware, too.  I was lucky I had a trial license to make sure the new Mac VMware fixed my issues.  The very week that trial ended, they had a sale.  That's how they get you!   >:D

I've never seen a need for ECC RAM.  Too expensive for current technology memory, and older tech isn't compatible with my hand-me-down PCs.  Most, if not all, of my MBs don't support ECC.  For home use, as long as I have lots of backups, a dropped bit isn't the end of the world.

This is all growing out of a new desire to simplify and take advantage of my media collection -- mostly music, but also movies.  So, I went down the PLEX rabbit hole.  My current NASes are not -- current.  Running streaming services on them wasn't a consideration 5 or 10 years ago.  I just wanted a NAS to store the files, and I would store transcoded movies to run on whatever streaming devices I had.  I used iTunes to support, well, Apple more than anything, but to use for iPods and access the iTunes store.  Not the case as much today.  I'll still have my media on the NAS, but the PLEX server will access the files, buffer it, transcode as needed, and provide real horsepower to make sure 2 or more people using it won't see any slow-downs.  I believe I can use RAM and SSDs to make buffering, transcoding and backup areas to increase performance and avoid the PLEX metadata being destroyed by HW failure or a brain fart.

I was getting an itch to upgrade my PC anyway.  Now I have a 9700K 9th Gen to replace my still very capable 7700K 7th Gen CPU, which is to become my new server once all the parts are segregated.  Just waiting on a Fractal Design Celsius S28+ water cooler.  I tried to stay away from the RGB fans -- too many additional wires, and the better fans are way more expensive than non-RGB.  I'll have enough pretty colored lights on my GPU, RAM, CPU Cooler pump, and motherboard.  :thumbsup:

Having said that, I have support on the new MSI board for both ARGB and RGB.  It's only a matter of time until I get suckered into this fad...   :shake:  My daughter bought me 2 LED strips powered by USB a few years ago.  Maybe that'll suffice for now.   :rofl:
« Last Edit: May 02, 2020, 11:32:19 AM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #29 on: May 07, 2020, 08:33:32 PM »
Well, I have 2 PCs up and running, so now I need to try migrating my workstation OS drive to the new CPU/MB.  It's that, or sitting for hours reinstalling everything .... again.  If Windows 10 plays nicely, it should load most, if not all, of the device drivers I need.  Otherwise, I should be able to come back and load just the drivers that show problems in Device Manager.  A clean install is always preferred, but I just went through that when my boot SSD died in Jan.  Don't like the idea of doing all that installation and setup again.  Although now I do have a working model (my old system) that I can use as a template this time if I go that route.

My old CPU/MB will be loaded with FreeNAS (FreeBSD) using the ZFS filesystem.  I just received 32GB of RAM which posted fine and runs at the same 3200 MHz as the other RAM I had.  This kit of 2 DIMMs leaves room for another 2, maxing out the MB at 64GB should I need it.

After inventorying my HDD collection, I see I have ZERO SATA III disks.  I do have a few that are 7200 RPM, so that's a plus.

I remembered I upgraded my MacBook Pro a few years ago to 16GB of RAM, and I still have the 8GB (2x4GB).  It so happens that RAM is the same form factor and speed as the 2GB single stick in my ReadyNAS RN314.  Since the max RAM for that thing is 4GB, I figured an upgrade was in order.  The memory test has been running for just over two hours without a single error.  I'll let it go for 3 hours as recommended and call it good.

While checking on my system's existing memory, I see that the Atom D2701 processor can't support ECC memory.  Seems odd that a NAS is not using ECC RAM, huh?

I've never cared about ECC RAM for home use.  I use incremental backups and snapshots on the RAIDs, so if I find a corrupted file, I can usually locate a copy, too.  I really can't think of a single instance when a file was corrupted by anything other than a power outage (before the affordable UPS came along) and/or a hard drive failure.I only buy top brand RAM with great reviews anyway.  Only had one set ever go bad, and that was the power supply's fault.  Before I figured it out, I'd lost a MB, 16GB RAM, a CPU AND the power supply.  I finally bought a PS tester and figured out who the culprit was.

There are some big shake-ups in the CPU market this week.  AMD now has a major advantage in the performance, workstation and gaming CPU segments now.  Their new Ryzen 3 3100 and 3300X 4 core/8 thread CPUs rival the 3-year old i7-7700K.  Intel's price new is $379 on Newegg.  The Ryzen will be selling for $100 and $120 respectively.  I think the best thing Intel has going for them still is better compatibility with operating system and driver software.  AMD isn't that far behind, though. 

Gotta love good ol' capitalist competition!  It drives innovation and keeps prices lower for the consumer.
« Last Edit: May 10, 2020, 09:40:07 AM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #30 on: May 09, 2020, 01:18:16 AM »
I opted to do a clean Win10 install on the new PC.  With the way MS has the system identity key embedded, I'm have to remove all the drivers from the old system I won't need, run sysprep to reset  the SID,  and move that media to the new PC. Then boot up the new PC and reinstall any drivers and hope I can reactivate Windows.

I figured it's easier and safer to bite the bullet and do a reinstall of everything ... again.  Downloading Office 365 now.

When the new workstation is completely setup, I need to install the OS for the PLEX server.  I've been planning to use FreeNAS on Ubuntu, but I've seen this YT channel where the guy is having great success using UnRAID.  After doing some research, I've decided FreeNAS (as in "costs nothing") not only suits my needs, but also has many features and capabilities that unraid lacks.

This is 2 years old, buit I found nothing to tell me it's changed significantly.

https://linuxhint.com/freenas_vs_unraid/

This paragraph alone from another article doesn't sound good to me relative to FreeNAS:

Quote
Traditional RAID systems use RAID 0, RAID 1, RAID 5, RAID 6 or some combination thereof. These RAID levels stripe data over
multiple disks and, for non-RAID 0 or RAID 1 systems, also distribute parity blocks across the member disks. unRAID is not like
these traditional RAID systems. The closest it can be compared to is RAID 4, a system in which data is striped across member
disks and parity is always written to a dedicated parity disk. In the case of unRAID, the data is never striped. A given file is written
to only one of the member disks. A dedicated parity disk enables recovery in case of a single disk failure. In addition, the disks can
be of different sizes, as long as the parity disks is the largest of the lot.

And this from the Wiki:

Quote
What are the disadvantages of unRAID compared to similar products?

No striping. So, although it performs well, better than many NAS solutions, it's generally slower than a RAID 0, RAID 5, RAID 6, or RAID 10, etc.

If performance is very important for an array you may be building, then one of those 4 RAID types will be a better choice than unRAID.

I do like the unraid "JBOD" design, in that you can use a collection of different size drives.  The parity data will be written to the drive with the largest capacity, but for the array itself, it uses all the space on all the other drives.  In a normal RAID array, the space available is limited to the size of the smallest drive if they are unequal.  So, 2x6TB and 2x8TB means you have basically 4x6TB drives less 1x6TB for parity in RAID 5.  Unraid would make one 8TB the parity drive and then allocated the additional 2TB on the remaining 8TB drive for storage, too.

Not sure that is worth the license fees.  unraid is including lots of other things like virtualization, too.  Seems like I'd be going down a stovepipe of only unraid-provided tools,  I much prefer integrating the "best of the best" when it comes to a Linux/Unix server.  Painting oneself into  a corner never seems to end in a happy place.   :rofl:
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #31 on: May 18, 2020, 01:31:31 AM »
The saga .... continues! 

Dun!  Dun!  Dunnnnnn!!

After much research, I've pretty much settled on unRAID.

The license isn't free, but you can start out cheaper and increase later (for an additional $9 upgrade "fee").  The license is based on how many disk devices are attached to the server.

Up to 6 devices:  $49
Upgrade from 6 devices to 12 for another $39
Upgrade from 6 devices  to unlimited for another $79

Up to 12 devices:  $89
Upgrade from 12 devices to unlimited for another $49

Unlimited devices:  $149

https://unraid.net/pricing

You can use the 30-day unlimited license to test your hardware, but then you need a paid license for the number of devices you'll make part of the array, part of the cache pool, unattached (non-array) storage, etc.

unRAID supports up to 30 devices in an array (28 data drives + 2 parity drives) plus another 24 devices to use as caching storage.  Additional devices can be added as "unassigned", meaning not part of a storage array, which can store anything, including VM files that probably don't need to be on the protected array anyway.  Performance matters more for VMs as long as you make a backup occasionally.

I guess the number one reason for choosing this over FreeNAS or the open source version of Synology's DSM is the way the filesystem manages the individual disks,  I've managed RAID arrays for decades, and they all have the same problem:  no matter how many spares you add to the array, there is always the chance of the array being lost if you have 2 drives fail. 

unRAID uses 1-2 parity drives (2 gives more protection). 

Let's say one data drive fails.  You replace it with a new disk, and the system starts rebuilding the new disk to replace the bad one.

Now let's say another drive fails.  The parity for the array is still being protected by 2 non-storage parity drives.  Not a problem.  Replace the bad drive #2, and it begins the rebuild along with bad disk #1.  And so on, and so on.  As long as one parity drive is present, the array can rebuild itself.

In a RAID 5 array, which has been the industry standard forever, that scenario would mean having to completely restore the array from the most recent backup (you do have a full backup, right?).  That's because the parity information is contained on the data storage drives.  If you lose a disk, there is sufficient parity information striped across the remaining drives to rebuild it onto a new disk.  But, once you lose disk #2, assuming #1 is still in the process of rebuilding, it's GAME OVER, MAN!

RAID 6, which nobody I ever worked with used or recommended, creates another disk to act as a failsafe against 2 bad drives.  That's okay for that array as long as 3 drives don't fail.  These things can, and do, happen close together.  My co-worker was demonstrating ON A LIVE PRODUCTION SERVER how he could remove a drive and not lose the array.  Unfortunately, he didn't plan on the offline spare in the RAID chassis being bad!  A little while after it started rebuilding, it went offline.  OOPS!

To make things worse, the good drive he pulled was still not part of the array, because he didn't know how to tell the RAID controller that drive bay was no longer "dead".  Each time he replaced the drive for a rebuild, it was marked "dead".  So, at that point, his spare disk and parity protection were offline.  He had zero data protection.  Had another drive failed, he'd have been up the proverbial creek.

The next day, he was finally able to contact the external RAID manufacturers, and they explained how the reset the status of a dead drive.  Super easy, but only if you know how.  There's no option in the GUI.  You change it, and it won't stick.  Instead, you have to change SOMETHING ELSE (like the setting for write-back operation) and save.  Then reverse the change to what it was and save again.  The first save makes the RAID re-poll the drives, and the status is magically set to ONLINE.  LOL!

He wound up spending the night in the facility doing a complete backup to tape -- twice, just in case -- and then reinitializing the RAID array and restoring the backup.  I was in San Diego and Vegas that whole time.  Poor guy.   :rofl:

With unRAID, every single storage disk in the array can be rebuilt, because there is no striping   And, if the parity is lost, any surviving storage drives can be mounted individually and the data extracted, unlike a striped array where the files are written across multiple volumes.

That whole concept of storing an entire file on a single disk makes too much sense.  You lose the performance gained from striping, but that can be compensated for by using SSD cache drives.

Right now, I'm going through all my old disks and running the unRAID "Pre-Clear" utility.  it stresses the disks and does full disk read-zero-read passes.  A 750GB drive on my PC took over 11 hours.  2TB drives took a day and a half.  All up to the speed of the PC, number of simultaneous disks being cleared, and the speed of the disk: SATA 2 or 3, hardware cache, and RPMs.  I'd doing 4 750GB disks right now. The tool gives more confidence that the drive is unlikely to fail soon.

unRAID boots from a USB flash drive.  It has to be that way, because the license is tied to the drive.

I've ordered an adapter card and a set of SAS-to-SATA connector cables.  The one card can support up to 8 SATA drives in HBA (Host Bus Adapter) mode as opposed to RAID mode,  All RAID functions will be software-based.  That, along with the 6 ports on the motherboard, puts me at 14 SATA devices.  Add the M.2 slots, and I'm at 16.  Looks like I'll be getting the unlimited license.  I can fit 8 drives in the 3.5" bays, 2 in the 5.25" bays, and 2 2.5" SSDs.  That's 12 storage devices that the case can handle plus the M.2 sticks.  Works out to where I have plenty of controller ports for the maximum number of SATA devices I can squeeze in my case.

At one point, I was ready to just install Linux and PLEX, then call it a day.  But, this unRAID product is interesting.  If it helps shore up my 12+ year old NAS storage as well as do PLEX media, VMs and anything else I throw at it, then this looks like a good long term way to go.

This is a lot more than I intended to type tonight, but I do like to share!   :thumbsup: :geekdanc:


"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #32 on: May 18, 2020, 01:50:05 AM »
Almost forgot.  I read that only Intel network adapters should be trusted on unRAID and pretty much any other server.  But, unRAID had no problem with the Killer 2500 adapter on the motherboard,  Lots of people posted that was a "Windows-only" NIC.  I guess mine is different?

Anyway, I wanted to add some throughput for multiple client media streaming, so I ordered an add-on intel-based NIC.  I only needed one port, but 2 would be better.  Instead of 2 cards, I wanted one card w/2 ports.  But, I found a good deal on a new 4-port card instead,  incase one port dies, I'm covered!   :rofl:

Now I have 5 total NICs.  I intend to bond them as one IP address using a Round-Robin mode.  That'll allow the server to run at theoretically 5GB/s.  That assumes 5 files are being sent or received simultaneously.  I'll probably use 4 for the RAID and PLEX connections, and use the on-board NIC for a Windows or Linux VM.
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #33 on: May 18, 2020, 02:12:56 AM »
Man!  All of this so far is just leading up to having a system that I can start using. 

The parts are all hand-me-downs from my daily-use PC except for:

1. 4-port Network adapter
2. LSI 9211-8i SAS/SATA controller
3. Set of 2 cables for LSI controller
4. 500GB NVMe M.2 drive for unassigned storage (PLEX metadata, VMs, etc)
5. 32GB SDRAM (3200 MHz)
6.  unRAID licence

Not too bad.  I have a stack of 3.5" drives I removed each time I upgraded my NASes.  Hopefully this little project can put off the next NAS disk replacement until 6TB drives are selling for $40!
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #34 on: May 20, 2020, 10:34:57 PM »
I know you all were dying for an update, so ...   :geekdanc:

Monday, I received my Host Bus Adapter (HBA SAS/SATA Controller) and the SAS-to-4XSATA cables.  Good timing between eBay and Amazon!

Hooked up 8 drives to the controller and started a Pre-Clear in unRAID on all 8 at in one process. 

One 500GB disk died right off the bat!  I think those are my only 7200RPM disks, so it's not surprising they are running 10+ degrees Celsius hotter than all the others.  Probably why they weren't in service all that long before I went with 750GB disks.  Of the 3 I tested, one failed early, and the other two are racking up some high SMART stats.  Only one is looking like it's likely to fail anytime soon, and the other looks fine.  However, they both are really slow considering they have the best platter speeds.  Hmm....  Might be good to set them aside and not waste time adding them into an array.

After several cycles, I can safely separate the disks into 3 groups:  bad, good but showing possible signs of potential problems, and Good w/99% confidence. 

Once I completed the tests on those disks, I updated the firmware on the controller.  It's an LSI 21XX-8i with a 2008 LSI controller chip.  That's probably the most common and well-liked non-Enterprise controller out there.  It's a 2008 design (hence the designator), but it's been getting FW and BIOS support up through 2016 by various companies that acquired LSI/Broadcom or by those who write drivers for servers like Dell, IBM, HP, etc.

Anyway, the card came with the IT mode already flashed onto it.  That's the non-RAID FW.  The RAID FW is the IR mode version. 

After researching, I erased the controller's P10 firmware completely, flashed the latest P20 onto it, and skipped flashing the UEFI and Legacy BIOSes.  Unless you're using the boot menu on the card to configure it, build RAID volumes, or boot from an OS device attached to it, leaving that part of the controller empty makes the boot process quicker -- like the controller isn't even there.  The OS driver only needs the firmware to work, so all's working fine.  The card was $38.88+tax on eBay, and a set of 2 8087 SAS-to-SATA cables was $11 on Amazon.  Controller card fits in any x8 or x16 PCI-E 2.0 or better expansion slot.

So, after all the Pre-Clear jobs finish, I hope to have at least 10TB for my array using 5 or more disks.

These Prep jobs take forever, but it could be worse.  I read it takes 3-4 days to Pre-Clear 10+ TB drives.

Now that I have 2 well-tested 750GB drives, I can use them to replace a 1TB surveillance video recording drive].  I noticed problems this weekend, and discovered one of 2 disks was dead.  I'll save my 2TB disks for the unRAID server.
« Last Edit: May 20, 2020, 11:06:59 PM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #35 on: May 21, 2020, 06:39:44 PM »
Just finished 3 passes of Pre-Clear on 4x320GB disks -- the original NAS disks I bought in 2006.    :shake:

The three Pre-Clear passes ran on all 4 disks simultaneously, and took just under 15 hours to finish without errors.   :thumbsup:

Only one drive is showing worrisome SMART stats.  I'm running a long self-test on all to see what that says.

I ran those in the NAS until 2008 when I upgraded to 500gb disks.  I then used the 320s in a Linux RAID config for a development machine.  After migrating to VMware, I no longer needed that, and the 320s have been stored -- I'm guessing about 10 years.

These disks are 7200 RPM, too.  Unfortunately they are SATA II - 3Gb/s, as are all my internal disks at the moment.  They run about 50-54 deg C, which is pushing todays thresholds, but I guess they were built back when components were less susceptible.

At some point, I'll upgrade my newer NAS to 4x8TB disks.  Then I can reuse its 4x2TB disks in unRAID.  My old NAS has been maxed out for awhile at 2TB disks.  8TB will max out the new NAS at 24TB useable space.

One day I'll ditch these old disks, but when they cost $100-$275+ each new, I'm reluctant to dispose of a working storage device. 

This year I finally got rid of a stack of EIDE and SCSI disks.  Some were dead, but most just had old bus connections, slow speeds and low capacities that made them next to useless.  Some of them had capacities less than a $10 USB flash today!  So, I'm getting better about letting go.   :wacko:

My 2 new USB flash drives arrived today.  I already made them unRAID boot devices.  I figured when I finally license unRAID, I need to use a new vs years-old USB stick, since you can only replace a USB boot stick once every 12 months without jumping through hoops.

Next step -- decide on the drives to use.  Then I can request a 30-day temp license to build an array and really start playing with unRAID.



"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

robtmc

Re: How do you keep your digital music files?
« Reply #36 on: May 22, 2020, 04:30:58 PM »
Mine are still on the CD they came on. 

How else would I play them in the stereo system?

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #37 on: May 22, 2020, 05:01:43 PM »
Mine are still on the CD they came on. 

How else would I play them in the stereo system?

CDs don't last forever.  Most music CDs are lucky to be playable after 10 years if kept away from warm temps and sunlight.

I have vinyl that's almost 50 years old, and the only real problem is with some sleeves disintegrating, but very few with that.

There's a convenience factor with having your CD collection copied onto a media server.  You can then enjoy YOUR music on YOUR devices, such as your iPod and original iPhone.   :shaka:

That doesn't even bring in the potential for scratches, other damages or theft/loss preventing you from enjoying your CDs. 
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

Flapp_Jackson

Re: How do you keep your digital music files?
« Reply #38 on: May 23, 2020, 01:34:19 AM »
Now that I've tested and cleared the 4 x 320GB drives, I'm going to be listing them on Craig's List.

Just seems counterproductive to take up 4 drive bays with barely 1 TB of usable space: 4*320 = 1,280 GB less whatever overhead.  Probably right at 1.1TB.  For $100, I can get a 4TB WD Red NAS drive and still have 3 bays and 3 sets of cables for other drives.

One YouTuber that does a ton of unRAID videos (Byte My Bits) uses "shucked" drives.  He buys the WD external USB drives when they go on sale for less than the same capacity internal drives cost new,   He then shucks them by removing the external case and, if needed, covering up a power pin that prevents those drives from working on a direct SATA connector.  The drives are almost all WD "White Label" disks with the same specs as Red Label NAS disks.

Prices are dropping on the lower capacity NAS disks (4TB or less).  I'm thinking of trying the shucking process if I see a nice sale on 8TB drives for my NAS device.  Then I can use the 2TB drives presently in that for the new unRAID box.   :thumbsup:

Sometimes I wish all these old drives would just crash at once, and I could refresh them with faster, larger capacity types.  I can't remember the last time I had a mechanical drive suddenly die.  Usually they present problems that worsen over a short/long time.  My Samsung 850 EVO 500GB SSD is another story.  One day, possibly right after a power outage that lasted a few minutes, it just stopped responding.  Samsung provided a warranty replacement, but still ...
« Last Edit: May 23, 2020, 01:48:41 AM by Flapp_Jackson »
"How can you diagnose someone with an obsessive-compulsive disorder
and then act as though I had some choice about barging in?"
-- Melvin Udall

robtmc

Re: How do you keep your digital music files?
« Reply #39 on: May 23, 2020, 08:50:28 AM »
n YOUR devices, such as your iPod and original iPhone. 
I cannot use Apple products...................