This has been a thing ever since external hard drives came out. But it's important to know that shucking a drive could void your warranty. It's not entirely clear though and also depends on the manufacturer. I've read threads where people said WD will still honor the warranty for shucked drives and other threads that said they refused. Also, depending on where you live you might be legally guaranteed a warranty.
Comment has been collapsed.
Yeah but maybe there are still a few that don't know or like me found it scary.
Thanks, well i got about 15 WD external drives and always had WD internals too and knock on wood never had to use their warranty, and offcourse hope to keep it that way, but this is good information to know.
I am from the netherlands usually things would be legally set i think.
Comment has been collapsed.
I've been doing this for some time to replace disks on my NAS, otherwise it's very expensive. Regarding the warranty, you always want to make all sort of tests to the disk before shucking it. Basic tests would be a short SMART test, a complete surface read and a long SMART test. Advanced tests include reading and writing the disk multiple times while seeking random sectors (a final long SMART test is recommended), this will stress the disk and if it's one prone to failing it might make it fail, It's usually called a "preclear" test and it can take quite some time (days on big disks).
You might also want to read about CMR and SMR hard drives and avoid the smaller disks (less than 10-12TB) because they're all now SMR and, TLDR; SMR disks suck, especially in NAS environments.
If you need lots of space and don't want to spend a fortune, shucking+NAS+RAID(for securityredundancy -don't forget backups-) is the way to go.
Comment has been collapsed.
Thanks maybe it was wise to do a test before shucking mine yeah, but usually they (WD Elements) been good to me and without backup (knock on wood). The pc is in the bedroom and i am sensitive to sound so having the pc on for days is no option.
I got like 15 external drives ranging from 2tb to 18tb because i started buying externals many years ago.
This is the first time i got an 18tb one and the first i been shucking, luckily enough it came with 60 euro discount so it costed me 300 euro.
I don't know anything about CMR or SMR or what that is about?
I got 8 bays in my pc case. although my motherboard allows only 5 to be connected at 1 time, i haven't really looked into a NAS but just the amount of space (i discovered unexpectedly), i would need up to 50-100tb easily (and that's without a backup). And something that was definitely not in any budget (i need to have been saving for furniture and stuff for my own place like 7 months ago)..
I will probably do it in baby steps, maybe atleast get the ones in my pc case replaced with f.e 18tb drives, that will be enough for a while and then if it ever gets financially better think about NAS etc. Backups would even be a bigger fortune i know.
Comment has been collapsed.
CMR is the traditional technology.
SMR is a new one, more efficiento to store more information in the same disk space (so it's cheaper to manufacture) but with the disadvantage that at the end of the day it's slower.
In short burst it's not noticeable (they say) but in big files... disaster.
The site I took reference from
Comment has been collapsed.
Thanks, can you define big files, how big?
Sigh so far its not recognizing the drive (yeah also not in disk management, so i can't initalize/format). I tried a sata connection from a working drive, but no cigar either, it doesn't seem to get power, i am not hearing anything. :(
I got 8 bays so the sata wiring is a bit messy.
Putting the usb connection back on then it does seem to show up (thankfully).
Comment has been collapsed.
No, i just actually saw a comment in that youtube clip i posted, i need to replay the whole video to know what he's talking about, and what pins, i think that is it.
Comment has been collapsed.
Yeah make sure you use Kapton tape. Hopefully it should work after you do that.
Comment has been collapsed.
I don't have that handy, wouldn't scotch tape work? It's some fickle work i must say.
Nope scotch tape doesn't seem to stick well.
Comment has been collapsed.
Kapton is used for this because it is non-conductive, can handle high temps, and won’t leave sticky residue like black electrical tape. I bought a roll for taping one tiny pin on my HDD, so I know it’s annoying having to buy a whole roll for that.
Comment has been collapsed.
I tried a think called masking tape i had laying around, but again the drive is not recognized.
You are right about high temperatures i didnt think about.
Comment has been collapsed.
Hmm other tape is not ideal, but it should have worked. I think it’s only the third pin that needs to be taped, but it’s ok if it covers the first two as well. Don’t let it cover any other pins. You can look up 3.3v pin issue for more info
Comment has been collapsed.
I plugged the sata out, found out it pushed away some tape on the third pin (why it probably didn't work) and i wouldn't have been feeling secure with some tape that wasn't good anyway, my case is very cool, but harddrives can still get hot.
I ordered some Kapton tape for 10 euro, and it will arrive tommorow, it's just as easy. :)
Thanks for the suggestion.
Comment has been collapsed.
Well i got the tape, but it didn't work, it doesn't want to stick (well everything but the pins), i still can't get it to power up unless with 18tb it's even more different?
When i unplug the sata drive each time the tape comes with it.
Comment has been collapsed.
Did you tape all three 3.3V pins or just the third? Because you can do all three if it feels more stable.
Do you hear it spin up in your PC or does it spin up but is just not detected? If it doesn’t spin up at all, it is probably the 3.3V issue. If it does, it could be something else like your BIOS settings. Make sure you’ve selected AHCI in BIOS and that you're using UEFI instead of Legacy.
If you use Reddit, I would ask the folks at r/datahoarder. They are very knowledgeable and should be able to help you with any HDD issue.
Comment has been collapsed.
All pins but it just does not want to stick properly, same with the masking tape which i feel is more stickier, but everytime you plug out the sata cable the tape moves, especially and exactly from that third pin, the other 2 pins stay fairly covered, and thus it probably also not properly taped when you plug it in.
And even being able to put such a small piece of tape on 3 small pins is not an easy task either i must say.
No it does not spin in the pc, it's completely silent, i think it needs to atleast spin and power up before even trying a bios, but i don't think it would even be the issue, it just need to get power, somehow (and yeah probably the pins).
There is the molex option devotee posted below, but using a wrong one could actually be very bad, but there is no sticker saying you have a good one or not..
Comment has been collapsed.
Yeah I would not try the molex adapter. I’ve heard they can be a fire hazard. Some people just completely remove that pin, which works, but I would not recommend it because it definitely voids your warranty. I would keep trying the Kapton. It’s very fiddly I know! If it still doesn’t work I’d check out r/datahoarder. They’ve helped me before.
Comment has been collapsed.
Thanks, will do. And yeah i thought about removing the pin, but i kinda find that drastic too, if something wrong breaks, it's 300 euro..
Comment has been collapsed.
Yay, preserverence wins, i tried some Kapton on the pins itself and then masking tape over it for the sturdyness (not sure how it would work heatwise but i gonna just trust on that), and it worked. :D
Comment has been collapsed.
If you have any SATA power cable extender around you can try this (at least to discard a problem with the drive):
Also, most Molex-to-SATA power cable adaptors seem to fix the issue.
Comment has been collapsed.
I don't think i have such a power cable extender, that molex to sata seems familiar i might have that.
But i got 1 bedroom (luckily not small but i propped 35 computers in my closets and a small portion for some clothes. As long as i don't touch things they are fine without falling). :p
Might as well just wait on the order of the tape to come in, if for some reason that's still not working i can try to look for the molex thing.
And a whole nightdresser stand just with 1 usb cable and 1 adapter for each of the 15 external drives i got, i probably should get rid of them, not like i have them all on at the same time, maybe 2 at a time tops.
Oooh and labeling adapters someday, a charger for a steamdeck, for a switch, for a phone, the external drives, i got adapters i don't even know what they are for anymore.
Comment has been collapsed.
Why? I have some HDD that I took from external cases and I haven't had to tape any pins.
Do they have something extra now?
Comment has been collapsed.
Maybe it depends on the model of external HDD and your PSU connections, but many of them just won’t power on inside your PC unless a certain pin is taped. I had to for my 10TB WD drive a few years ago.
Comment has been collapsed.
I don't know but perhaps anything greater than the drive's internal cache if there is no time to empty it before.
https://www.reichelt.com/magazin/en/guide/smr-cmr-which-hard-drive-is-best-for-which-purpose/
Comment has been collapsed.
just for refernce:
if one has a linux device laying around, this script is great for doing the initial testing:
Comment has been collapsed.
Thanks for the link, it's very similar but way simpler than the one I use. However, what I use is a script for UnRAID that I have modified over the years:
The main differences with the script you pointed to is that it can do multiple cycles and it also keeps the disk head moving between random sectors while it reads/writes/reads and compares the whole disk, so it puts a lot of stress on the mechanical parts of the disk.
On Windows, you can use something like Smartmontools (for SMART tests) and HD Tune (for surface checks). There's also a neat program called Hard Sentinel that does it all for you and also keeps track of your disks' health.
Comment has been collapsed.
is there a non unraid version available somewhere as well? Next build stage for the homelab will have a proper zfs system where I'll have to prepare "a couple disks" ;) but the prep machines will be linux...
Comment has been collapsed.
Same here, not a single problem with external WD drives in years. Can't say the same about external Seagate drives, though.
Another benefit of shucking drives (besides the noise and the poor reliability of USB connections) is heat. External hard drives tend to get VERY hot. Again, I saw that on a couple of Seagate Backup Plus Hub external drives with very poor cooling. You just have to compare their temperatures while in the USB enclosure and while used as internal disks in a pc case to see what a difference it makes.
Also, USB communication is not as ideal as a direct SATA connection. Under high loads, external drives tend to choke. Try to use them to store and seed torrents and you'll see what I mean. Anyway, I guess they're meant for storage.
Regarding NAS, it's just a name. You have fancy All-In-One NAS devices that are small and compact (and expensive and not upgradeable). My "NAS" is a big pc case (Fractal Design Define R5) with 8 full size bays (3.5" or 2.5" drives) and two 2.5" extra spaces on the side, with an IBM Megaraid SATA / SAS Controller 6Gb/s 8 ports controller and running Debian Linux with LVM and RAID.
Cons of it? It's big. Pros? It can hold up to 8 big drives, I can put any motherboard/cpu/ram combo inside (from inexpensive to insanely pricy), I can upgrade it wheneverI want and I'm not limited to a "proprietary" OS/Firmware that might stop receiving updates because the company that makes the NAS decided that you expensive device just reached its EOL (support-wise).
Comment has been collapsed.
That's how I upgraded my PS4 to a 2TB HDD, by using an external HDD. Best bang for the buck. Never had any issues..
Comment has been collapsed.
Yeah, found this out when my dad gave me his old external HDD to fix.. He had broken the USB socket or whatever its called.. Torn it open as I saw its broken and nothing can be done and turns out its just your regular internal hard drive inside with adapter connected to it ... Took off the adapter, stuck it in my PC
But didn't know this process actually had a name. I just did it because that one physically had a broken port and warranty had ended and I wanted to see if it works
Comment has been collapsed.
and turns out its just your regular internal hard drive inside with adapter connected to it ... Took off the adapter, stuck it in my PC
Just as a side note, this won't work with WD My Book drives because the USB controller (the USB case) cyphers all the data on the disk. Yes, you still can connect it to your PC, format it and use it as a regular internal hard drive but you can't recover the data it contained (while inside the My Book enclosure).
WD 2.5" external USB drives also have their own USB connector soldered to the hard drive, they don't have an standard SATA connector, so they can't be shucked and, if you have problems with the drive, data recovery is a mess :(
Comment has been collapsed.
Ah, good to know. Thanks
Yeah I got lucky then - I could access the data and worked fine.. Still does actually after several years
Comment has been collapsed.
Hi, really mybook always cyphers all data when in case even if i never used any of its software or set passwords and when I formatted it after buying? I didn't know that and i bought it knowing there is regular internal disk inside which could be possible to use if something happen to usb/power supply and now it looks like it would lose all the data? So what would be best option if usb/power would stop to work and i need copy/recover my data? Lugum always bought wd elements, not my book?
edit: luckly it looks like my model is not using this if i didn't set password, my model is this first from table here: https://support-en.wd.com/app/answers/detailweb/a_id/1837 (Readable w/o
WD Enclosure - yes). But i still have older external drives in which shucking can't work, what kind of usb case would be best option as a replacement if usb fail? (to not lose access to data).
Comment has been collapsed.
"noisy and i believe it hurts the drives if you upload to multiple connections"
is it? maybe when use those external one with USB powered only
Comment has been collapsed.
Maybe it doesn't, if it's basically the same as an internal (it did even say internal on the one 18tb i shucked) then it should just handle multiple connections right? It just was always a thought in my head. Still you can't beat the noise though, WD sounds are decent, but i got 1 seagate and believe a hitachi too and boy they will drive you crazy after some hours.
Comment has been collapsed.
afaik so far only older drives do that, maybe that
Comment has been collapsed.
Nah seagate is 3-5 years old, i can't remember but it wasn't that old.
But i am just very sensitive to noise, i even struggle sleeping with a fan on (which is nice in these warm days...) i sleep with cotton balls in my ears half the time.
Comment has been collapsed.
I'm not sure about it, but I was told that there are actually worse drives put in external cases. At least in the ones by internal drive manufacturers (WD, Seagate, Toshiba). They are cheaper for a reason. Quality control detects potential issue, like some bad sectors a little beyond normal threshold, the drive goes to external case and is sold cheaper. The assumption is that external drives aren't usually on all the time, and because of less use, the potential issue won't reach catastrophic levels during warranty time.
I was told that you can only assume that 100% good drives go into external cases only in some brands, for example LaCie, Sandisk.
Comment has been collapsed.
varies by manufacturer and the times. for a while, shucking a particular external drive from a certain big box retailer in the USA was about a 70/30 chance of getting a highly-rated NAS drive for much less cost than buying that same drive normally. that was pre-COVID days though, by a few years, and the same external drive now houses a "white label" drive that may or may not be the same NAS drive (no one seems to know for certain)
(reading your reply closer, I see you're mentioning the possible inherent differences in the same model drive. and you could be right. I don't know.)
Comment has been collapsed.
I remember reading some time ago that some vendors like to mess with controllers and firmware of specific models of external HDDs making them unusable without the shuck. Can't recall what models exactly, but I remember it was about WD and Seagate. Didn't pay too much attention so can't even tell if this was true or not.
Personally I shucked 2 externals Seagate drives to replace my failed one, but turned out both were the same, very problematic model (st2000dm001) and thus died not so long after :D
Comment has been collapsed.
An idea is to buy internal hard drives and use an HDD docking station.
Comment has been collapsed.
That's just the opposite of what we're discussing here. If you check prices for internal drives vs external USB drives you'll see the huge price differences. a 14TB internal hard drive can cost between 300 and 500 euros. You can purchase a 14TB WD Elements external USB drive for around 200€ (they usually go on sale).
Using an HDD docking station still gives the problems with the USB interface (unless you use a SATA or eSATA docking station), plus the hard drive will always be cooler inside the pc case (if you did your cooling right, ofc). The only advantage of buying an internal drive is that you know exactly what you're buying, with external USB drives you are somehow "gambling".
However, you can always buy it on Amazon, use any SMART utility to check which hard drive model is inside the USB enclosure and, if you're not happy with what you got, you can return it. One of the many wonders of Amazon service. Usually, there's no need to open the USB enclosure to know which drive model is inside.
Comment has been collapsed.
After posting I realized I'm off topic, sorry about that. I agree with what you say, but with the docking station, when you need to transfer files you turn it on, so cooling the hard drive is not necessary, something that Lugum mentions at the beginning of his post.
Comment has been collapsed.
No worries, I do use HDD docking stations myself to connect old hard drives I have around. I don't like very much to turn on/off hard drives too much, electronics tend to fail when you do that. You can leave a computer (or a hard drive) running for years 24/7 and it won't fail, but you might turn it off one day and realize that it decided to stop working when you try to turn it on again. I've seen Linux servers last for years working without interruptions and office computers (with similar hardware) that used to be switched on and off every day (at the beginning and the end of work) die way too soon. That's also true for that "stand-by" feature external drives tend to have (they "hibernate" after some time).
Comment has been collapsed.
Personally, when I buy HDD, I get from, let's say, good series (WD Purple or Blue), to at least feel safe about my data (it's placebo, I know...).But so far no issue appeared. And also, I try as few on/offs on them as possible.
Comment has been collapsed.
I got a 200 euro 25kg Darkpro pc case, with Noctua fan, my pc is relatively cool even with last weeks heatwave.
What i mentioned is i will gonna need dozens of TB's, i could replace 5-6 disks in my pc case, wether if/when i get more (if money allows) i could then think about a docking station or nas, but nas seems expensive and usually limited to it's drive space.
But if i go for a docking station how would i store the extra internal drives safely (i don't know how sensitive they are to dust etc) that would be my issue.
Comment has been collapsed.
how would i store the extra internal drives safely (i don't know how sensitive they are to dust etc) that would be my issue.
I don't think they're specially sensitive to dust, I would say water/humidity/moisture and, of course, accidentally dropping them are worse than dust.
You might want to look into the various types of storage cases that are available:
https://www.amazon.com/ORICO-Protective-Anti-Static-Shockproof-Dustproof/dp/B009HIZKGK
I've used the plastic ones (last picture) in the past but depending of the manufacturer they're very tight and once you put the drive inside it's hard to take it out :/
But, meh, you might just want to reuse their old WD Elements USB enclosures to store them (the enclosure should be compatible with any drive). Since they have a nice form factor (like a fat book) they can be conveniently stored on shelves.
Comment has been collapsed.
Thanks, they seem relatively cheap. Yeah but wd enclosures are very big, especially if you got about 15 it fills almost half a shelf, they also don't stack very well. I think i might keep the smaller ones for something else though with enclosure, maybe media harddrives or something. I would only shuck big drives like you said.
And yeah if you see some people's pc cases, i guess they could handle some dust. :p
Comment has been collapsed.
Well you could still shuck and put them in a docking station. I got room for about 5-6 in my pc case which is cool even in a heatwave.
When the time comes i need more i have to decide for a nas or docking station, but nas is usually expensive and limited to drive space.
I tried a docking station once where i worked and i must say it was fairly quiet, much more then an external drive, might depend on brand, all i know it was a fairly cheap brand.
The only downside there is all the drives i would have that would not fit in my pc case, how would i keep them stored safely?
I would have the need for some drives to download/upload, those will be enough that rest in my pc case, mostly the rest is for permanent storage and i would barely touch and have them turned on much.
Comment has been collapsed.
Note that hard drives are designed to run hot since the oils for the berrings are rated for 30-40C. Too cold and the oils won't do it's job. I'd check the temps with a smart utility.
Also, I store my unused drives in a drawer... with a cloth under and over to they don't get damaged. The only thing is cheap drives have a live span... even unused. They use scealant to close the drive and once it goes bad the drive will heat up like crasy and it won't be reliable.
For a NAS... well I've purposely built mine for 300$ in a cheap PC case, but you can use anything with Freenas.... i've built mine years ago and wont ever go back... having 2 parity drive if one drive goes bad is such insurance.
Comment has been collapsed.
To have the files of the hard drive accessible at all times, then only inside the PC case. The docking station is suitable for backup.
The one I use takes 2 SATA drives 2.5" or 3.5" with no capacity size limitation, and it was relatively cheap.
When you say it won't fit in your case, what do you mean? If your case has 6 slots for 3.5" and you want to put 2.5" somewhere, then there are some components, called Brackets. I don't know if I'm helping.
Comment has been collapsed.
Well, I read in the two first comments that there was like 15 external drives... so it could be either not enough space or ports. I'd go for both... if it's a standard pc case.
Not sure investing in a SAS adapter to add more ports would be advisable on standard pc hardware.
Comment has been collapsed.
The benefits of reading! I'm always in a hurry, and I don't have time to read...
With so many hard drives, I can't help, because I've never used so many at the same time. The price is prohibitive for me.
Comment has been collapsed.
No prob, I've read the thread lightly too. I've got only six drives in Z2... went with 3tb years ago since price was high... but thinking back at the price I paid (like 5 years ago) it's almost the same with the markup today.
Comment has been collapsed.
It's actually cheaper when you pay little by little :)
Expensive investments, I believe, also require significant amounts for maintenance or in case anything happens.
Comment has been collapsed.
Well a bunch of those wd externals are 2-4tb because i bought them so early, apparently they aren't of use in my pc anyway or shuck them, i might use them for mediabox drives, or even one for the WII i guess, or even backup i never did..
I can only hook up 6 sata connections at a time to my motherboard (I already lose 2 because of a M2 is hooked up too) that is the main issue.
And i got 4 usb ports in the front, but one is used by a controller, 1 by a printer (short cable) and sometimes i use one to charge my phone.
I got 4 in the back, but i rarely use them (and as i said connecting to usb is rather noisy) i spend extra on a quiet case, quiet GPU, all on purpose for that.
Maybe i could look at another motherboard, i dunno how many maximum connections are possible even, but computers used to be easy to replace hardware with, this time i had it build by the store, and honestly i have no clue how i would replace some parts in the future, like a cpu too, it's so tightly build in all, also having such a big Noctua fan, the 7 bays, even when it's a big case, it's still already pretty build full to have extra's.
It's not like all drives are permanently on or even a few times, some data can be put on a disk, and then rest on a shelve for that very rare moment i do need it (it's an archiving/collecting thing) the few that could fit in my pc will be enough for downloading/uploading.
Comment has been collapsed.
You're not gonna find a consumer motherboard with more than 6 sata connexion. Nice case, BTW... but the specs says only 5 HDD... so basically you're stuck with 4 drives.
To have all those drives running (the best case) would be to use a SAS card since you'll be able to use a splitter to plug multiple drives per connector.
Depending if you're willing to keep this setup a long time... i'd probably get a old used pc and plug 'em all in there and access 'em by the network... or ISCSI.
Having drives on a shelf with data isn't something i'd recommand... i've been this route and lost data because a sector got corrupt or a bit flipped and the data got unusable. It's always frustrating to plug a disk to findout the file cannot be recovered.
Comment has been collapsed.
Thanks.
That's what i expected that there would be no such motherboard. It's a very nice case, but boy heavy at 25kg, not something you just easily pick up to clean like i did with my previous pc's. And it does what it advertizes, it gives a very quiet pc.
I don't need to have 12-15 drives running all at the same time, if i replace the 4-5 in my pc case to 18tb per drive, that's 90tb that would be more then enough. The other ones i would just store and perhaps grab it the rare occasion i need it, but for the rest it can be put away as storage.
I do have some old pc laying around (dunno what i gutted out) but electricity (and gas) prices are going through the roof right now, it probably already costing a bunch to have this one on for about 16 hours a day and then i got a 8700k cpu and 2080ti, when there are much more power consuming ones. Why i don't feel the need to upgrade for a new cpu for a long time.
That sucks, especially if it's always important data like pictures or work stuff. Devotee suggested plastic cases, i can assume they will be save in there? Just not stack too many on eachother i bet, because not to put too much weight on them.
Comment has been collapsed.
Okay. Yeah Devotee's plastic cases seems nice... but it's mainly for them to be more or less intact after a fall. I doubt there's much padding inside the case so it might not absob much of the fall damage... for 20$ it seems a little overkill. I'd just put 'em on a shelve of a library or in a drawer it's cheaper and there's not much risk of them falling.
90tb with 5 drives yeah that's a lot of storage... but no parity... if one drive fails the whole drive is lost. Depending if the data is important i'd consider adding a parity drive... event if it reduce the capacity by 18tb.
Comment has been collapsed.
It was 5 per $20 i believe unless i misread it, $20 for 1 is a lot, $4 per 1, is mweh okay.
Yeah but of the 90tb i could easilly get them all filled, i could even go higher then that, everything will be important, it's probably data i could get again if something does fail but i am sure others won't be happy with it, being so much data.
As i said elsewhere i should have been saving for furniture and stuff, this kinda came unexpected, it's not even wise to get the 18tb drives for what i am doing now, but i can't help myself there.
But parity might be an idea, i think i will go till 50tb for the moment to gain in data, then have 40tb free for the time being and then see what the future brings. What i want doesn't run away (i hope).
Comment has been collapsed.
Yes sorry, I didn't see that it was for 5 case.
Anyway think about it.
A little add-on for parity. Note that, I've seen someone suggested UnRaid... which would probably more sense than FreeNas if you go for a Small PC with Paruty Raid... since ZFS (FreeNas) use preallocated spaces for parity and UnRaid has a custom mode that place parity one 1 disk for all parity instead of distributing it. The only requierments for parity on UnRaid is the parity disk need to be equal or bigger than the biggest disk in the array... meaning if you have 5 disk ( 5tb + 10tb + 18tb + 12tb + 18tb). The parity will be one of the two 18tb... but the other disk size doesn't matter... and disk can be added on the fly (if i remember correctly).
Comment has been collapsed.
and disk can be added on the fly (if i remember correctly).
I saw one neat solution, I don't remember if it was unRAID, FreeNAS or QNAP/Sinology, I do remember it was "proprietary". You could add new drives to a "RAID" on the fly, you could even replace one of them with a bigger drive. Below the surface it was just a clever use of LVM and RAID, so nothing you can't do on a Linux system (you'd be missing the shiny and easy interface).
That was a real advantage because if you build a RAID5 of 5x18TB (72TB total) it's very difficult to "grow" it in the future without having to do some juggling (¿how do you easily migrate 72TB to a new RAID?). That's even worse on NAS with limited drive bays (2/4).
With the method you're referring to, it's just as easy as adding new disks, or replacing your old disks one by one with bigger ones. Everything is done automatically in the background. One of the drawbacks was that every time you make a change, the RAID has to be rebuilt, meaning it takes a lot of time on big storages and there's a risk of something going wrong if one of the old drives decide to die at that particular moment.
But, as I said, when I investigated about it I thought it was a very clever way of combining LVM/RAID/JBOD to make these storages more flexible and upgradeable.
Regarding ZFS, it's a very cool filesystem but I thought it required 1GB of memory per TB of storage (or at least it was recommended) :/
PS: It seems that unRAID is the one with "auto grow" feature:
https://www.youtube.com/watch?v=-bEM5oJKfSM
https://www.youtube.com/watch?v=DZHv6ImVQ9k
https://youtu.be/aXsRIrC5bjg?t=468
And here's a guide on how to replicate that on a Linux system:
Comment has been collapsed.
What you're talking about is QNAP/Sinology. They "made" a hybrid system... a software raid with Btrfs on top of it. It has the advantages of having software raid with disk... but the advantages of BTRFS parity... since raid parity is not too great. They use Btrfs for realtime parity and Software Raid for managing the raid... which is cool in a way, but break Btrfs in my own opinion.
UnRaid uses a custom Raid 4 mode... I'm not sure I'd trust that without extensive testing.
ZFS (what I use on trains (FreeNas being their old name)) is another beast all together... it's targeted at enterprise made by Sun. It's expandable, but extremely unfriendly... as you can only grow with the same thing. Not of a problem for enterprise, but consumer, it's a hard limitation to swallow... but I did choose this path.
LVM and Software Raid are interesting, but doesn't match the features of BTRFS or ZFS. Raid cannot be trusted for corruption like them.... you can do the tests as I did with VMs... if you create a Raid 5 and corrupt a disk,.. The raid will not see the corrupt data and will serve it to you unless you do a parity check. ZFS and Btrfs do a parity check for each block read... so in the event the data is corrupted on one drive, you'll see the Checksum counter go up and the correct data will be written back to the failed disk... if it's still usable if not it'll be ejected.
Btrfs is great as it allows to convert a Raid5 to a Raid6 or Raid0... on the fly, like the old Raid system, but Raid5/6 aren't stable... especially with compression... so for real data you have to use Raid 1... which sucks. I did like the file deduplication with is zero cost on Btrfs
ZFS isn't that much demanding... it uses all available ram to create a cache... I had 6*3tb drive on 8 gig of ram... with VM, jails and ISCSI. It was running fine, but I mean I dont have like 30 users on it coping file on NFS and SMBFS. I did endup raising it to 16gig of ram to have my Windows 11 VM.
ZFS isn't for everyone, but I do like the failsafe that both ZFS and Btrfs have is realtime parity. I'd have to check if UnRaid does realtime parity check... if it's like linux raid... it's not and that in my book is a deal breaker.
Comment has been collapsed.
Sorry for the dual Reply, but here's the sonology page I was refering to (search of Btrfs file self-healing) : https://www.synology.com/en-us/dsm/Btrfs
It says they use Btrfs over Raid x (Btrfs in single mode which isn't really recommanded on Btrfs). They kinda did a custom raid / btrfs crossover in their code since this combinaison wouldn't work out of the box.
Comment has been collapsed.
Thanks a lot for your replies! I agree that ZFS is not for everyone, I use it on my ProxMox server and I still don't fully understand it...
I've been looking a few unRAID videos and I think I'm going to try it on a test machine I have, I do like how you can throw in drives and grow your storage very easily. The only condition is that your parity drive(s) must be at least as big (or bigger) as your biggest data drive. Seems interesting when you want to recycle old drives you have around.
I think it uses XFS by default (Btrfs is available too), it would be ideal if it used ZFS (because of the extra security you get with it) but it seems that the whole unRAID approach makes ZFS not viable (I saw a few videos about using ZFS on unRAID but they are quite experimental). You can't have everything, I guess.
Comment has been collapsed.
Can't say for ProxMox as I haven't used it... but it does seems good a good UI. I think it does both ZFS and Btrfs with modules.
As for unRaid, well I'll just say don't use Btrfs or ZFS over unRaid... since you'll have dual parity... unRaid custom Raid 4 and Btrfs / ZFS. Btrfs and ZFS are designed to handle bare disk not being on top of a virtual drive.
Btrfs is somewhat comparable to ZFS for features... but it's highly unstable for Raid 5/6. Do keep in mind that when you format a pool with ZFS (a pool being a array of disk) you'll loose about 12% of free space for metadata.
The main advantages of theses file system is Copy On Write... that's incredible. I sometime need to test something on a bunch of files... just clone and try on the clone... the clone won't take any space only the delta for the modifications. I do the same for the VMs... create a base one... clone it 5 times... the 5 VMs then only take the delta of what was installed.
Now when i need extra storage for my Windows Rig, i use ISCSI... just create a ZFS ZVol and export it... it'll be like having a new drive plugged on a internal port... but it'll be over Network. You can resize at will... so i'm using this instead of adding new disks. The data is protected by parity and compressed with zstd on the server, not my pc... so it's a win / win but... network is slower than having a sata drive... but I don't mind.
Comment has been collapsed.
That i only got 6 sata connections on the motherboard, but somehow using a M2 makes 1 void so there i lose 2 and thus can only connect 4 other harddrives at one time. I looked at a sata adapter at one point but i don't know if that's an option or if there are other options.
This is my case. https://www.bequiet.com/en/case/1472
Comment has been collapsed.
For backups and permanent storage, if you need lots of HDDs, you can check DAS systems. They are like NAS, but without the 'network' thing and thus usually noticeably cheaper.
But then again, if you are a handy person, you can build your own DAS/NAS using regular PC components.
Comment has been collapsed.
Never heard of that thanks, it depends on being handy on what it is, what regular pc components are we talking about?
Comment has been collapsed.
Depends. For NAS it's pretty much the regular PC but with power-efficient (=low-end) components (e.g. Celeron). Plus some software like FreeNAS/Windows Server/etc.
DAS is essentially just HDDs with PSU and sata hub plus HBA controller.
Obviously that's just the gist of it but you get the idea.
Comment has been collapsed.
I have used WD shucked drives on an LSI SAS HBA card (old one, $20 or so on ebay) in my computer & a couple Synology NAS units and never had to mess with taping on the pins. But I know it's common when hooking up to standard motherboard SATA plugs.
So far after about 15 WD shucked drives over the past few years, ranging from 8TB to 14TB each, I had 1 fail spontaneously (no longer spins up) after about a year. It was part of a RAID array so no data was lost. Hooked it to a USB adapter for fun and it would spin up about one out of every 10 power cycles. No bad sectors or errors on the disk itself but into the recycling pile it goes anyway...
Had bad experience with Seagates dying in high numbers a few years ago between home and work, so I personally avoid those.
Comment has been collapsed.
Interreting, I've avoided WD for Seagates for years since all WD I had died like after 18 months. I've had my Seagates for over 5 years... and most of them are still fine.
Maybe it depends where those drives are shipped from... in Quebec I didn't have much luck with WD.
Comment has been collapsed.
I feel like they are probably all about the same given a large enough sample size. It's like if I go to a restaurant and get a bad meal, I don't care if 1000 other people got a good meal, I will not want to go back. :)
With proper redundancy and backups it really doesn't matter what brand, get the cheapest and replace as needed!
Comment has been collapsed.
Sorry i had to. https://youtu.be/-q4LjWvGft8?t=224
Comment has been collapsed.
IME external drives are not held to the same QA standards as internal drives. Even in cases where the drive in the case is the same model as one you can buy to be an internal drive, the one sold as an external drive is usually of poorer quality. Haven't had a shucked drive make it a full year yet.
Comment has been collapsed.
What brand, and how many? That alone makes a difference, it seems WD elements hold good ones.
The place i know of people have even over 100TB and it seems they all do it, and they probably won't do it without reason.
Comment has been collapsed.
One Seagate, three WDs, and an uncountable number of Toshibas.
Comment has been collapsed.
I am not happy with leaving externals plugged in all day (noisy and i believe it hurts the drives if you upload to multiple connections) but i was told shucking is the thing. Removing the cases and then use them as internal ones.
Now why would you want to do that? Well because in general externals are cheaper, and there are good drives inside, and it also helps if your usb connection got loose.
https://www.youtube.com/watch?v=tfSHzozwIF0 I got 2 left hands and even i managed to do it.
Comment has been collapsed.