NikonGear'23
Gear Talk => Other => Topic started by: elsa hoffmann on February 03, 2016, 14:19:21
-
http://petapixel.com/2016/02/02/photographers-beware-seagate-slapped-with-a-class-action-suit-over-bad-hard-drives/
Seagate gets slapped with a class action suit over bad hard drives
-
These are huge failure rates.
-
That particular 3TB drive from Seagate (ST3000DM001) has earned itself a pretty bad reputation over time. I have personal experience with these failing on me, so recommend *never* to build large RAID systems with such 3TB drives. A RAID 1 (mirror) will do though. Always purchase additional drives for spares. Each time a drive fails, buy 2 equivalent drives (replacement + new spare).
The tabulated data on the linked page is slightly puzzling - what is a failure rate of 222% ??
-
Thay explained that if a disk is replaced by a new one under warranty, and it fails again is a twice failure... 200%, thought
-
So > 200% means even some of the third disks (in row) died? Then your backup system and RAID topology become hugely significant.
On the other hand, the metric is of dubious value, as *any* disk eventually fail. Thus seen over a long enough time span, the "failure" rate will go to infinity ....
-
Appreantly the 1.5GB HDDs are more prone to fail. I wonder if a 3TB HDD uses two of those sloppy 1.5G platters?
-
Currently I have more than 100 drives of the Seagate 2 TB model up and running and they behave very well.
-
Backblaze posts some interesting harddrive statistics and does it regularly: https://www.backblaze.com/blog/hard-drive-reliability-q3-2015/
-
Currently I have more than 100 drives of the Seagate 2 TB model up and running and they behave very well.
Yes, that's why I thought the 1.5TB platter was a dog.
Jørgen, thanks for the useful link.
-
I've learned that Seagate drives had failed on two of my friends. :(
Another fried of mine who is system engineer says that Seagate drives have in general been crap. I'll keep that in mind.
-
The "crap" designation has over time been applied to drives from all makers I'm afraid. IBM/Hitachi for example had a series named 'Deskstar' that pretty soon was nicknamed 'Deathstar'. I can attest to the nickname being highly apt. Western Digital and Samsung to mention two other big makers have had poor products as well.
One simply has to acknowledge the fact that any disk drive, be it HDD or SSD, from any maker, has limited longevity.
-
I currently have two of the Seagate ST31500341AS 1.5 GB drives, not mission critical but I wouldn't want to lose the data. One is online the other is a copy drive of that online one. So far they are running and get tested regularly via Disk First Aid, Disk Warrior and Onyx.
I gave up on RAID a few years ago when I had a spate of WD drives fail, they were RAID drives too. The problem with getting a matching drive kept my RAID in jeopardy. I now use Carbon Copy Cloner on a copy daily schedule to make back ups. Then if I want to change the size of one drive, no problem. After a shoot I do an immediate backup by clicking one button.
-
The question of matching drives is precisely why one has to plan ahead and buy a stack of extra drives in the first place. I keep at least 20 drives as spares at any time and always ensure there are matching replacements for my RAID arrays. Plus base them on the same size of drive if possible. Consider it an insurance premium.
-
Yeah I knew that going in but at the time drive capacity was not where I wanted it and rather than building a huge array I bailed on RAID and use what ever drives are current and big enough for my needs. Thunderbolt 2 is now my connection method and OWC make some nice drive arrays in that format. I also employ a toaster to pop in a drive to back up files and take off-site.
-
This is not a new problem. In my researches for the Commemorative Plant Names book I have come across numerous botanists and researchers who have lost 'life works' and expedition notes and drawings (pre photography) to fire and shipwreck, some are sad tales, sometimes the notes and drawings were passed to others for safe keeping (backup?) but still they were lost. Large consignments of plants and seeds lost to shipwreck and sometimes frost.
Just trying to add perspective...
-
The "crap" designation has over time been applied to drives from all makers I'm afraid. IBM/Hitachi for example had a series named 'Deskstar' that pretty soon was nicknamed 'Deathstar'. I can attest to the nickname being highly apt. Western Digital and Samsung to mention two other big makers have had poor products as well.
One simply has to acknowledge the fact that any disk drive, be it HDD or SSD, from any maker, has limited longevity.
Another friend of mine who is Finish living in Japan just told me that he fell victim of the Deathstar...
Personally I try not to use an HDD as C drive longer than about three years. I'm moving to a new Windows 10 machine now. It has an SDD as C drive, but I will still use HDDs for storage. I'll keep your advice in mind.
-
Pay heed to the fact that also SSD-drives don't last for ever .... They might have even shorter longevity than some HDDs.
-
After IBM shut down their HDD business I only used Samsung and
Western Digital for serious storage. For a game system of my son
where no data of interest are saved I might use a Hitachi or
Toshiba.
My professional Backups are all on WD green series. Never failed
in me once...
-
Pay heed to the fact that also SSD-drives don't last for ever .... They might have even shorter longevity than some HDDs.
Yes, I will. Also, I will use the SSD "C" drive only for the system, application and the data I'm working on (I always a USB memory as temporary storage).
After IBM shut down their HDD business I only used Samsung and
Western Digital for serious storage. For a game system of my son
where no data of interest are saved I might use a Hitachi or
Toshiba.
My professional Backups are all on WD green series. Never failed
in me once...
My friend system engineer says he only trusts Hitachi. Maybe the (bad) experiences vary.
WD green series seems to have been merged to the blue series.
-
All the makers take turns having their bad spells.
Right now, Hitachi is one of the "good ones".
Recently, Western Digital purchased Hitachi's HDD business, but curiously kept the HGST brand separate.
-
All companies have experienced problems with a series of harddrives. All you can hope is that the ones you buy aren't part of such a series and prepare for when the harddrive eventually fails. I use 5x 2TB Samsung drives in my NAS and they have so far been running reliably for around 5 years. I did have to firmware upgrade the harddrives shortly after getting them because Samsung found a critical bug. The NAS is just as old, so it might as well be my NAS that suddenly decides to stop working.
Flash can only be written to a limited amount of times before they stop working, but it does look like they have gotten that resonably under control:
https://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead
That doesn't mean that SSD's don't fail, they certainly do. Just like all other computer equipment :)
-
SSD 'wear levelling' algorithms get better over time, but much of the uses such drives are subjected to will sooner or later kill them no matter how cleverly the wear is distributed over the microchips inside. To wit, the OS reads and writes all those small temporary files all the time, and even if the OS is smart enough to understand disk defragmentation is a bad idea with SSD, the user might not have understood the same and do the defragmentation anyway. Or equally bad, updates some huge disk files over and over again (did I hear monolithic image library bases like LR??) which will kill the drive equally effective over time. The SSD needn't even be filled to capacity when it eventually gives up the ghost.
I' m currently looking for a replacement of a SSD drive of a highly regarded brand. The drive is not 4 years old and filled only to 30%. It is beginning to show the usual signs of a premature death with hiccups and getting awfully slow.
Jørgen: with old NAS units, the fan tends to die followed by the power supply. Not necessarily in that order. Both items can easily be replaced and in particular the fan should be switched to a truly silent-running type. Noctua fans are whisper quiet and my choice when I put the old fan out of its misery.
-
SSD 'wear levelling' algorithms get better over time, but much of the uses such drives are subjected to will sooner or later kill them no matter how cleverly the wear is distributed over the microchips inside. To wit, the OS reads and writes all those small temporary files all the time, and even if the OS is smart enough to understand disk defragmentation is a bad idea with SSD, the user might not have understood the same and do the defragmentation anyway. Or equally bad, updates some huge disk files over and over again (did I hear monolithic image library bases like LR??) which will kill the drive equally effective over time. The SSD needn't even be filled to capacity when it eventually gives up the ghost.
I' m currently looking for a replacement of a SSD drive of a highly regarded brand. The drive is not 4 years old and filled only to 30%. It is beginning to show the usual signs of a premature death with hiccups and getting awfully slow.
A lot has happened in 4 years. The development of SSD's have gone fast and continues to develop. Samsung recently showed a 16TB SSD.
Jørgen: with old NAS units, the fan tends to die followed by the power supply. Not necessarily in that order. Both items can easily be replaced and in particular the fan should be switched to a truly silent-running type. Noctua fans are whisper quiet and my choice when I put the old fan out of its misery.
Yes, it's not unlikely that either the PSU or the fan dies. When I had a desktop machine (self built), I used Noctua fans. The noise is not that big a deal though. The NAS is not in any of the rooms we spend much time in. We usually have pretty stable power, but I do have my NAS connected to an UPS, which might help in keeping the PSU in the NAS working well. The UPS actually failed a few years ago due to a swollen battery. I replaced the battery and it's been running since.
Whether the NAS eventually fails or not, I will likely replace it within the next few years.
-
I'm fully aware that SSD technology is rapidly evolving. But most of us cannot afford to be constantly on the pricey and cutting edge of technology with heavens know what undetected bugs lurking under the shiny surface. So we have to do with more mundane gear and rely on these for some years before they are replaced. I have spoken to so many people believing that now they have switched to SSD, they no longer risk a disk failure. Nothing could be more wrong.
-
I wasn't suggesting that you or everyone should be replacing gear more often or always live on the cutting edge of technology. It was merely a statement that the SSD technology has evolved a lot since whatever drive you're now experiencing problems with.
Thinking SSD's never fails is clearly a misunderstanding, but I do believe they are generally more reliable.
-
I'm fully aware that SSD technology is rapidly evolving. But most of us cannot afford to be constantly on the pricey and cutting edge of technology with heavens know what undetected bugs lurking under the shiny surface. So we have to do with more mundane gear and rely on these for some years before they are replaced. I have spoken to so many people believing that now they have switched to SSD, they no longer risk a disk failure. Nothing could be more wrong.
I hadn't used any SSD until I built my new PC this week simply because I don't want to jump on the latest technology in genearl, so far as my main PC is concerned. I always want to incorporate the tried and tested hardware or technology method. I don't blindly believe that SSD is more die-hard than HDD, but I decided that it has been tested & tried long enough to be common. I've used Mac Book Air (my mobile machine, not the main one) for a year to get the hang of SDD.
I wouldn't have bought any SDD as early as four years ago, no matter how much budged I could have had.
-
Jorgen,
based on personal experience I'd be careful to claim that SSDs are more reliable than HDs. While a total failure is indeed quite rare, performance variation with many brands is much higher than with HDs.
Bjorns comment on SSD performance degradation with relatively low usage and being very slow is an indication that the mentioned SSD is using a threshold based GC (Garbage collection) approach vs. realtime GC/WL. If the SSD controller is fast enough to do realtime GC faster than the max write rate of the SSD, write performance variation diminishes almost completely over the lifetime of the SSD.
The combination of realtime GC and faster GC than the max write rate for a particular SSD type is a necessary precondition if SSDs should be combined in larger raid configurations with predictable (write) performance. Most consumer SSDs failed in this regard when I did my analysis a while ago.
I built quite a few of those open systems for very high sustainable combined I/O and compute performance (my second hobby besides photography).
Due to cost reasons I had to use consumer SSDs. Given the high number of SSDs needed (approx 100), I spent more than 2 months to evaluate 15 different consumer brands, before choosing the right model for my use case.
Here is a picture of one of these machines. 4 sockets, 32 physical cores, 512 GB RAM, 96 SSDs
This particular configuration had applications running with very high computational intensity (measured via energy consumption) and high sustainable I/O (average write rate > 10 GB/s for a full week per run). Over the course of the year, each of the SSDs experienced a significantly higher write volume than guaranteed by the manufacturer. The drives didn't fail completely, but at the end write performance in about 10% of the total number of drives went down. (Was ok for me, as they served much longer than anticipated. Some drives had 20x the max. write volume the manufacturer listed). The other consumer drives I evaluated had much more erratic write behaviour under sustainable load.
BTW, loading a 100 GB data file from the SSDs into main memory took less than 5 seconds - a very convenient speed to work with....
(http://www.pbase.com/andrease/image/152623444/original.jpg)
Getting back on the HD failures rates and causes.
For those interested: Google published such a statistical report (http://static.googleusercontent.com/media/research.google.com/de//archive/disk_failures.pdf) (they have quite a few drives)
rgds,
Andy
-
Andy: Impressive setup!
Jorgen,
based on personal experience I'd be careful to claim that SSDs are more reliable than HDs. While a total failure is indeed quite rare, performance variation with many brands is much higher than with HDs.
I was probably not specific enough. I believe they are more reliable due to not having any moving parts, that's particularly important with devices that gets moved (laptops, tablets, smartphones, etc.).
-
I was probably not specific enough. I believe they are more reliable due to not having any moving parts, that's particularly important with devices that gets moved (laptops, tablets, smartphones, etc.).
Hi Jorgen. I agree with this perspective.
regards,
Andy
-
i have like 12 of these drives 3TB drive from Seagate (ST3000DM001) yikes..
but i use them in two RAID5 banks.. the others are secondary backup and get virtually no use other than to pop them into a thunderbolt SATA dock.
-
Basing RAID 5 arrays on unreliable disk drives is like playing Russian roulette. Even the occasional bit error on one or more of the remaining drives could wipe out the entire array when you try to rebuild after a disk crash.
You would be better off by using a RAID 6 layout. Or investing in heavy-duty NAS-dedicated drives that have much lower bit error rates.
-
i guess i am &*@#+#. kinda far into the game to change horses.
good news both arrays are not on all the time.
-
External back-up hard-drives are a bit similar to how we view death. We think it wont happen to us, just to the guys next door. Yet the evidence is there that we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.
-
Except for the trailing phrase, what you describe can be assigned probability 1.
-
according to a systems engineer (my classmate), he said that seagate's QC became shoddy when they started to make their drives in china. he uses lots of them (HDD) at home and at work at google. according to him again, the current best is hitachi (as was mentioned here in one of the posts). i currently use WD and so far so good :o :o :o
-
Andy that looks impressive!
That Google report is really interesting, thanks for posting! Nice with real statistics ;)
-
Andy that looks impressive!
I built a few of these setups and was always impresssed, what kind of performance level was possible with standard HW when systems were properly balanced for the task at hand.
Parallel is the way to go :) (photo taken in the early days of the project)
(http://www.pbase.com/andrease/image/145577975/original.jpg)
While SSD'S are preferred from a performance perspective, economics still require hard disks for large capacity servers. My home-built home server "powering our digital household", uses 24x 6TB hard disks. Special care has been taken to keep noise, overall energy consumption and temperature with the HDs low, while providing very good performance via 10 GbE connections. Wrt to HD's, most are WD drives. To check for drive health, it is recommended to take regularily a quick look in the SMART logs of the drives. If the soft error rate increases with a particular drive, retire it rather early than late.
rgds, Andy
-
BTW, loading a 100 GB data file from the SSDs into main memory took less than 5 seconds - a very convenient speed to work with....
rgds,
Andy
Impressive array and obviouly lots of time spent constructing and maintainig it. Du you sell these?
Or is it a hobby in the sense of spending time and money for the joy of it?
-
External back-up hard-drives are a bit similar to how we view death. We think it wont happen to us, just to the guys next door. Yet the evidence is there that we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.
It is a good idea to have every Bit you value on at least two different physical media in two different physical locations at every point in time. "in the cloud" is no gurantee for this, esp not if the passwords and crypto keys you need to access are the subject of your backup.
Either these are too accessible (probably alo for the unauthorized) or they are lost and with the keys your property.
The right strategy is not easy to find.
After our death what will happen?
Will our children value our electronic legacy as much as we do?
Happy man whose work is valued so high that agencies or museums will care for the assets...
-
//off topic
Frank,
it was a hobby. Kind of "is it possible?" questions to go for, or rather a "it can be done" approach.
It is not related to photography, but a few idea's:
One machine had faster I/O performance at application level as the largest mainframe at that time.
One machine surpassed the performance of a cluster with 432 servers and 2400 disk drives which established a world record in a particular field 5 years earlier.
A set of machines equalled the compute performance of the leading system in the Top 500 list 11 years earlier (without spending the 120 mio USD of this super computer :) )
Submitted a paper for a particular energy efficient workload in a contest - still leading in this category.
... a few other competitions ....
It was insightful, I learned a lot and it was fun.
Most components are used by my son now. He studies physics and computer science and has ample ideas to use the systems for.
rgds, Andy
another setup. CPU's are water cooled, lots of GPU compute power
(http://www.pbase.com/andrease/image/150099632/original.jpg)
-
You aren't involved in the password cracking business, are you ??
Interesting to know what OS you run on these mighty machines, by the way.
-
You aren't involved in the password cracking business, are you ??
Interesting to know what OS you run on these mighty machines, by the way.
Oh no - no password cracking :D (BTW, password crackers usually use AMD GPU's, not NVidia. AMD does support bit operations better)
My background is computer science and I am fascinated by all things parallel and high performance. Especially GPGPU (Using GPUs for "general purpose" computing) with its peculiarities in programming is an area of personal interest.
For those interested: hgpu.org (http://hgpu.org/)is a good place to find thousands of scientific papers where GPUs were used for particular research interests.
It is relatively easy to write programs for GPUs. It is very hard to write for best possible performance. At least for me, but it is an interesting endeavour to keep all transistors spinning.
32.384 GPU cores, approx 16 TFlop/s DP
(http://www.pbase.com/andrease/image/151806042/original.jpg)
wrt to your OS question: Win10 Pro, WinServer 2012/R2/2016, Ubuntu, Debian
The OS is less of an issue, as both WIndows and Linux can deal with NUMA architectures quite well. Yet, it is very rare to find applications which utilize the power of dual and quad socket machines. Especially with distributed I/O architectures modern Xeon's have.
rgds,
Andy
-
External back-up hard-drives are a bit similar to how we view death.
.....we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.
. "in the cloud" is no gurantee for this,
Frank, I read Elsa's comment as black humour - when we die and are up somewhere "in the cloud(s)" our memories will be there too.
My advice to anyone's kids - if your parents had outstanding or valuable photos, make sure you help to keep them safely stored somewhere :)
-
Backblaze has posted their stats for 2015: http://arstechnica.co.uk/information-technology/2016/02/hgst-hard-disks-still-super-reliable-seagates-have-greatly-improved/
-
fascinating discussion, and really excellent hardware pr0n.
ironically, I have always assumed hard drives will fail - but the only ones that I've ever had fail are backup drives.
admittedly, my current setup is rather simple by comparison. a pair of toshiba 5TB drives internally in a raid zero array, backed up daily to one of a pair of external 8TB drives, which alternate locations each week. an easy system for a modest amount of data - worst case from a computer failure is the loss of a day's change, worst case from a fire, flood, theft, earthquake, etc, is a week's worth.
before his retirement many (20ish) years ago, my father ran the company that supplied hard drive heads to most of the drive manufacturers of the day (seagate was among the few which made their own, if I recall.) he had many very entertaining tales of the generally shoddy and unprofessional nature of most of their customers with regard to the "consumer" category of product.