Seagate Drive failures - class action

[Bjørn Rørslett]

Basing RAID 5 arrays on unreliable disk drives is like playing Russian roulette. Even the occasional bit error on one or more of the remaining drives could wipe out the entire array when you try to rebuild after a disk crash. 

You would be  better off by using a RAID 6 layout. Or investing in heavy-duty NAS-dedicated drives that have much lower bit error rates.

[bobfriedman]

i guess i am &*@#+#.  kinda far into the game to change horses. 

good news both arrays are not on all the time.

[elsa hoffmann]

External back-up hard-drives are a bit similar to how we view death. We think it wont happen to us, just to the guys next door. Yet the evidence is there that we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.

[Bjørn Rørslett]

Except for the trailing phrase, what you describe can be assigned probability 1.

[richardHaw]

according to a systems engineer (my classmate), he said that seagate's QC became shoddy when they started to make their drives in china. he uses lots of them (HDD) at home and at work at google. according to him again, the current best is hitachi (as was mentioned here in one of the posts). i currently use WD and so far so good 

[Erik Lund]

Andy that looks impressive!
That Google report is really interesting, thanks for posting! Nice with real statistics

[Andy]

Andy that looks impressive!

I built a few of these setups and was always impresssed, what kind of performance level was possible with standard HW when systems were properly balanced for the task at hand.

Parallel is the way to go (photo taken in the early days of the project)


While SSD'S are preferred from a performance perspective, economics still require hard disks for large capacity servers. My home-built home server "powering our digital household", uses 24x 6TB hard disks. Special care has been taken to keep noise, overall energy consumption and temperature with the HDs low, while providing very good performance via 10 GbE connections. Wrt to HD's, most are WD drives. To check for drive health, it is recommended to take regularily a quick look in the SMART logs of the drives. If the soft error rate increases with a particular drive, retire it rather early than late.

rgds, Andy

[Frank Fremerey]

BTW, loading a 100 GB data file from the SSDs into main memory took less than 5 seconds - a very convenient speed to work with....
rgds,
Andy

Impressive array and obviouly lots of time spent constructing and maintainig it. Du you sell these?

Or is it a hobby in the sense of spending time and money for the joy of it?

[Frank Fremerey]

External back-up hard-drives are a bit similar to how we view death. We think it wont happen to us, just to the guys next door. Yet the evidence is there that we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.

It is a good idea to have every Bit you value on at least two different physical media in two different physical locations at every point in time. "in the cloud" is no gurantee for this, esp not if the passwords and crypto keys you need to access are the subject of your backup.

Either these are too accessible (probably alo for the unauthorized) or they are lost and with the keys your property.

The right strategy is not easy to find.

After our death what will happen?

Will our children value our electronic legacy as much as we do?

Happy man whose work is valued so high that agencies or museums will care for the assets...

[Andy]

//off topic

Frank,
it was a hobby. Kind of "is it possible?" questions to go for, or rather a "it can be done" approach.

It is not related to photography, but a few idea's:
One machine had faster I/O performance at application level as the largest mainframe at that time.
One machine surpassed the performance of a cluster with 432 servers and 2400 disk drives which established a world record in a particular field 5 years earlier.
A set of machines equalled the compute performance of the leading system in the Top 500 list 11 years earlier (without spending the 120 mio USD of this super computer )
Submitted a paper for a particular energy efficient workload in a contest - still leading in this category.
... a few other competitions ....

It was insightful, I learned a lot and it was fun.

Most components are used by my son now. He studies physics and computer science and has ample ideas to use the systems for.

rgds, Andy

another setup. CPU's are water cooled, lots of GPU compute power

[Bjørn Rørslett]

You aren't involved in the password cracking business, are you ??

Interesting to know what OS you run on these mighty machines, by the way.

[Andy]

You aren't involved in the password cracking business, are you ??
Interesting to know what OS you run on these mighty machines, by the way.

Oh no - no password cracking (BTW, password crackers usually use AMD GPU's, not NVidia. AMD does support bit operations better)
My background is computer science and I am fascinated by all things parallel and high performance. Especially GPGPU (Using GPUs for "general purpose" computing) with its peculiarities in programming is an area of personal interest.
For those interested: hgpu.org is a good place to find thousands of scientific papers where GPUs were used for particular research interests.

It is relatively easy to write programs for GPUs. It is very hard to write for best possible performance. At least for me, but it is an interesting endeavour to keep all transistors spinning.

32.384 GPU cores, approx 16 TFlop/s DP



wrt to your OS question: Win10 Pro, WinServer 2012/R2/2016, Ubuntu, Debian
The OS is less of an issue, as both WIndows and Linux can deal with NUMA architectures quite well. Yet, it is very rare to find applications which utilize the power of dual and quad socket machines. Especially with distributed I/O architectures modern Xeon's have.

rgds,
Andy

[ColinM]

External back-up hard-drives are a bit similar to how we view death.
 .....we WILL all eventually pass on to photographers-heaven, where all the data will be safe in the cloud.

. "in the cloud" is no gurantee for this,

Frank, I read Elsa's comment as black humour - when we die and are up somewhere "in the cloud(s)" our memories will be there too.

My advice to anyone's kids - if your parents had outstanding or valuable photos, make sure you help to keep them safely stored somewhere

[Jørgen Ramskov]

Backblaze has posted their stats for 2015: http://arstechnica.co.uk/information-technology/2016/02/hgst-hard-disks-still-super-reliable-seagates-have-greatly-improved/

[schwett]

fascinating discussion, and really excellent hardware pr0n.

ironically, I have always assumed hard drives will fail - but the only ones that I've ever had fail are backup drives.

admittedly, my current setup is rather simple by comparison. a pair of toshiba 5TB drives internally in a raid zero array, backed up daily to one of a pair of external 8TB drives, which alternate locations each week. an easy system for a modest amount of data - worst case from a computer failure is the loss of a day's change, worst case from a fire, flood, theft, earthquake, etc, is a week's worth.

before his retirement many (20ish)  years ago, my father ran the company that supplied hard drive heads to most of the drive manufacturers of the day (seagate was among the few which made their own, if I recall.) he had many very entertaining tales of the generally shoddy and unprofessional nature of most of their customers with regard to the "consumer" category of product.