A friendly warning ...

[Bjørn Rørslett]

... to heed. Hard disks do not last forever. In fact, their longevity apparently declines proportionally to lowering of price and increase of storage capacity. All too often photographers have no idea about how vulnerable their mass storage is until something crashes and take with it all those great masterpieces of photographic art. Or memories of loved ones. Or snaps from the latest travel abroad. And so on.

The salient point here is that you have to plan for the contingencies arising when one or more of your hard drives fail. To support the point, this is the current crashed disks piling up on my bookshelf. I estimate the crash rate to be about 0.7% per month of my total disk volumes in use or approx. 1 disk per month. Automated SMART monitoring usually gives a warning prior to the problem becoming acute or if failing to do so, the various RAID systems on the network tend to absorb the danger and provide time to replace the crashed disk. These days, I always order at least 2 spares for each crashed disk so the standing inventory allows hot swapping and a minimum of downtime.

Do not fall into the trap of thinking a particular brand is superior to all others. They are not. Price is not decisive either. Hard drives truly are mass production items and hence consumables. Do keep that in mind. You are not a sissy just because you run backups ever so often.

(While I was writing this post, sudden beeping from a NAS alerted me of yet another disk problem. As if I needed that reminder )

[Bjørn Rørslett]

In case any one should wonder, yes SSD disks crash or malfunction too. They are not exempt from failure by any means but their issues frequently are on another level. The drive becoming sluggish or having the occasional hiccup is a sign that you are about to exhaust its wear levelling tolerance.  Not much you can do about that other than cloning the drive to another unit, and of course absolutely ensure that remnants from the spinning platter area such as automated content indexing and disk defragmentation are turned off.

[Jørgen Ramskov]

1 disk per month seems like a lot, but then again, I don't know how many disks you have running in total?

My latest recent thoughts about backup is that it is one of the major IT problems that no one seem to have properly solved, which is why we time and again hear about people loosing data. The move towards smartphones and tablets haven't made the issue any better or easier to solve.

[Jan Anne]

The general idea is to have a proper IT infrastructure at my new place with dual Internet connections (ADSL and 4G), NAS storage and backup, wired and wireless network, wall sockets with USB-C charging ports (5-20V), media consumption hubs, etc, etc.

Currently I have a lot of offline disks lying around with hopefully duplicate data, will be a hell of a job sorting things out so would also be nice to have some sort of catalogue system in place to keep track of all my images and other files.

Any suggestions are welcome btw

[Jørgen Ramskov]

Let me know if you find a good catalogue system. I'm not a professional so my primary way is to use resonable folder and file names. Video is a sore point, I have never really gotten around to handling them properly. So far, they are simply put in monthly folders...

[Bjørn Rørslett]

1 disk per month seems like a lot, but then again, I don't know how many disks you have running in total?

A few hundred overall. Totalling about 200 TB at the last reckoning.

My latest recent thoughts about backup is that it is one of the major IT problems that no one seem to have properly solved, which is why we time and again hear about people loosing data. The move towards smartphones and tablets haven't made the issue any better or easier to solve.

Agree about the common ignorance of backups. Even people using RAID systems think these are "backups" and they really should know better.

I make a boot disk clone of all system disks and store in a safe place. Then, periodic backup of data and [program] configurations though a backup server cluster. Important data tapes are stored off-site.

[Bjørn Rørslett]

JA: simply make a complete directory/file listing of every disk and store all the data in a data base.  Give each disk a unique label and store them sequentially in a safe place.

Then, put a bar code label on each disk. Bar code scanners are cheap if you go for the knock-offs and they easily cope with your modest requirements. This way you are updated where your file(s) are.

[Jørgen Ramskov]

A few hundred overall. Totalling about 200 TB at the last reckoning.

That'll do it I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks.

Agree about the common ignorance of backups. Even people using RAID systems think these are "backups" and they really should know better.

I make a boot disk clone of all system disks and store in a safe place. Then, periodic backup of data and [program] configurations though a backup server cluster. Important data tapes are stored off-site.

It's clearly doable, it's just way too difficult for most people and there's also the question of it costing money. Like spending a bit money on a good password manager instead of using the same password for all websites, people hope for the best (or aren't aware that it could be an issue at all) instead of spending the money. For the vast majority of users, it needs to be at set and forget thing and it needs to be something they should be forced to make a decision about when setting up a new device/machine. Apple is in a unique position to do something like that for their customers across all their devices. Their current solutions doesn't cut it at all and are too pricy as well.

[PedroS]

Backups are a nightmare and you'll know if they are ok only when you'll need them...

Another point not focused here that I consider even more important, is a different location for your backups, in case of fire or robbery...
Robbery reminds me another issue, protection of data...

Well, well, long are the days of negatives, but they also had/have their issues.

[Bjørn Rørslett]

"I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks."

Think proactive, don't wait for the disaster to strike. Order at least 2 spare disks for your NAS (same size or next step up). Sooner or later they will be needed. A five year system is entering the danger zone.

Also remember other hardware components might fail. Did I mention power supplies? (for stationary machines, laptops, NAS etc.) They die on you as well. Perhaps less frequently than hard drives, but within a year I typically replace several units because they cease to function or deliver erratic power.

Fans fail too and although a bad fan mostly is an audio annoyance, system ventilation can suffer and the unit become overheated. Thus they need periodic maintenance or replacement too.

[Jørgen Ramskov]

Very few people verify their backups but it is possible to do. Another thing that costs time and money and it's rarely something people consider a fun thing to do in their spare time.

Seconday, remote backups are important.

[Bjørn Rørslett]

I'm not advocating being paranoid about data security, but there are a sensible middle station for everyone when you assess risk factors and proactive measures. The main point is that you *think* about what can happen and make a rational strategy. Like taking an insurance for your home. You don't expect it to burn down immediately or at all, but just in case... Same approach should be applied to your photographic data storage.

[Jørgen Ramskov]

"I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks."

Think proactive, don't wait for the disaster to strike. Order at least 2 spare disks for your NAS (same size or next step up). Sooner or later they will be needed. A five year system is entering the danger zone.

Also remember other hardware components might fail. Did I mention power supplies? (for stationary machines, laptops, NAS etc.) They die on you as well. Perhaps less frequently than hard drives, but within a year I typically replace several units because they cease to function or deliver erratic power.

Fans fail too and although a bad fan mostly is an audio annoyance, system ventilation can suffer and the unit become overheated. Thus they need periodic maintenance or replacement too.

Indeed, I probably should have a spare disk. However, I'm not using it professionally so I can live without it for a few days if neccesary. If/when a harddrive fails, I will simply shutdown the NAS, order a new disk and replace it. I have both local and remote backups of my data.

You are certainly right that everything else might just as well fail, the PSU, the motherboard or something else entirely and I'm actually more worried about that as that's not something I can easily replace myself. My NAS will not recieve updates for that much longer either so I will have to replace it sooner rather than later.

[frankv]

My brother (who works in IT) have a saying: "Grown men don't take backup's. But they do cry a lot!"
So he has assured that I have a working back-up regime. And he also recomends that I replace the harddrives in my NAS after 40.000 hours of use, even if they are working fine. Because they will fail, eventually.

That said, Jørgen has an imortant point. A backup that's not verified is not a backup. It's just hopefully a backup....


-frank-

[Jørgen Ramskov]

Linus Thorvalds have once said "Only wimps use tape backup. REAL men just upload their important stuff on ftp and let the rest of the world mirror it."

If my calculations are correct, 40.000 hours is something like 4.5 years. I guess I should replace mine then But I have no doubt that if I do that, Murphy's law will hit me right after by making sure something else in my NAS fails

The fact that you had your brother (who works in IT) to configure or making sure your had a proper backup solution just proves my point: It's way too difficult.