NikonGear'23
Gear Talk => Other => Topic started by: Bjørn Rørslett on December 07, 2015, 11:23:18
-
... to heed. Hard disks do not last forever. In fact, their longevity apparently declines proportionally to lowering of price and increase of storage capacity. All too often photographers have no idea about how vulnerable their mass storage is until something crashes and take with it all those great masterpieces of photographic art. Or memories of loved ones. Or snaps from the latest travel abroad. And so on.
The salient point here is that you have to plan for the contingencies arising when one or more of your hard drives fail. To support the point, this is the current crashed disks piling up on my bookshelf. I estimate the crash rate to be about 0.7% per month of my total disk volumes in use or approx. 1 disk per month. Automated SMART monitoring usually gives a warning prior to the problem becoming acute or if failing to do so, the various RAID systems on the network tend to absorb the danger and provide time to replace the crashed disk. These days, I always order at least 2 spares for each crashed disk so the standing inventory allows hot swapping and a minimum of downtime.
Do not fall into the trap of thinking a particular brand is superior to all others. They are not. Price is not decisive either. Hard drives truly are mass production items and hence consumables. Do keep that in mind. You are not a sissy just because you run backups ever so often.
(While I was writing this post, sudden beeping from a NAS alerted me of yet another disk problem. As if I needed that reminder :D )
-
In case any one should wonder, yes SSD disks crash or malfunction too. They are not exempt from failure by any means but their issues frequently are on another level. The drive becoming sluggish or having the occasional hiccup is a sign that you are about to exhaust its wear levelling tolerance. Not much you can do about that other than cloning the drive to another unit, and of course absolutely ensure that remnants from the spinning platter area such as automated content indexing and disk defragmentation are turned off.
-
1 disk per month seems like a lot, but then again, I don't know how many disks you have running in total?
My latest recent thoughts about backup is that it is one of the major IT problems that no one seem to have properly solved, which is why we time and again hear about people loosing data. The move towards smartphones and tablets haven't made the issue any better or easier to solve.
-
The general idea is to have a proper IT infrastructure at my new place with dual Internet connections (ADSL and 4G), NAS storage and backup, wired and wireless network, wall sockets with USB-C charging ports (5-20V), media consumption hubs, etc, etc.
Currently I have a lot of offline disks lying around with hopefully duplicate data, will be a hell of a job sorting things out so would also be nice to have some sort of catalogue system in place to keep track of all my images and other files.
Any suggestions are welcome btw :)
-
Let me know if you find a good catalogue system. I'm not a professional so my primary way is to use resonable folder and file names. Video is a sore point, I have never really gotten around to handling them properly. So far, they are simply put in monthly folders...
-
1 disk per month seems like a lot, but then again, I don't know how many disks you have running in total?
A few hundred overall. Totalling about 200 TB at the last reckoning.
My latest recent thoughts about backup is that it is one of the major IT problems that no one seem to have properly solved, which is why we time and again hear about people loosing data. The move towards smartphones and tablets haven't made the issue any better or easier to solve.
Agree about the common ignorance of backups. Even people using RAID systems think these are "backups" and they really should know better.
I make a boot disk clone of all system disks and store in a safe place. Then, periodic backup of data and [program] configurations though a backup server cluster. Important data tapes are stored off-site.
-
JA: simply make a complete directory/file listing of every disk and store all the data in a data base. Give each disk a unique label and store them sequentially in a safe place.
Then, put a bar code label on each disk. Bar code scanners are cheap if you go for the knock-offs and they easily cope with your modest requirements. This way you are updated where your file(s) are.
-
A few hundred overall. Totalling about 200 TB at the last reckoning.
That'll do it :) I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks.
Agree about the common ignorance of backups. Even people using RAID systems think these are "backups" and they really should know better.
I make a boot disk clone of all system disks and store in a safe place. Then, periodic backup of data and [program] configurations though a backup server cluster. Important data tapes are stored off-site.
It's clearly doable, it's just way too difficult for most people and there's also the question of it costing money. Like spending a bit money on a good password manager instead of using the same password for all websites, people hope for the best (or aren't aware that it could be an issue at all) instead of spending the money. For the vast majority of users, it needs to be at set and forget thing and it needs to be something they should be forced to make a decision about when setting up a new device/machine. Apple is in a unique position to do something like that for their customers across all their devices. Their current solutions doesn't cut it at all and are too pricy as well.
-
Backups are a nightmare and you'll know if they are ok only when you'll need them...
Another point not focused here that I consider even more important, is a different location for your backups, in case of fire or robbery...
Robbery reminds me another issue, protection of data...
Well, well, long are the days of negatives, but they also had/have their issues.
-
"I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks."
Think proactive, don't wait for the disaster to strike. Order at least 2 spare disks for your NAS (same size or next step up). Sooner or later they will be needed. A five year system is entering the danger zone.
Also remember other hardware components might fail. Did I mention power supplies? (for stationary machines, laptops, NAS etc.) They die on you as well. Perhaps less frequently than hard drives, but within a year I typically replace several units because they cease to function or deliver erratic power.
Fans fail too and although a bad fan mostly is an audio annoyance, system ventilation can suffer and the unit become overheated. Thus they need periodic maintenance or replacement too.
-
Very few people verify their backups but it is possible to do. Another thing that costs time and money and it's rarely something people consider a fun thing to do in their spare time.
Seconday, remote backups are important.
-
I'm not advocating being paranoid about data security, but there are a sensible middle station for everyone when you assess risk factors and proactive measures. The main point is that you *think* about what can happen and make a rational strategy. Like taking an insurance for your home. You don't expect it to burn down immediately or at all, but just in case... Same approach should be applied to your photographic data storage.
-
"I've been lucky so far (crossing fingers), haven't had a disk failure yet on my 5 year old NAS with 5 disks."
Think proactive, don't wait for the disaster to strike. Order at least 2 spare disks for your NAS (same size or next step up). Sooner or later they will be needed. A five year system is entering the danger zone.
Also remember other hardware components might fail. Did I mention power supplies? (for stationary machines, laptops, NAS etc.) They die on you as well. Perhaps less frequently than hard drives, but within a year I typically replace several units because they cease to function or deliver erratic power.
Fans fail too and although a bad fan mostly is an audio annoyance, system ventilation can suffer and the unit become overheated. Thus they need periodic maintenance or replacement too.
Indeed, I probably should have a spare disk. However, I'm not using it professionally so I can live without it for a few days if neccesary. If/when a harddrive fails, I will simply shutdown the NAS, order a new disk and replace it. I have both local and remote backups of my data.
You are certainly right that everything else might just as well fail, the PSU, the motherboard or something else entirely and I'm actually more worried about that as that's not something I can easily replace myself. My NAS will not recieve updates for that much longer either so I will have to replace it sooner rather than later.
-
My brother (who works in IT) have a saying: "Grown men don't take backup's. But they do cry a lot!"
So he has assured that I have a working back-up regime. And he also recomends that I replace the harddrives in my NAS after 40.000 hours of use, even if they are working fine. Because they will fail, eventually.
That said, Jørgen has an imortant point. A backup that's not verified is not a backup. It's just hopefully a backup....
-frank-
-
Linus Thorvalds have once said "Only wimps use tape backup. REAL men just upload their important stuff on ftp and let the rest of the world mirror it." :D
If my calculations are correct, 40.000 hours is something like 4.5 years. I guess I should replace mine then :) But I have no doubt that if I do that, Murphy's law will hit me right after by making sure something else in my NAS fails :P
The fact that you had your brother (who works in IT) to configure or making sure your had a proper backup solution just proves my point: It's way too difficult.
-
I beg to disagree. A backup strategy is less complex than the digital camera we take for granted, and their manuals. Basically it boils down to having more than one copy of any valuable data and knowing where to retrieve and restore it. Whether this aim is achieved by a script, canned OS procedure(s), or a third-party software package, is immaterial. A lot of options exist and cost from nothing to much less than a camera.
Verifying the backup tape(s) is an automatic procedure that needs virtually no user intervention. You can use disk-disk backup (virtual tape) in the same manner if tape is not feasible.
-
I think I have around 1TB of images which isn't very much so I'm thinking of a single NAS with enough disks that one or maybe two can crash without any dataloss.
As flooding will be a major concern in my new place I intend to have a couple of off site copies lying around which will be exchanged for a fresh copy whenever I visit said location. In the future when faster Internet is available on my remote location I might co invest with others to have two or three NAS boxes synced online. Using the cloud might be another option but I'm not a belieber in the concept that others can determine what they can do with my data.
-
In perhaps 25 years of using home PCs and laptops I have had one (1) disk crash.
Bjørn, I cannot understand how you accumulate so many! Maybe you need some new machines to run them? Something is going on there.
-
Crashes tend to follow a clustered extreme value distribution. So there can be many months without any, then all of a sudden several in a row. Some disks die very young, whilst most give up the ghost after running for years.
The number has to be seen relative to the population size, thus failure rate is not much different from other mass production items, photographic lenses included. Actually, single-point estimates of lens failures are higher.
-
I switched to cloud service. No worries about backup, hardware crashes......I hope
-
I beg to disagree. A backup strategy is less complex than the digital camera we take for granted, and their manuals. Basically it boils down to having more than one copy of any valuable data and knowing where to retrieve and restore it. Whether this aim is achieved by a script, canned OS procedure(s), or a third-party software package, is immaterial. A lot of options exist and cost from nothing to much less than a camera.
Verifying the backup tape(s) is an automatic procedure that needs virtually no user intervention. You can use disk-disk backup (virtual tape) in the same manner if tape is not feasible.
I think you misunderstand me. A backup strategy isn't complex for sure, but making sure you have a backup of everything is as it is often spread out to many places. My primary point is probably that it's something people simply don't care about until it's too late. I think it's a problem that a company like Apple could really take to the next level.
-
I think I have around 1TB of images which isn't very much so I'm thinking of a single NAS with enough disks that one or maybe two can crash without any dataloss.
As flooding will be a major concern in my new place I intend to have a couple of off site copies lying around which will be exchanged for a fresh copy whenever I visit said location. In the future when faster Internet is available on my remote location I might co invest with others to have two or three NAS boxes synced online. Using the cloud might be another option but I'm not a belieber in the concept that others can determine what they can do with my data.
The remote backup I have in place is exactly like that. My brother and I am fortunate that we both have fast, fibernet based internet connections, so we're simply running remote backups to eachothers NAS.
I'm curious why you'd choose to move to a place where flooding will be a major concern?
-
In perhaps 25 years of using home PCs and laptops I have had one (1) disk crash.
Bjørn, I cannot understand how you accumulate so many! Maybe you need some new machines to run them? Something is going on there.
He's got a lot of harddrives running, he's bound to experience a lot more crashes. Having worked in IT outsourcing, I have witnessed how often even server disks and hardware fails.
Another thing I read several years ago is that I believe it was Google, had done some RAID studies on a massive scale and warned about using RAID5. The reason was the size of the disks made a resync of the RAID take quite a lot of time and their statistics showed that there was too high a risk that a second disk would fail before the failed disk had been replaced and the RAID back in sync. Google don't use RAID at all I believe, they have created their own filesystem and I believe it automatically makes sure that all data is located in 3 different places at all times (a very simplistic view, it's of course much more advanced than that).
-
I switched to cloud service. No worries about backup, hardware crashes......I hope
Interesting, could you share some more details about your setup?
-
It is correct that the bit rate error of consumer disks with their current huge size makes building vast RAID5 volumes more risk-prone than earlier. Hence it is wise to take the array off-line during the rebuild and have additional spare(s) at hand. I have yet to encounter a failed rebuild due to bit errors, but this is sooner or later going to happen for sure.
There are alternate RAID configurations which may suit today's massive space requirements better than RAID5 in terms of security, but not when economy is factored into the equation as such RAIDs generate more overhead for additional parity and striping disks. Up to around 10 TB should still be feasible for consumer-class RAID5 systems though.
-
RAID6 would be the obvious next step. It's depends on now critical the system is and the discussion isn't relevant for many of the most critical systems - they are connected to a SAN instead.
We live with the possibility of hardware failure in our cameras too - flash cards fails regularly too, which is probably why it's possible to create a mirror with two cards in many cameras.
-
Data Storage on DNA Can Keep It Safe for Centuries:http://www.nytimes.com/2015/12/04/science/data-storage-on-dna-can-keep-it-safe-for-centuries.html?_r=0
Not yet available unfortunately :)
-
The fact that you had your brother (who works in IT) to configure or making sure your had a proper backup solution just proves my point: It's way too difficult.
It's because of him that I decided to work out a backup strategy. But I set it up myself. It's not terribly advanced, but it suits my needs. I have learned that a verified backup is important, after a harddrive failed and I had to recover a day from my (not verified) backup. No need to tell you all it failed. :-)
So a revision of my strategy was needed.
I use a free tool called Cobian, and cycle two external harddrives, one running backup, and one stored outside the house. I swap them around every two weeks or so.
-frank-
-
If you have a Blu-Ray DVD writer, it will write to M-Disks (Millennial disks), which are said to last 1,000 years if properly handled. A Blu-Ray M-disk will store 25 GB of data, which is not too bad. More info here:
http://www.mdisc.com/
-
Unfortunately, although the media itself just might survive a millenium, the chance that the reader will do the same is virtually nil.
Thus we push the problems ahead of us.
-
Unfortunately, although the media itself just might survive a millenium, the chance that the reader will do the same is virtually nil.
Thus we push the problems ahead of us.
I understand, but the standard response to the view: if there is a reason (or money) to keep it current, someone, somewhere, will offer it as a service. And, it is not that we have much choice, just now. If the M-Disks were 100 GB, we would all be using them. At least they give us a little time.
-
RAID6 would be the obvious next step. It's depends on now critical the system is and the discussion isn't relevant for many of the most critical systems - they are connected to a SAN instead.
We live with the possibility of hardware failure in our cameras too - flash cards fails regularly too, which is probably why it's possible to create a mirror with two cards in many cameras.
Well, SANs consist of (raid) disk sets in the background, and the disks share a quite consistent dying rate with what we discuss here.
SANS are abstractions, they just help to operate disk space more efficiently and independently from servers.
And they often have supportive measures build in to deal with the disk imperfections.
For our purpose:
There are quality differences between drives: desktop SATA worst, server SAS/SCSI best. Server SATA better than average. You get what you pay for. Think in price factors.
Always add hot spares. Always.
Cold spares (sometimes called stand-by disks) only make sense when they are not sharing the same mechanical vibration domain of the main cage or case.
Read performance: Raid 5,6, and 10 deliver about the same read performance.
Write performance: Raid 10: 50% write performance, Raid 5: 25%, Raid 6: 17%.
Space effficiency (minimum 4 disks): Raid 10 has only 50% constantly, Raid 6 at 50%, Raid 5 at 75%.
Space effiiciency ( 8 disks): Raid 10 has only 50% constandly, Raid 6 at 70%, Raid 5 at 85%.
Reliability (4 disks): Raid 6 is 450 times raid 10 and raid 10 is 10 times Raid 5
Due to the propability of a disk having bit errors it is good advice to keep arrays of very cheap disks, say SATA/P-ATA desktop disks, small. 10 Gbytes (4 disks) or so.
Size and time make up for failure.
Keep copies. Use different media. Encrypt and load to the cloud.
Did I talk about power supplies?
-
Data Storage on DNA Can Keep It Safe for Centuries:http://www.nytimes.com/2015/12/04/science/data-storage-on-dna-can-keep-it-safe-for-centuries.html?_r=0
Not yet available unfortunately :)
Quite fascinating, haven't heard about that before. Sounds like part of the technology needed to create a "babel fish" :D
-
Thomas G: You are of course correct. We are moving away from HDD's to SSD's and such. Samsung recently displayed a 16TB SSD.
-
Longevity of SSDs needs to increase really fast, and prices come down, are they to become a viable alternative for large-scale data storage. Seen in an overall perspective they are early days and their technology is immature. Not unlike our digital cameras in that respect :D
-
Quite fascinating, haven't heard about that before. Sounds like part of the technology needed to create a "babel fish" :D
It is brand new stuff indeed. Read only was demonstrated in 2012. Read/Random Edit was published in Nature in September this year and the Microsoft paper will be published next year only.
Babel fish for instant translation?
-
Data Storage on DNA Can Keep It Safe for Centuries:http://www.nytimes.com/2015/12/04/science/data-storage-on-dna-can-keep-it-safe-for-centuries.html?_r=0
Not yet available unfortunately :)
So now we just need to worry about real viruses attacking our storage...
-
Or evolutionary mutations :D
-
That could lead to new types of creativity...
-
and organic editing ...
-
Babel fish for instant translation?
Yes, the babel fish from the Hitchhikers guide to the galaxy :)
-
Interesting, could you share some more details about your setup?
I got tired of keeping everything in sync across devices and computers. Although I do have a 12TB Mirrored Storage Unit (Promise) and several other units, I use Dropbox as my cloud storage. They take care of backups and when all of my computers crash and need new computers, I just download everything.
Files (photo libraries and raw files) are synced across laptop and mac pro automatically. I decided to keep everything in NEF and RAF because software evolves and what is impossible to do with today's software might be possible tomorrow. I know I should only do it for precious images but I don't spend time on selecting the precious ones so I keep everything in raw. I made once the mistake to convert to DNG and/or JPG and won't do that again.
I can even access from my iPad if I want/need to.
So life has become substantially easier, also in migrating to another computer. The only tricky thing is to make sure plugins etc are in sync across computers.
Having said so, the old dog in me still tells me to backup occassionally on my (mirrored) network drives. ;-)
PS: If you worry about security, I think data in their centers is more secure than data in our own personal networks. They have dedicated firewalls and security and network exports. I don't although I customized my firewall.
-
Appears you need a substantial bandwidth to make remote storage efficient?
Will your solution scale to say storage requirements of 1-200 TB within acceptable economic overhead?
-
Dropbox have a €12/month plan with unlimited storage. I use their basic, free offering and it seems decently fast, but you're limited to one central folder under which everything needs to be stored. I don't know if that's different if you switch to one of the paid versions.
-
Appears you need a substantial bandwidth to make remote storage efficient?
Will your solution scale to say storage requirements of 1-200 TB within acceptable economic overhead?
Mine is 1TB so I can work with one local folder/disk. Your situation is more comparable to that of businesses who use cloud services. I haven't checked the availability of 200TB solutions but there is a Dropbox solution for businesses that has unlimited storage. I bet that file management differs too so perhaps you could contact them. Or another provider. There are multiple vendors for cloud storage for large data amounts.
The sync solution of dropbox is fast and only syncs changed files. A 200TB databse might be a problem but I assume you have multiple libraries and files.
-
That wouldn't quite work for me as I use a laptop with a 512GB SSD and move processed images off it to my NAS as I don't have enough room on my laptop to store all my images. Other than that, I can certainly see the advantage of using Dropbox, they do offer a nice service.
-
That is all really weird to me. In 20 years I have not had a single crashed hard drive.
-
I don't believe my experiences with hard disk failures are exceptional. Lots of friends have had similar encounters with reality.
If failure rate is approx. constant, the expected number of actual failures becomes a function of the total number of disks only. A small rate multiplied by a large number of units rapidly becomes a quantity > 1.
-
I use level 5 RAID with a redundant HDD backup archive of the active RAID arrays
-
I have experienced CF card, computer HD, mainboard and memory failure. Not too often but it happens.
-
... to heed. Hard disks do not last forever. ....
timely reminder .. and what do you think happens in the days since this post and yesterday! .. crashed primary (image) storage drive.
I have no idea how long it's been dead for, but noted it only yesterday.
Work has kept me too busy to do anything photography related over the past few months, and haven't had much time to log in since about last weekend.
Of course I have a secondary backup of the image archive .. but work is so busy ATM, I don't even have the time to get to the local store to acquire the necessary replacement drive too.
Secondary drive has now been taken off line(to maximise preservation) until a new drive(or two) is/are eventually purchased.
This is my first drive failure in about 15 or so years when the last one dies(an old 80G drive, then currently a large capacity drive) .. so drives have been good to me over the years considering my usage pattern.
I've had many friends come to me to sort out dead drives for them, which seemed to be all too often up to about a year or so ago(maybe two) .. when it got to the point where I had to acquire a docking station just to make it easier to sort them out, rather than forever opening my PC case to connect it to a cable .. etc.
So of course, not long after I acquired the docking station my friends/family's HDD's all behave properly now, so the docking station has become almost redundant(and there was much rejoicing).
moral of the story:
.. if you want security in failure resistant hardware, acquire the necessary tools to fix the problem(s) before the failure of the hardware .. this ensures that you'll never get to use those tools ;D
-
This could easily a problem if you deal with a few disks containing mirror copies of the archive. Before a disk obviously fails, it already may contain corrupt data. These corrupt data/images may then get copied along onto the upcoming mirror disks.
-
I have had my fair share of dying HDD over the years and hear about them from our IT dept at work all the time, so many possible things can go wrong trying to install the backup that it's puzzling that all is not lost more often...
Currently I use multiple HDD and Drop box but looking in StremNation.com since its cheaper and has full support for NEF and DNG raw files on top of the JPG files.
I'm testing a 4G on top of my normal connection to keep upload speed as high as possible.
-
I'm testing a 4G on top of my normal connection to keep upload speed as high as possible.
I am planning to do the same, in my new place the closest Internet switch box is 6km away so the fixed line speed will be something like 1-3Mbit/s :-\
So I've chosen a provider that offers an additional subscription with 100GB of 4G data for €45 a month available for people living in rural areas.
This way I can still do NG admin work while watching Netlfix or HBO ;D
-
I chose 45€; 300GB free data at between 20 and 30 Gbit/s down and up
-
I chose 45€; 300GB free data at between 20 and 30 Gbit/s down and up
Niiiiice, no such subscriptions in Holland I'm afraid :'(
-
This could easily a problem if you deal with a few disks containing mirror copies of the archive. Before a disk obviously fails, it already may contain corrupt data. These corrupt data/images may then get copied along onto the upcoming mirror disks.
On the topic of corrupt data:
Is there some software that can quickly confirm the integrity of a collection of NEF files without the need to try to view them all.
I was going through my collection a while back and got back into tagging/keywording what I could reasonably do in a new session.
I noted that some images(only in one directory) had been corrupted. No program would display them as an image nor the preview file(embedded jpg file).
I tracked the corruption of those images down to an issue I had with the docking station when I initially got it and started using it.
On USB3 the docking station would suddenly quit transferring data and hang. (turned out to be a badly made driver for the device).
I initially started to transfer my archive to a new HDD connected via USB3, and the transfer would hang after a short time.
Gave up on that, connected the docking station to USB2 and it would transfer the archive without issue.
Finally go the driver sorted and USB3 would work fine .. but unknown to me in the few attempts I made to transfer the data via USB3 where the transfer quit, the hanging process must have corrupted some of the images.
From memory I think 8 images were lost and I can't figure out a way to recover them(but that's not the issue).
The sudden loss of connection must have caused the loss, as the last written file date of those images corresponds with the time when I got the docking station(and new HDD and started the secondary backup process).
I know you can do a md5 hash checking thingy to compare one archive set compared to another, but this is no good if both archive sets are corrupted in the same way.
Is there a program that can simply scan or check to see if the image file(tiff or NEF) is corrupted in any way.
The only reason I noted those files was that the preview files in that folder weren't displaying at all, so went in to view what those files were.
-
I know you can do a md5 hash checking thingy to compare one archive set compared to another, but this is no good if both archive sets are corrupted in the same way.
Is there a program that can simply scan or check to see if the image file(tiff or NEF) is corrupted in any way.
The only reason I noted those files was that the preview files in that folder weren't displaying at all, so went in to view what those files were.
I use dcraw for this. dcraw -c filename.nef >/dev/null will print an error if the nef-file is corrupted. I use this as part of my copy-from-memorycard-script. I also use md5sum and am saving the result for comparing with this in the future. In this way I can easy check if the file is the same as it was from the camera or if it has changed.
-
Some years back I had the hard drive that contained my operating systems and programs was over heating. I was in the process of setting up and new HD to replace the one that would surely fail. I forgot a major rule of mine and did not disconnect my data drives. The OS was WinNT which was quite stable but may have hiccuped due to bit rot. The system crash took out the master boot records on all four HD that were spinning.
I had one 640MB portable USB HD with most of my NEF files. It was about 30-60 days out of date. I bought a 1T portable HD and Phoenix data recovery software. After many hours of work I had lost only 30 Photoshop PSD files and one TIF. The PSD files were very small and contained only a thumbnail JPG not the full image. In the end nothing important was lost as PSD could be remade and would usually be better that the one's lost. This was a very close call for me.
Now I have all my photos, original NEF(s) and JPG(s) from P&S cameras on four HD(s). I backup to three HD every time I upload photos. My system is well short of perfect but I'm better covered than I was some years back.
For those with no plan please start now. When I had the system crash I wasn't cloning the drive. I was only partitioning the new one. I restored my operating system from a ghost image. Starting a consistent back system would be a good new years resolution, one to keep. Maybe buy some new HD(s) on December 26 and start before the New Year.
Best,
Dave Hartman
-
Logical disk errors (corruption of file system or partitions) can be handled pretty well by many easily available software packages - provided you pull the brakes immediately and don't try to do anything further with the affected drive(s).
I have enclosures that enable me to insert troublesome disks and read them independently from their hosting environment.
-
Using the cloud might be another option but I'm not a belieber in the concept that others can determine what they can do with my data.
The venerable Steve "Woz" Wozniak expressed similar trepidation, regarding the "cloud", urging all concerned to reconsider, for one, optical media held in environmentally secured storage. Tape -- the "real stuff" is a great option if you have access to a mainframe computer, situated safely in a salt mine or... I don't know. A hollowed-out mountain in Sweden? Arkansas?
Fine art photographers, and highly subsidized and staffed university, scientific, industrial and forensic photographers, might also consider "hard copy" in the form of printed proofs on archival stock, held in suitably secure storage. (Last year I saw a 150 year old photo produced in Paris'. Last night night I attempted to rescue files on a hard drive, produced last month.)
I do realize many NG members have tens, even hundreds, of thousands of images to store, making print-file solutions prohibitively expensive and/or impractically time consuming. For this, I have no definitive answer, save for rewording what was already said by others. The rest of us (artiste') retain relatively little of what we shoot; create artist's proofs, then run limited editions of prints offered for sale. (Sometimes the ruse works, and people actually buy our "art".) All else is in a book -- somewhere -- hopefully forever or until the world tires of us. (If you ever have trouble sleeping, I'll show you my portfolio. I guarantee you'll sleep for a month after that -- but don't blame me if you smash your camera against a rock later, or paint your walls black or something. I was even once told, a near-hysteric suffered psychosomatic blindness for weeks, mumbling something about never wanting to see another "cow, bug, tree or cute baby" again. But I think it was really just for days, not "weeks". You know how people inflate facts to drive a point home. Or, in my case, a spike into an eyeball.)
Just for the record: I'm not a fan of Apple™, nor a disenfranchised one time fellow traveler who fell out of favor with the cult of "Jobs". However, I have always respected "the Woz" for the engineer he was (still is) and if he says he smells smoke, I don't have to see flames to believe him.
-
You don't need mainframe access to have tape drives :D
However, "tape" isn't necessarily the same thing, although there is a tape underneath. Beware of the DAT kind as they are susceptible to "tape salad". The more modern LTO tapes fare much better. I have built my backup server cluster on these. Works very well.
-
YIKES! And there goes my last excuse not to start drinking before noon. Thanks doc! I'll get you for that. ;D Now, where did I put my head? It was on my shoulders just a minute ago.
Incidentally: Nice array! And that black, is very sexy! How does it handle heat dissipation? Gamma shielding etc? Is it impervious to small rodents?
-
I'm living in a cold Nordic country. So heat dissipation is guaranteed ... Running a few tape stations is not that a burden for the box (4x4 Xeon).
Black seen through the eye(s) of a Noct-Nikkor at f/1.2 is even sexier. Or so my girl friend tells me.
-
The recommended computer for Nordic people...
http://www.qarnot-computing.com/qrad
-
Interesting concept, but does not appear to be versatile enough for a true server station.
-
It seems they sell the computing power to companies.
Speaking of cloud backup, I recently became aware of https://www.backblaze.com/b2/cloud-storage.html
Their pricing seem quite fair, something like $5/month for 1000GB.
-
The recommended computer for Nordic people...
http://www.qarnot-computing.com/qrad
I followed the link, read the sales pitch etc, and am still not sure if it's a practical joke, something lost in translation, or that they are merely being ironic -- and honest in the process. (In the processor?)
Anyway, I bet it's a blast for pre-heating socks and underwear on chilly mornings.
-
Interesting concept, but does not appear to be versatile enough for a true server station.
It seems they sell the computing power to companies.
I followed the link, read the sales pitch etc, and am still not sure if it's a practical joke, something lost in translation, or that they are merely being ironic -- and honest in the process. (In the processor?)
Anyway, I bet it's a blast for pre-heating socks and underwear on chilly mornings.
The essential problem is what to do with it in the summer!
-
Keilschrift.
Visual reminder that you don't want it. Acustically easy to identify as the pecker started to work on conterfeiting our ancestors scripture thus reducing the information densitity to zero.
-
My Goodness Thomas
I'm imagining the sound this produced was worse than that produced by scraping your nails down the old chalk-style blackboard.
-
Colin,
your imagination seems quite matching. Harddrives die - this one shouted it out. The neighbour disk in the set died about a day after the rebuild was done.
The data got along, but all disks had to be replaced in a carefully orchestrated session, rebuilding the raid data much slower then usually to keep the disks alive for sufficient time.
-
Very cool image! :D Thanks for sharing