Backing up hard drives; Bluray?

[vhx]

My 2TB hard drive has been hiccuping lately and I am getting a little concerned it may be dying. Video playback was starting to get choppy and just freezing with any video I played with that drive. (Seatools seems unreliable, the short DST's fail but the Generic tests pass for two of my drives?) Interestingly enough it seems to be working fine now... but still concerned as who knows how short lived that will be.

I was thinking of getting another hard drive (perhaps external; portable seems attractive) to permanently store my data, but do not want to run into the same problem in the future. 2 TB of data is quite a lot of data to lose in a hard drive loss. I need a more permanent storage for my files. So... I'm thinking Bluray. A permanent storage medium that is portable with 25 GB - 100 GB disks seems the only choice (DVDs are too small these days). Obviously the down sides are that very few computers even have Bluray players and it will still take many disks to back up TB's of data. It seems the price per GB of Bluray disks and hard disks are pretty close together.

My questions are: How reliable are Blurays in long term storage of data? How tolerant are they in terms of scratches? Does anyone else currently do this? (Or did) Is there a better solution? Is it better to just rely on a hard disk for storage even with the chance of it dying and taking all of your files with it?

 

[StorageGuru]

Since a blu-ray you burn at home is burned and not stamped... I would not consider it very stable at all. Burnable optical media contains dye.. and the dye will degrade over time. Wikipedia have a decent article about this... check it out.
And you need way too many blu-ray disks to just back up one hard drive... at least 20 disk per hard drive... or even 40 or 80 if you use the older blu ray tech..

A better solution would be to buy two additional hard drives, or maybe even a tape drive if you have lots and lots of data.

 

[velillen]

If the data is that important I'd definitely be using multiple sources for backups. Maybe online then a portable hard drive. If its really valuable go with two mirrored drives?

I just font think bluray is really a viable option for that much data. To easy to lose 1 disk or scratch a disk.

 

[taltamir]

permanently storing your data in an external drive is a terrible idea.
External drives are much more likely to fail then internal drives.

Using an external drive as a BACKUP is fine.

You should get a replacement drive ASAP to copy all the data off this drive then RMA it or scrap it.
And get another drive to act as a backup.

Using optical media is actually a good idea for backups because thieves would steal your external drives but not the burned disk labeled backup...

But optical media has very short shelf life, its supposed to be 90% retention after 10 years on a burned media and 100 years on a pressed media (what you buy at store).
In reality even 2 or 3 years under correct storage conditions could leave lots of degraded media, even if you don't have any scratches.

making multiple copies and protecting each copy with http://dvdisaster.net/en/index.html helps a lot.

you might also be interested in an online backup called backblaze. only one I know of that offers real unlimited storage

 

[AdamK47]

I have a 4TB drive backed up to a 3TB drive. The 3TB drive is on a SATA2 port that's been configured as hot-swappable and it resides in the hot swappable bay on my HAF-X case. I use Acronis TrueImage to create an incremental backup of the data on the 4TB drive. This is about 2.7TB of data. It creates one giant backup file on the 3TB drive using normal compression. It took about a day to create the initial backup file which is about 2.1TB in size. I then do incremental updates on the backup and Acronis will only update/add files to the backup archive that have changed since the last backup was performed. It's very convenient.

 

[pauldun170]

Do not bother with discs
Get an external usb3 or esata harddrive (or buy an external enclosure + hd of your choice)
Backup old drive to new drive.
Unplug\turn off new hard drive. Store in cool dry place.

Incremental backup as needed or per schedule.

 

[Cerb]

making multiple copies and protecting each copy with http://dvdisaster.net/en/index.html helps a lot.

Do implementations of this exist that work reliably? I've tried a few times, but couldn't get a successful final image, at least not in Windows 7.

Do not bother with discs
Get an external usb3 or esata harddrive (or buy an external enclosure + hd of your choice)
Backup old drive to new drive.
Unplug\turn off new hard drive. Store in cool dry place.

Incremental backup as needed or per schedule.

...then have to deal with bitrot, because the data never got read again, never tripped an error-handling procedure in the firmware, and thus never got rewritten with fewer wrong bits.

Been there, done that.

Regularly used HDDs: good.
Resting HDDs: bad.

IME, good optical media is far superior for something that will likely sit untouched for a many months at a time. Data that is commonly touched will not face that kind of problem, and makes optical media somewhat of a PITA to use, anyway.

 

[jaydee]

permanently storing your data in an external drive is a terrible idea.
External drives are much more likely to fail then internal drives.

Can you expand on this? Is it because externals are mostly 2.5" drives?

 

[greenhawk]

My questions are: How reliable are Blurays in long term storage of data? How tolerant are they in terms of scratches? Does anyone else currently do this? (Or did) Is there a better solution? Is it better to just rely on a hard disk for storage even with the chance of it dying and taking all of your files with it?

personally, go the HDD, and use more than one (stored in different locations) if the data is really important to you.

Going the blu ray path I think is still floored. The drives to write with might have improved, but write speed is still slow. While I was surprised to see they have released something higher than dual layer, espically when availablity of dual layer writable disks were like finding hen's teeth for a LONG time.

Looking at a local shop, the writer is $120 and disks are $30 for 25 single layer disks. Nothing about any higher density disks.

So for 2TB drive, 80 disks, about $96 without issues. and only slightly cheaper than a 2TB internal drive (locally anyway).

Does not seem to make it a worth while thing (as you still need the writer) and swapping 80 disks to do a restore is well outside my attention span (cousing did about 50 cd's back in the day when upgrading drives, what a joke).

 

[Red Squirrel]

I like to use separate hard drives with a drive dock. Then I can rotate the drives around and treat them kinda like you would tapes in a corporate environment. When I see sales on hard drives, I sometimes buy a few to throw in the rotation pool. Since they are backups reliability is not as important as if it was production so you can get away with inferior brands provided it's not so crappy that it is guaranteed to fail. (like a known bad batch or something) And since you should have multiple copies anyway even if one fails it's not like you actually lost data. Just rerun a backup on another drive to ensure you are up to date.

I have a PO box where I put these as well as offsite backups.

 

[tynopik]

the other problem with bluray or other discs is, how do you update them?

a month goes by, stuff changes, can you backup only the new stuff or do you have to write a full backup out again?

that gets old real quick

writing it all out to a drive is much simpler

 

[Concillian]

I buy 3 identical hard drives for any data I think is important... 2 are for backups.

You can use Blu-ray, but make multiple copies and be careful of how they're stored and store them in different locations. 2 TB via 50GB blu-rays is a lot of discs though (40)... it's a lot easier to use multiple hard drives. Much faster and you can "set and forget" without having to worry about 40 disc swaps (80 for multiple copies.)

 

[Cerb]

the other problem with bluray or other discs is, how do you update them?

a month goes by, stuff changes, can you backup only the new stuff or do you have to write a full backup out again?

You don't store that kind of backup to that kind of medium.

Photos taken years ago don't change. Video taken years ago doesn't change. Rips of my horribly-scratched Quake, Quake II, Tribes, Diablo II, etc. disks...don't change. Likewise, financial records and eprsonal communications are set in history, and don't change.

A backup of data which is regularly updated and tested is probably best done to flash or HDDs, balancing media and time involved. Sufficiently large data ends up needing HDDs or tape, and with HDDs, at least one copy should be regularly updated and tested, such that any long-term stored copy is a copy of last resort.

 

[thelastjuju]

^ Exactly, things are so much easier when you separate the two types.

I also don't think backing up TB's of data spanning many many discs is a very good idea.

Multiple TB hard drives just aren't ready for today. 2-3GB hard drives seem to fail more than any of the smaller hard drive choices. Probably why we haven't seen 5-6GB hard drives yet (where it seems like we should), and why manufacturers are so hesitant to even release more 4TB drives.

I've got triplicates of all my data on WD Blacks (500's 640's, 750's, but no large than 1tb).. one set kept at a completely different location in case I'm robbed or my house burns down or floods.

 

[taltamir]

Can you expand on this? Is it because externals are mostly 2.5" drives?

Its not the drive size.

What breaks drives is, in order:
1. Bad power
2. Vibrations
3. Excessive heat (minor heat is irrelevant according to google study)

External enclosures (compared to internal HDD):
1. Super bad PSU give dirty power
2. Lots of vibrations
3. EZBake oven
4. Pathetic quality electronics in the enclosures USB to SATA adapter which tend to damage the drive as well as fail on its own (requiring a replacement enclosure)

Do implementations of this exist that work reliably? I've tried a few times, but couldn't get a successful final image, at least not in Windows 7.

I had no problem making a final image with that program. Did you set it to augmented image mode rather then ECC file mode?

Now, testing it is a bit of a problem since my DVD burner recently died on me for no reason... will get a new one soon.

 

[Red Squirrel]

Last I checked, a 1TB green drive is also much cheaper than the equivalant disk space in bluray discs, so that's another thing to consider. Those discs arn't cheap.

The very very very important data though could go on optical media, as in the event of an EMP or similar incident, all hard drives wont work, but optical media will survive it. So it probably does not hurt to at least archive stuff like your resume, code, software that you've paid for but don't have a CD for, etc.

 

[jaydee]

Its not the drive size.

What breaks drives is, in order:
1. Bad power
2. Vibrations
3. Excessive heat (minor heat is irrelevant according to google study)

External enclosures (compared to internal HDD):
1. Super bad PSU give dirty power
2. Lots of vibrations
3. EZBake oven
4. Pathetic quality electronics in the enclosures USB to SATA adapter which tend to damage the drive as well as fail on its own (requiring a replacement enclosure)

What about those external docks (like such http://www.newegg.com/Product/Produc...E16817153071)? Temp is lower than inside the computer case, I don't move it so there's no vibrations, and I connect it via e-sata. Only thing I don't know is the AC-DC power supply quality.

 

[taltamir]

Temp is lower than inside the computer case

true

I don't move it so there's no vibrations

Not necessarily, its sitting on your table rather then the floor so its likely to get more vibrations
I am not too sure but i think the large tower case helps cushion some of the vibrations too.

and I connect it via e-sata

That helps, a lot .

Only thing I don't know is the AC-DC power supply quality.

Which is the most important factor for reliability. And I am guessing the answer is "not very good"

Also 24/7 operation at a weird angle might be an issue.
But I am guessing (haven't used those docks) that they are better then your average external enclosure.

Its not that you shouldn't use any external drives, its that keeping something primarily on an external drive without a backup because you think its safer then being on an internal drive without a backup is silly.
You should back up regardless.

 

[tynopik]

You don't store that kind of backup to that kind of medium.

Photos taken years ago don't change. Video taken years ago doesn't change. Rips of my horribly-scratched Quake, Quake II, Tribes, Diablo II, etc. disks...don't change. Likewise, financial records and eprsonal communications are set in history, and don't change.

Point remains. You take NEW photos, you make NEW rips, you create NEW financial records, can you reliably distinguish between what's new and what's not?

You make a backup at a point in time, then a month later you've added new stuff all across the drive and maybe reorganized some photos. How do you ensure all the new stuff gets backed up?

 

[Charles Kozierok]

Point remains. You take NEW photos, you make NEW rips, you create NEW financial records, can you reliably distinguish between what's new and what's not?

You make a backup at a point in time, then a month later you've added new stuff all across the drive and maybe reorganized some photos. How do you ensure all the new stuff gets backed up?

Speaking personally -- create a whole new backup, delete the old one.

 

[tynopik]

Speaking personally -- create a whole new backup, delete the old one.

exactly

which is kind of difficult to do if your backup is burned to bluray

have to burn a whole new set everytime you backup

 

[Cerb]

Point remains. You take NEW photos, you make NEW rips, you create NEW financial records, can you reliably distinguish between what's new and what's not?

By keeping important documents organized in the first place, just so that your documents directory doesn't become a rat's nest, if nothing else.

New photos have new dates. Are you going to get confused and think the 2012-06-06 Cat Caught Ruining The Living Room folder is on the 2012-01-08 backup? Of course not.

There's also a lovely mtime column avilable in pretty much every GUI FM known to man, which allows for easy sorting for more mixed files.

WORM media is fine and good for WORM data. For WMRM data, it's not so good.

You make a backup at a point in time, then a month later you've added new stuff all across the drive and maybe reorganized some photos. How do you ensure all the new stuff gets backed up?

A backup of data which is regularly updated and tested is probably best done to flash or HDDs (em. added)

 

[Charles Kozierok]

I don't use writeable optical media for backups. Period. Just don't trust them.

 

[Cerb]

I don't use writeable optical media for backups. Period. Just don't trust them.

Trust in any single copy is misplaced. I trust healthy HDDs for some near future, as long it regularly runs, and the static data is re-read occasionally. I trust good optical media (and TBH, I don't even know what the good BD media to buy is) resting somewhere for long periods more than I trust a HDD in that same situation. I don't expect either to last forever, nor do I expect either to fail on my preferred schedule.

 

[tynopik]

By keeping important documents organized in the first place, just so that your documents directory doesn't become a rat's nest, if nothing else.

New photos have new dates. Are you going to get confused and think the 2012-06-06 Cat Caught Ruining The Living Room folder is on the 2012-01-08 backup? Of course not.

There's also a lovely mtime column avilable in pretty much every GUI FM known to man, which allows for easy sorting for more mixed files.

do you actually use this method? or are you talking in the theoretical sense? because I have a feeling this works better in theory than in practice

first of all 'keeping important documents organized' has nothing to do with the problem

if you have separate folders for pics of all your family members, that's organized, but it would still be a pain to get the latest from each folder

and even if you had some global file search to find all files modified after a certain date, you still have to recreate the directory structure on the backup media. That or just dump them all into the same folder where suddenly they aren't 'organized' anymore.

then do you remember when exactly the cutoff is for each backup? let's say you backup every saturday, but what about changes you made last saturday AFTER you backed up?

It just seems like a very labor-intensive, error-prone, sucky way to do things.