Discussion Backup Strategy and Hardware (Which Storage Media Do YOU Support?)

[MalVeauX]

Hey all,

I think we all consider redundancy and all that for our data to an extent. But now it's 2020. It's a great year to really stop and think about backup solutions. Actual backup solutions, not just minor redundancy. But true archival level backup solutions that could survive a 2020 catastrophe. For example, I'm in Florida, with two potential hurricanes coming down on my coast line at the same time. This is a great year to think about my current backup strategy and question it and what it will survive potentially.

So what's your backup solution(s)?

Hard drive? Second hard drive? 3rd hard drive? (Taking the multiple physical copies approach, and refreshing every 3+ years?)
Redundancy across several hard drives? Multiple locations (including off-site, in a fireproof waterproof container)? (Plus refreshing very 3+ years)?
Optical media (such as M-disc class engraving)?
Flash/SSD options for short term and travel? (refreshing within the year?)
Cloud storage (if so, how much capacity, storing RAW files or just display files with compression)?
Tape? Does this still exist for today's capacity needs?

How often do you refresh your data?
Do you test your backups?

Do you think your current system will survive a physical catastrophe (fire, flood)?
And if you had to retrieve data from your off-site backup (if you have one), will get you get back ALL your data? Or will you only get back some? And how long will it take you to get it all back (if cloud based)?

Very best,

 

[MalVeauX]

Hrm,

I'm really interested in the idea of being able to backup the media on my family's phones after connecting to wifi or something. I could do it manually I suppose, but the idea of just letting them connect and it handle it would be really nice. This is a low priority. Just curious about how to implement this.

I'd like to find a good software solution that will allow backups, sync of locations across a network, and scheduling maybe. I don't need daily syncs. But maybe a monthly sync scheduled, and the ability to tell it to "just go ahead and sync now" and maintain the schedule regardless. Just for living data that isn't crucial for the long run, but just to keep a fault from being the total loss. Crucial data will be manually burned to optical media and archived for the long haul.

Very best,

 

[MalVeauX]

Update on M-disc:

The drive & discs showed up in the mail today. I got right to work on it.

I just burned a 100Gb M-Disc (BDXL) with a typical drive, nothing costly or special, using free software (ImgBurn) and it detected the full capacity (100Gb) and layers of the BDXL without any effort or special settings and the drive wrote the files to the 100Gb disc without any error, followed by a complete verification post-write at low speed (4x) to ensure accuracy of the data. The drive is a LG WH16NS40 from May of 2020 per its engraving of manufacture off Amazon ($65 new), latest firmware (nothing special for this though). The disc is a Verbatim M-Disc BD-R branded surface BDXL 100Gb multi-layer disc ($11 and change, but I bought a couple in a pack).

For this disc content, I did our 2013's photos. Just the RAWs and JPGs and that's it. It was 95Gb (after culling chimps out) so it fit just right on the 100Gb disc. I did not write on the disc with a market, and will not, to avoid any chemical reaction over time. File system is UDF, and it handled deep directories (8 directory trees) and long file names and folder names without issue. This is read natively by any OS on the planet. I checked with a Windows and Linux platform just to satisfy my own curiosity.

This was successful and error free, so this backup copy of the photos will now go into a fireproof/waterproof safe and be the 3rd physical copy of the data (the other two physical copies life on redundant drives that are separated), it will be hearty to environmental conditions and not suffer magnetic issues, or dye/pigment issues, and no mechanical failure options. Should last my lifetime and then some in fairly well controlled settings. $11 and change for the disc covers all our family photos for that year. I'll do each year separately until its all archived. Then it all goes into the safe.

I will test the disc in a separate drive, likely an external, to see how it is handled; since these drives are so low cost, having one or two new ones in packing to read the backups and refreshing hardware every decade or so should handle file retrieval if needed.

So far this is about the cheapest true backup that will survive most issues and be handled with the most common and least expensive hardware with the least complexity that I can figure without doing the cloud approach.

The capacity per cost is not the best compared to other mediums, but the survivability of the medium was the goal. As a 3rd physical copy that is. Not all of my data will be treated this way. But things like family photos, documents, etc, will all be "engraved into stone" so to speak with this for now. Unless a better method comes up in the future.




Very best,

 

[MalVeauX]

Update,

I just completed my 4th M-disc burn; all data written and verified to be 1:1 bit perfect copies. 400Gb down. This was made up of my family photos from 2013 to 2015 so far. I have a bit more to go. Each disc takes 1.5 hours to burn, another 1.5 hours to verify before I call it good and store it. So far, averaging 200Gb per year on average based on this projection is putting me at $22 per year to have a 25+ year archival format for irreplaceable data (family photos, documents, scans, etc) that can survive an EMP and has no chance of mechanical failure and will live in a safe; being the 3rd physical copy of the information completing the idea of a true backup. This projection works out to be a $1 per year cost to produce most-of-life archival level backups.

Very best,

 

[razel]

Why the need to support one over another? There is no one backup strategy for everyone.

 

[MalVeauX]

Why the need to support one over another? There is no one backup strategy for everyone.

You're of course 100% correct, one strategy is not for everyone.

However, one's concept of "backup" however is not often a true backup and merely an uptime supportive strategy. I've been curious what folk who are trying to truly backup information are doing, at this level, relative to what commercial levels do. We learn from each other after all.

Beyond that, I'm simply sharing what I'm doing and why.

What do you do?

Very best,

 

[Chiefcrowe]

I use Macrium Reflect to do image backups occasionally on 2 drives, keep one off site.

Also last year I decided to finally start backing up most of my data automatically online with Backblaze. The price is really great for what they offer and they offer you the ability to set your own private key so they company can't recover the data for you in the event the key is lost.

I feel like I should do more though!

 

[UsandThem]

I feel like I should do more though!

Having the data stored in three separate locations is enough. That's already more proactive than 99.9% of people out there, and the odds of totally losing your data from all 3 locations at once is astronomical.

Just be glad you aren't one of the users who sign up here when their hard drive dies, they really need the data, and have no backups at all.

 

[piokos]

Having the data stored in three separate locations is enough.

That depends what you're trying to achieve. You can always work on backup frequency, shortening recovery time, automation, costs.
And as I mentioned earlier: separate locations aren't enough if one points to another (hence, the thief can easily do both).
So there are lots of things that can be done, depending how serious you are about not losing your data.

That's already more proactive than 99.9% of people out there

Then again, backing up data - much like staying fit or cleaning the house (and unlike, say, overclocking) - fundamentally isn't about being better than others, but about achieving a level that satisfies your needs.

 

[MalVeauX]

It's true, everyone's needs are indeed different.

The interesting thing is what each person is employing to backup after they have figured out they need a backup system of some kind and what the options are that are real world and not just enterprise level stuff.

When someone starts searching for solutions, there's all kinds of stuff that points to externals, NAS, cloud, etc, but most of them are not talking about true backup and instead are mostly calling it backup when its really just availability redundancy, as the platforms all have rather significant opportunity for mechanical, magnetic and business failure. It's a tricky subject talking about archival class backup solutions, versus a casual backup that is just another copy of the data on a separate medium that is also rather susceptible to failure. Not all data needs that level of backup. But as more and more people have irreplaceable information (be it videos, pictures, etc) from their mobile devices, the more backups will become important as people experience loss in some way and realize there's just no recovering it without a real backup.

Very best,

 

[fcorbelli]

Normally for enterprise copies I prepare at least 7, of which at least 5 are checked daily.
For "home" use I rent a FreeBSD machine (about 120 euros / year) with 2TB of space to make copies (verified)

 

[mikeford]

If you have a RAID array where two or more drives can fail without interruption, smart status checking and ready source of replacements for failed drives, plus two levels of mirror, onsite and offsite, that is usually good enough, all you might add is a rotate and archive of the offsite copies.

OTOH that doesn't even touch on external or internal attacks like Ransomeware.

 

[MalVeauX]

If you have a RAID array where two or more drives can fail without interruption, smart status checking and ready source of replacements for failed drives, plus two levels of mirror, onsite and offsite, that is usually good enough, all you might add is a rotate and archive of the offsite copies.

OTOH that doesn't even touch on external or internal attacks like Ransomeware.

A RAID array is more about uptime, not about backup. Rebuilding that RAID array with 2+ drive failure tolerance implies parity and rebuilding parity is slow, even on hardware, when you're talking about a significant data set. Having to have a backup of that data off-site is more of a backup. You're way better off with a large capacity mirror than RAID array of 5 or more, involving parity. A failure in a RAID 5/6+ array performance really slow, and rebuild is slow, and its totally vulnerable during rebuild. Not a good backup, that's just an uptime setup. A mirror with high capacity (since we can get 12~16GB or bigger drives now) can be made, simply, performs fine even if a drive failed, and rebuild is the same speed as copying (no parity calcs). Just as good for uptime, easier to deal with when failure occurs. The on/off site mirrors to this data set is your 2nd and 3rd physical copies, so that's a backup strategy, sure.

If you have off-site true backups, then you're mostly set for internal and ransomware situations. You can just fold the system and re-build from the backup. Just depends on how often the backup is. This just happened to Fujifilm. Their Japan facility got hacked a few days ago, ransomeware, they turned it all off and told them to go suck it and just reloaded from their backup. The way it should be.

Very best,