Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rjm]

Here is a first cut at a backup tutorial. Comments to correct errors or omissions are welcome. If there is enough interest perhaps it could be added to the wiki.

1.0 Why backup?

There are four good reasons to backup data.

1.1 Data Loss Prevention

1.1.1 Hard Drive Failure
All hard drives will eventually fail; it's just a question of when. Good failure rate data is hard to come by, and the rate of failure is influenced by many factors including temperature, vibration, power quality, design, and manufacturing variability. The best report I have seen was a study conducted by Google on their server farms. It indicated that the failure rate is much higher than you might expect - something like 20% of all drives within 5 years. My personal experience has been that 1 of my 15 drives fail about every 2 years. I expect this to increase as they age.

Sometimes drives fail with advance warning like a noise or operating system message. Other times they die with no warning. If you do lose a drive, and it's not making a clickity-clack sound, there is a reasonable chance you can recover the data with a good tool like SpinRite, but don't count on it.

1.1.2 Data Corruption (power failure, application bug, system crash, virus, etc.)
Even with a healthy drive you can still lose data for a variety reasons including power failure, motherboard failure, application bug, system crash, or malicious virus. My personal experience has been that these types of data loss are very rare.

1.1.3 User Error
Every user makes a mistake from time to time. Sometimes these user errors result in data loss or bad data. Sometimes you may not realize you made a mistake until days or weeks or months later.

1.1.4 Theft
A newish computer will be a priority target if your home is broken into. A laptop left in a vehicle lasts about 30 minutes in my town.

1.1.5 Disaster (fire/flood/earthquake)
Judge the risk for your own location.


1.2 Previous Version Restoration
You may encounter the need to restore a previous version of a file (see above 1.1.3).


1.3 Data Portability
If you backup your data on a portable device (like an external USB drive) then in addition to having a backup you also have a means of taking your data with you. In my case, I like having my data with me when I travel and my laptop drive is not big enough so I rely on external drives.


1.4 Drive Health Checkup & Preventative Maintenance
You won't know if your drive has a problem unless you attempt to read it. Therefore, regular backups, in addition to backing up your data, also act as drive health checkups by letting you know that the drive is still ok.

Modern drives have a lot of intelligence and rely heavily on error correction to achieve today's amazing densities. Drives monitor the error correction rate for each disc sector they read, and when the error rate climbs sufficiently high they automatically flag the sector as bad and copy the data to a replacement sector. For this reason it is a good preventative maintenance practice to force your drive to read every sector from time to time. Making a complete backup is a good way to achieve this while accomplishing something else useful at the same time.


2.0 What backup device/media to use?

2.1 Hard Drive
Given their low cost per GB and excellent performance nothing beats a hard drive as a backup device for large amounts of data. There are several ways to use hard drives for backup, but in all cases you should backup to a different physical drive. Never rely on a backup to the same physical drive (see above 1.1.1, 1.1.2, 1.1.4, and 1.1.5). Also be aware that one physical drive can be partitioned into multiple logical drives that you see in Windows; so make sure you know what drives are in your system and how they are configured. The Windows tool Administrative Tools\Computer Management\Storage\Disk Management provides a nice view of your drive setup.

2.1.1 RAID
RAID uses multiple physical drives to achieve data redundancy in one logical drive, and can protect you in a transparent way from a drive failure. There are pros and cons for using RAID. On the plus side, you are protected from a drive failure without having to do anything. On the minus side, RAID adds complexity which adds failure mechanisms, and it scrambles your data across multiple drives which can be inconvenient at times. I personally don't use it. Others swear by it. I am not going to discuss RAID further in this tutorial.

2.1.2 Extra drive in same system
On the plus side you get maximum drive performance. On the minus side you are not protected from theft or disaster, and your data is not portable.

2.1.3 Drive in different system on network
On the plus side you get more tolerance to theft. On the minus side, a wired network is about 2x slower than a local drive (and a wireless network is about 10x slower). I do not recommend using a wireless network if you copy more than a few gigabytes on a regular basis.

2.1.4 External USB/eSata drive
Mostly pluses here. USB drives are faster than network drives, and eSata drives are as fast as internal drives. In addition, if you disconnect and hide the external drive when not in use you achieve good tolerance to theft. And your data is portable. The only negative is clutter from yet another device with its own power brick and cable.

There is another possible advantage to using external drives for backup that is not often discussed. Some people believe that hard drive life is extended by leaving them powered up. Other people believe that drive life is extended by powering them down when not in use. Other people believe that frequent power up/down cycles reduce drive life. I do not know what the truth is. I therefore keep my desktop system on and my external drives off on the assumption that if someone is right then I have all the bases covered. In addition, if my home gets hit by lightning my powered down external drives should survive.


2.2 DVD-R
This is a cheap and simple (but slow) solution for modest amounts of data. The primary advantage of backing up to dvd-r is that you create a read-only snapshot of data, and assuming you keep old backups, it means you have a simple way to recover earlier versions of files. I backup my personal data (like databases, finances, email, etc.) once a month to dvd-r (because it is slow), and once a day to a different hard drive (because it is fast).


2.3 USB Flash Memory Drive
Prices are dropping and capacities are increasing. Read speed is excellent. Write speed is slow (about the same as a wireless network drive). The primary advantage is convenient portability. I keep all of my personal data on an 8GB drive in my wallet. But I only update it about once a week because of its slow write speed.


3.0 What data to backup?
Imagine losing your data. Then think about your level of pain. In my case, I back up everything.

A quick comment on data encryption. You should think about data security when designing your backup system and make a conscious decision whether to encrypt your data. In my case, I decided to only encrypt the data on my USB flash drive because I keep it in my wallet and the probability of loss or theft is reasonably high. Check out the excellent free TrueCrypt tool if you are interested in encrypting your data.


4.0 How many backups?
At least one. Preferably two. I maintain two hard drive backups of all important data like my photos and media. My assumption is that the probability of two drives failing at the same time is very low but not impossible, especially if you consider theft or lightning strikes; whereas the probability of three drives failing at the same time, assuming one is a powered off and hidden external drive, is vanishingly small.


5.0 How often to backup?
It depends on the value of your new files, how often you change files, and the effort you expended to change them. I backup all data that I change on the same day that I made the changes.  On the other hand, I only backup my system (Windows and programs) about once a month because reconstructing a month's worth of system changes would not be that painful.

Continued here... http://yabb.jriver.com/interact/index.php?topic=44493.0

[John Gateley]

Here is a first cut at a backup tutorial. Comments to correct errors or omissions are welcome. If there is enough interest perhaps it could be added to the wiki.

Very nice, thanks! I'd suggest a couple of things: emphasize that a real backup is in a separate physical location (a
"backup" that lives on a separate drive on the same system fails when the building burns), and talk about cascading
backups along with how many backups are needed (especially in the case of user error, you may need to go back
a while to get the good data, sometimes it is a while before you discover that you accidentally deleted that really
important file).

j

[hit_ny]

Modern drives have a lot of intelligence and rely heavily on error correction to achieve today's amazing densities. Drives monitor the error correction rate for each disc sector they read, and when the error rate climbs sufficiently high they automatically flag the sector as bad and copy the data to a replacement sector. For this reason it is a good preventative maintenance practice to force your drive to read every sector from time to time. Making a complete backup is a good way to achieve this while accomplishing something else useful at the same time.

Speedfan is a nice free utility to check the health of your hardrive and it produces a human readable report. Look at the smart tab and the wide button below to 'perform an in-depth online test.'

For this reason it is a good preventative maintenance practice to force your drive to read every sector from time to time. Making a complete backup is a good way to achieve this while accomplishing something else useful at the same time.

True, but how many ppl actively do this ?

I was seduced by software that only syncs changes to the backup drive. So much faster than doing a complete backup on a monthly basis. Bigger the collection, the more time saved

I guess you could turn on byte-for-byte  comparison in the syncing software to accomplish the same but feel it may actually be slower than a complete backup. A compromise that could be done, say semi-annually.

2.1.2 Extra drive in same system
On the plus side you get maximum drive performance. On the minus side you are not protected from theft or disaster, and your data is not portable.

This option to be discouraged as it wears down the backup drive and as you say does not protect against a suicidal SMPS.

Some people believe that hard drive life is extended by leaving them powered up. Other people believe that drive life is extended by powering them down when not in use.

You might want to take a look at this and its folllow-up. However this only applies to external USB drives, the eSata ones can be controlled from the OS and honor spin down times just like any other internal HD.

Keeping a drive running 24x7x365 is only going to wear its bearings down faster ie not very different to how its case fans become increasingly noisy as time goes on.  The key is to spin it down when its not used or if its not necessary to always have online then power it down.

Other people believe that frequent power up/down cycles reduce drive life. I do not know what the truth is.

The problem here is an aggressive spin-down, if you can set it only spin down after say an hr or two of inactivity then you are better off. If however the drive has a fixed spin down time after only say 5 minutes of inactivity, then you get into excessive spin up & down cycles which is not good. The rated amt. of spin cycles  is approximated at about 50,000. You can reach that limit easily in less than 2 yrs of normal use with a very short spin down time limit.

I therefore keep my desktop system on and my external drives off on the assumption that if someone is right then I have all the bases covered. In addition, if my home gets hit by lightning my powered down external drives should survive.

You are covered here

2.2 DVD-R
This is a cheap and simple (but slow) solution for modest amounts of data. The primary advantage of backing up to dvd-r is that you create a read-only snapshot of data, and assuming you keep old backups, it means you have a simple way to recover earlier versions of files. I backup my personal data (like databases, finances, email, etc.) once a month to dvd-r (because it is slow), and once a day to a different hard drive (because it is fast).

This is a valid option provided your writer is equipped to do quality scans so you can test how good the readability fo the media is. If not then you want to get Taiyo-Yuden and not need to bother anymore. 10 yrs from now, all things being equal, your data will be readable.

[p7389]

You actually "guilted" me to set up a small backup solution with Amazon s3  . Only for the very most essential stuff. In case the house burns down.

At first I looked at the app you recommended, but ended up with a super simple dedicated S3 solution. http://www.maluke.com/software/s3-backup

Gonna try it out for a while at least.

[rjm]

You actually "guilted" me to set up a small backup solution with Amazon s3  . Only for the very most essential stuff. In case the house burns down.

At first I looked at the app you recommended, but ended up with a super simple dedicated S3 solution. http://www.maluke.com/software/s3-backup

Gonna try it out for a while at least.

Let us know how well it works. Online solutions have lots of advantages if you trust someone else with your data. I have not tried them because movng 200+ MB a day over my internet connection is not practical.

[rjm]

Thank for the comments hit_ny.

Speedfan is a nice free utility to check the health of your hardrive and it produces a human readable report. Look at the smart tab and the wide button below to 'perform an in-depth online test.'

Yes, speedfan is a nice utility. I use it to check my drive temperatues which is probably the most important variable that influences drive life. I am not sure how much to trust the SMART data. Steve Gibson of GRC says the value of SMART is very dubious because every manufacturer implements it differently, and his experience has been that SMART is useless for predicting drive failure. I think the Google study corroborated this.

True, but how many ppl actively do this ?

I was seduced by software that only syncs changes to the backup drive. So much faster than doing a complete backup on a monthly basis. Bigger the collection, the more time saved

I guess you could turn on byte-for-byte  comparison in the syncing software to accomplish the same but feel it may actually be slower than a complete backup. A compromise that could be done, say semi-annually.

I agree. My frequent backups only sync changes. A couple times a year I do a binary comparison (then go to bed and check results in the morning). You sometimes find interesting things by doing a binary comparison. For example, I learned that if you open and close an Excel spreadsheet without changing anything or saving it, Excel changes the file header content without changing the file modifed date/time. Unbelievably bad behavior!

This option to be discouraged as it wears down the backup drive and as you say does not protect against a suicidal SMPS.

I agree, but it's better than nothing.

You might want to take a look at this and its folllow-up. However this only applies to external USB drives, the eSata ones can be controlled from the OS and honor spin down times just like any other internal HD.

Keeping a drive running 24x7x365 is only going to wear its bearings down faster ie not very different to how its case fans become increasingly noisy as time goes on.  The key is to spin it down when its not used or if its not necessary to always have online then power it down.

Thanks for the interesting links. I think there are 2 failure mechanism to worry about: mechanical and electronic. I agree with your point that drives should be configured to spin down to save mechanical wear and tear. The life of electronics, on the other hand, may be shortened by frequent thermal cycles. So the best compromise may be to leave the drive powered up but configured to spin down. I have never seen a definitive study to confirm this though.

I really like eSata drives except Windows Safely Remove Hardware does not work with a lot of motherboards. I found a small free utility called HotSwap! that lets me eject my eSata drives.

The problem here is an aggressive spin-down, if you can set it only spin down after say an hr or two of inactivity then you are better off. If however the drive has a fixed spin down time after only say 5 minutes of inactivity, then you get into excessive spin up & down cycles which is not good. The rated amt. of spin cycles  is approximated at about 50,000. You can reach that limit easily in less than 2 yrs of normal use with a very short spin down time limit.

I agree.

This is a valid option provided your writer is equipped to do quality scans so you can test how good the readability fo the media is. If not then you want to get Taiyo-Yuden and not need to bother anymore. 10 yrs from now, all things being equal, your data will be readable.

I have tried many different brands of dvd-r and now use Taiyo-Yuden exclusively. I also always use the verify option in Nero which does a byte for byte comparison after completing the burn. This is very important because even with Taiyo I find the odd problem.

One more tip. Do not trust the Nero burn summary dialogue when it says the burn was successful. You have to open the Nero detailed burn report to confirm all is ok. I have seen the summary say all is ok and the detailed report say it found an errror. And I confirmed the error was real by doing a byte for byte comparison with another tool. Unbelievable but true.

[rjm]

Very nice, thanks! I'd suggest a couple of things: emphasize that a real backup is in a separate physical location (a
"backup" that lives on a separate drive on the same system fails when the building burns), and talk about cascading
backups along with how many backups are needed (especially in the case of user error, you may need to go back
a while to get the good data, sometimes it is a while before you discover that you accidentally deleted that really
important file).

j

Good points John.

Here is a tip for free high-capacity offsite backups. Find a trusted friend or family member that is salivating over your media collection, and offer them a read-only copy if they supply the hard drive space. I've had great success with this.

[hit_ny]

I am not sure how much to trust the SMART data. Steve Gibson of GRC says the value of SMART is very dubious because every manufacturer implements it differently, and his experience has been that SMART is useless for predicting drive failure. I think the Google study corroborated this.

SMART attributes can be monitored as regularly as the tool watching them is told to do so, usually hourly. So i don't really understand why steve gibson says they are not useful. Don't recall if the Google report had anything to support this.

Each manufacturer does have different values but that's taken into account by the SMART Thresholds specified for each HD. The manufacturer sets a threshold which if the attribute falls below, then the drive can be considered unreliable in the near future. Usually there will be a warning given by the SMART app in this case.

Steve Gibson's tool is certainly useful in recovering data if windows for any reason is unwilling to recognise a drive *provided*  that the drive still powers up. It seems more geared for recovering than just plain testing which the vendor's fitness tool can also do, it's what's used to RMA a drive or not.

I think there are 2 failure mechanism to worry about: mechanical and electronic. The life of electronics, on the other hand, may be shortened by frequent thermal cycles. So the best compromise may be to leave the drive powered up but configured to spin down. I have never seen a definitive study to confirm this though.

I have the feeling that the mechanical is the more vulnerable of the two. Electronics in my experience either works or does not, the current flowing to the HD is regulated by the SMPS and is a low 5V. Having said that there really is no user test out there specifically for the electronics side, other than watching the important SMART attributes which displays its result as a product of the two.

I wonder which of the two historically has been the leading cause for HD failure, guess that kind of info is only available with a signed NDA and would also vary depending on the vendor & family of the HD in question.

One more tip. Do not trust the Nero burn summary dialogue when it says the burn was successful. You have to open the Nero detailed burn report to confirm all is ok. I have seen the summary say all is ok and the detailed report say it found an errror. And I confirmed the error was real by doing a byte for byte comparison with another tool. Unbelievable but true.

I usually follow a write with a verify but are you saying that even if the verify says its ok, its necessary to look at the detailed report ?

why's it say its ok then ??

[rjm]

I wonder which of the two historically has been the leading cause for HD failure, guess that kind of info is only available with a signed NDA and would also vary depending on the vendor & family of the HD in question.

Yeh, even the Google study did not disclose manufacturer names because they consider it competitive proprietary information.

I usually follow a write with a verify but are you saying that even if the verify says its ok, its necessary to look at the detailed report ?

Yep, that's what I am saying. Did not believe it myself until I verified that the error reported in the detailed report was real. Don't get too worried though, I see this very rarely and I burn a lot of discs.

[rjm]

I caught one of those nasty Nero burn verification errors tonight. Notice that if you do not expand the details you only see the success message.

[nunisimo]

hi..

this is excellent..

do you have any recommendation on "how" to back up.. software etc..

[rjm]

hi nunisimo,

Have you read part 2? If yes, then I do not fully understand what you are asking. Please provide more detail on what you would like added to this tutorial and I will do my best to provide.