Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kiwi]

What is the most cost effective method of backing up media library?

I am planning on ripping my music collection to APE files.  After I have done this, I do not want to lose all of that data if my HD crashes.  Sure, I have the CDs, however, the time lost would be tremendous.  I have ~750 CDs, so if each one takes 8 minutes to rip and encode, the you are looking at 6000 minutes of encoding... or 4.1667 days of encoding.  I'm also looking at somewhere in the range of 300 GB to start with.  I don't mind doing it once, but I certainly don't want to do it again, and I am williing to buy extra equipement to back it all up.  

I am sure that there are other people out there with large music collections that they don't want to see destroyed by a faulty HD.

I can think of a few options:

1) Copy each album of APEs onto a CD.  Since CD-Rs are cheap, just rip and then burn and then store back on the spindle.  Rinse and repeat.  

Pros: Cheap. CD-RW will already be in system and blanks are quite cheap.

Cons: This however, is probably the most time consuming option.  Add 4 minutes to every rip that takes place.


2) Copy the albums to a DVD-RW+RW media.

Pros: Disks have higher capacity than do CD-Rs, so fewer disks will be needed.

Cons: Have to by DVD-RW+RW burner.  Plus, even if you are storing 8 albums per DVD, you are looking at 100 or so DVDs.  Still not a nice clean system.


3) External Tape backup

Pros: Even larger media. Increasing backup size is easy, get more tapes.  

Cons: Linear access to data.  Cost.


4) Buy Extra HDs and Mirror the files.  Possibly using a program like MirrorFolder to help automate the process.

Pros: Gives you an exact duplicate of the files.  Easy process.

Cons: Cost?  For every hard drive you buy for storage of your media Lib, you have to buy a second one for the backup.



This final method is what I intend to do.  I am getting a 4x  5.25 external bay firewire device and putting my drives in that.  For starters, I put in drives 1A and 1B.  Rip and fill 1A, copying all files to 1B.  Once 1A and 1B are filled and mirrored, remove 1B and insert 2A and 2B.  Repeat process (ripping to 2A, copying to 2B) until done with music.  Keep drives 1A and 2A in the multibay firewire enclosure for use and keep 1B and 2B in a safe place for problems.

I plan on getting 200GB drives since they seem to have about the same $/MB as other drives these days and they decrease the number of drives that I will have.

I'd love to hear any other suggestions, recomendations or tales from people reguarding backup.


-kiwi

[Jaguu]

I have Raid 1 (mirrored disks).

Pros:
Raid1 is cheap, either onboard or with separate controller. If a disk fails, you just replace your disk without major interruption.
Cons: Not safe against power-strikes unless you have UPS. If you incidentally delete files, they are deleted on both disks.

Probably the optimal solution is a combination of Raid 1 and half of your final solution. You will have one copy on your Raid1 on your pc and a backup copy on external disks that you can store away safely.

Or even better, you put your mirrored disks in your pc inside removable disk cases. Then, once everything is ripped you deactivate the mirror and take the mirrored disk out. You could even replace it with another disk and mirror again. Removable disk cases are definitely cheaper than external Firewire/USB2.0 cases.

On the other hand in some countries like Switzerland power-strikes are almost inexistent. Never had one since I bought my first PC some 15 ago. So this could be a minor risk depending on the country you live in.

[knickelfarz]

I wish I had the money for a *professional* RAID system  

I'm using two external Firewire/USB2-Combo drives for that purpose, with Maxtor 4D080H4 250GB/5400RPM harddisks inside. The drives are quite silent and don't need extra fans for cooling and they are fast enough ... they can be transported comfortable and can be connected to almost every computer, even if there is no Firewire - fast filesharing  

For backing up I use a simple batch for Xcopy, which was posted from C't magazin once:

set Source="D:\MyMusic"
set Target="E:\Backup\%date%"
set backlog="E:\Backup\backup.log"
set templog="E:\Backup\temp.log"
;
echo . >>%backlog%
echo Start Backup: %date% %time% from %Source% to %Target% >>%backlog%
md %Target%
xcopy %Source% %Target% /m /s /v /c /i /f /h /k /o /x /y >>%backlog%
echo Start Verification >>%backlog%
start /w windiff.exe -Sdx %templog% -T %Target% %Source%
find "different" %templog% >>%backlog%
echo Backup Finished >>%backlog%
start notepad.exe %backlog%
;
rem Erstellt 2003 von Axel Vahldiek / c't
rem mailto: axv@ctmagazin.de

All you have to do is extract the files "gutils.dll", "windiff.exe" and "windiff.hlp" from the "Support\Tools\SUPPORT.CAB" of your Win2K- or WinXP-CD to your windows folder.
Copy the code above into notepad and save it as "backup.bat" (for example).
Afterwards adapt the pathes in the first four lines of the batch according to your needs.
It's fast, simple and free  

[DocLotus]

I am using 3 Maxtor 80 gig drives...

* One drive for the working drive.

* Two for back-ups only.

Am running Win XP & find the Backup program that comes with Win XP to be very good (once you get around the password problem).

I use the backup scheduler to do a Daily backup on Mon, Tue, Wed, Thu, Fri & Sat around 10:05 PM.

I use the backup scheduler to do a Full backup on Sun around 10:05 PM.

Am currently looking for a housing adaptor that I can install into a 5 1/4 inch drive bay that will accept 3.5 inch harddrives as a plug in.  That wayI will be able to switch drives every week thereby giving me a history of backups.

[KingSparta]

DocLotus

what are you backing up?

Just music?

don't see why you need to back that up every few days.

[JorgeGVB]

I use a 2nd HD for backups.  I use Second Copy 2000 http://www.centered.com for backing up all important files on a daily basis.  Backups are automatically made in the middle of the night.

[DocLotus]

KingSparta;

I have set the Win XP Backup program to back up only files that have changed since the last backup for Monday - Saturday.  A daily backup is around 750 MB.

If you look at your files with a file manager that shows file Attributes you will see several letters such as R,H,S,O in the Attribute column.  Each time a file is changed the Attribute changes.  The Backup program uses this information to determind what needs backing up on a daily basis.

I have set the Win XP Backup program to back up ALL files. A Full backup is around 35 GB.

All Attirbutes are reset with a full backup.

This method is fully automatic & needs no attention from me other then to make sure the backup drive is not full.  If it is I switch drives for the next week.

Using this method I can store one full backup & about 30 daily backups on a single 80 GB drive.

[xen-uno]

> I wish I had the money for a *professional* RAID system

"professional"...as in a stand alone unit? Like a network storage device? That would be nice, no doubt. You can set up a RAID Level 1 mirrored volume for around $200. The controller (Adaptec 1200A) is about $60. I don't know what kind of space you need, but my (2) 60 GB IDE's were about $140 together. Is doesn't eliminate the need for a backup...but sure reduces the frequency. I'm considering the Sony multi-format DVD writer for that purpose.

Patiently waiting for Blu-Ray (or similar...spec'd at 23 to 27 GB per disk) HD-DVD writers to come out in force (and then drop in price). That may be a while, though.

10^-27

[KingSparta]

i agree with what xen-uno said using IDE RAID and a cheep IDE RAID card you can buy any drive case to put them in or if you have room in your current CPU case that will work also.

Rule #1 always buy A computer with a full tower case

[LisaRCT]

I use a 2nd HD for backups.  I use Second Copy 2000 http://www.centered.com for backing up all important files on a daily basis.  Backups are automatically made in the middle of the night.

I do the same . . .  Second copy allows simple copy , exact copy, synronize, etc. allowing me plenty of options.
Also, my automatic 'simple copy' setup will NOT delete any files on my back-up hard drive. This means if I lose my main library drive to a software error or accidental delete (unlike a mirror RAID setup) my backup will not also be erased.
Occaisionally I run an exact copy to clean up the accumulated trash, but ONLY after checking that all my original files are in fact intact, as I do NOT want to have the program delete my back-up to match a lost library costing me everything.  As added insurance the files that are culled-out go to a temp folder before I manually delete them, allowing me one more chance to insure they are safe to delete.

I am using duplicate hard drives and have not had any loss.  I run only ATA133 7200 rpm drives, and currently use a Maxtor 60GB for system and software, another 60GB for graphics files and work in process, a pair of Maxtor 60GB's RAID 0 (stripped) to 120GB for Multimedia files, and backed up the three libraries onto a 200GB WD. Looking forward to adding another WD200GB soon.  

[kiwi]

My only hesitation with going with a RAID solution is that all the drives will be running all the time.  I can't imagine having new audio that I add to my system every day, so having RAID done in real time seems a bit of over kill.  Especially, since the backup HDs would be getting lots of wear and tare.  Plus, they aren't electrically isolated from the system.

I'll have to check out Second Copy.  Anyone have thoughts on that vs. MirrorFolder?

-kiwi

[John Gateley]

In any backup strategy, don't forget to consider physical location. RAID is cool (and JRiver uses it a lot), but it is not complete. An external event (lightning, fire, flood, raging guinea pig stampede) will destroy the entire array.

The optimal solution is to get a friend fairly far away, and an extra hard disk and once a week or so copy your data across the net to your hard disk at your friends house (and of course you return the favor to him). This requires a pretty good internet connection.

Or just get a second cheap machine, put it in the opposite end of the house (with wireless) and use that. Not optimal, but better.

RAID is fairly expensive as a backup solution: it is intended for hot swapping - machines that cannot go down, so when the disk dies, you just plug another one in and live for a while with degraded performance. Don't forget to count the cost of having spare disks around.

Finally, consider the value of the data (you already made a start in that direction). How much does it cost to restore the data vs. how much to replace it. Your replacement cost is the ripping cost. Your restore cost depends on your backup strategy. Your idea of backing up to CD-ROM is bad because the restore cost is as great as the replacement cost (either way, you are still spending the same amount of time loading CDs).

j

[dobon]

Some arguments for cd-r backup:

1: Extremely cheap
2: Robust and works (almost) forever
3: Portable and compatible with any computer
4: When the backup is obsolete, the cd-r's can still be used on portable cd-player (if mp3/wma) or on another computer without high-speed internet connection like in the summerhouse.

Burning is easy work. You can listen to music or surf the internet while you wait.

I register the content of each cd-r in a simple excel-sheet using copy and paste from MC statistics.

This works well for me with appr. 400 backup cdrs now, mostly ape and WMA lossless.

[DocLotus]

>>> In any backup strategy, don't forget to consider physical location. RAID is cool (and JRiver uses it a lot), but it is not complete. An external event (lightning, fire, flood, raging guinea pig stampede) will destroy the entire array. <<<

Thats exactly why I want to use plug-in removable hard drives for my system.

As I continue to update my system I always have an old drive or two left over from the update.  I put them into a plastic holder & use them as plug-ins.  When one is full it goes to another location for storage.

This way I will never again suffer from a total loss of data.

Not only do i have 4000+ ripped music files to securely save but I also have over 200,000 images that would be hard to replace if lost not to mention all the years of programs that I have written that are not replacable.

[dobon]

It is necessary to distinguish between the day to day backup needs for data that change continuously vs. static data like mp3 files and images.

In the first case you need a dynamic automated system. In the second case storing the backup on a HD is a good solution of course, but more expensive than cd-r's which also are more versatile.

[Mastiff]

Why not do something with the backups? I use  my two 120 gig backup harddisks as the basis of my car MP3 player (30 000 + tracks in the car - I'd like to see the cd changer that can match that! It'll have to be as big as the car!) and use WLAN or the regular LAN to do the backups. And the carputer is actually a full computer without monitor and peripherals (but with an IRMan) built into a "metal" suitcase, so I can move it around. Last weekend I used it on my parents' cabin by the sea. Neat! And I of course control it all with NetRemote and IR remotes via Girder, like I do with my home system.  

Oh, and I use a program called FileSync whic has both sync on date and full bit by bit comparing. It's obviously transferring some part of itself over the network since the comparing takes a lot less time than it would take to send all that data over the network.  

[KingSparta]

>> Some arguments for cd-r backup
I can't see why anyone would do this ever

it reminds me of backing up to a floppy disk

if you want to back up to CD use a DVD writer you get 4.7 gigs per disk, but the requirement of putting disks in and changing them etc would be a pain.

[dobon]

I can't see why anyone would do this ever

Remember - this is a once only operation for each file. The backup is a small extra task after ripping and tagging.

Hei Mastiff. Your system sounds really impressive. I guess you need a high level of computer knowledge to build and maintain that though.

[knickelfarz]

Remember - this is a once only operation for each file. The backup is a small extra task after ripping and tagging.

If one uses the great possibilities of MC to work on collections (correcting tags, adding comments, renaming filenames and so on) there's something more to do...backing up to hundreds of cds would make a special kind of dj out of you

[Mastiff]

Thanks, dobon! Actually I'd say that the setup/building phase takes a certain knowledge, but the maintainance is pretty easy. The labour intensive part (but not by any means brain intensive) is to tag, gain and sort all new mp3 files.  

[knickelfarz]

And the carputer is actually a full computer without monitor and peripherals (but with an IRMan) built into a "metal" suitcase, so I can move it around.

How do you manage to dig in 30000 tracks without monitor? Quite good memory?  

[TimB]

I backup all my essential data (12,000 MP3's and 5000 photos and sometimes even my wife's ) every night between 2 and 3.30 to an HD on a different machine (the one I replaced with my current main machine) on my network using Karen's Replicator  http://www.karenware.com/powertools/ptreplicator.asp which I probably learned about here.  Very cool piece o'shareware.

-=Tim=-

[Mastiff]

How do you manage to dig in 30000 tracks without monitor? Quite good memory?  

My friend, did you not read the rest of my post? That part about "NetRemote"?   Also if you take a look at my webpage you can see how I manage CDs with IR remotes.  

[knickelfarz]

My friend, did you not read the rest of my post? That part about "NetRemote"?

Sorry for not being familiar with your setup yet...after some reading on your homepage I discovered that you are using a Pocket LOOX as a "monitor" for the mentioned application (that was my question   )...I certainly will have a closer look at your homepage again, I'm just building up a homenetwork similar to yours

[Mastiff]

No problem. But it's not a monitor, it's a full bi-directional remote control. I can access any track on the system within 10-15 seconds. It also controls my home theater and lots of other stuff. Pretty cool indead.  

[Jaguu]

Following the discussion in this thread, I adjusted my backup strategy and think to have now a pretty fast, reliable, secure and not so expensive backup system by using a Giga-Ethernet connection between my two pc's!

I have two pcs, a multimedia/office pc and one used mainly for video conversion and backup. After the last breakdown of my pc last January I decided that I could not go on with one single pc alone at home, as it was a real single point of failure - I was without pc for over one week!

Both systems have a motherboard including an onboard Raid controller. Extra onboard Raid is very cheap and easy to setup. So I set up my multimedia pc with 2 80GB mirrorered disks as drive D: with 2 main directory D:\Users and D:\Media. All of MC9 files are stored in the media folder except of course the program files. So if a disk crashes, I can continue with my work.

But I did not think enough of system crashes and user manipulation errors, so the ideas suggested above with replication to another system inspired me to try out a few things more.

Tried using Karen's replicator program mentioned above and it worked very well, but too slow over a 10Mb/s Ethernet line. So I bought 2 GigaEthernet cards from Surecom for about 55 USD each and an extra 120GB disk for Backup.

My two pc's are connected to a ADSL/Router/Hub that supports only 10Mbit/s. Replacing this with a GigaEthernet Hub/Switch was simply too expensive, so I decided to setup a multihomed network.

I just left the old 10Mb/s network adapters where they were and added the new GigaEthernet adapters to my pc's. So each pc has now 2 Ethernet adapters with 2 subnets. This is what is called a multihomed network.  The GigaEthernet subnet is setup between the two pc's with a direct connection of a crossed ethernet cable, while the original setup tied to the hub became the first subnet.

So I connect through the HubLAN to the Internet from any of the 2 pc's, while I connect through the GigaLAN from one PC to the other for backup and other purposes. And I tell you, this is really, really fast! It took me 70 minutes to replicate all my 40GB big media library and just 10 minutes for all my user data.

The big advantage now is that I can switch pc and set it up for work in a couple of minutes and I have all my user and media data at my disposition in the case that my main pc would totally crash.

By setting up my ZoneAlarmPro firewall in a way that the GigaLAN connection is part of the Trusted Network, I also added extra security, as nobody can connect to my disks through the HubLAN attached to the Internet.

So with about 220 USD and a little study of Microsoft technobabble, I added a lot of reliability, security and backup speed to my home network.

If anybody is interested, I can post the configuration and security settings later on.

Some people use external hard disks connected via USB2.0 or Firewire. Well, these gadgets are much more expensive here in Switzerland than the setting described above.

To add even more security, I could install a removable disk case (which are very cheap) in my backup pc and have a second backup disk to store away safely in some remote place.

[DV8]

Has anyone tried a RAID 5 solution?
I am willing to invest a little cash and time to get a solution to my backup problems. I have over 60,000 songs and more time than I can imagine in my content.
Has anybody any warnings or encouragements on using IDE drives in RAID 5 config under Windows 2000?

[Eccles]

Has anyone tried a RAID 5 solution?

Yes, my home file server runs 10x9GB SCSI drives in RAID5 on an IBM ServeRAID-3H controller.  The most expensive part was the external drive enclosures - I picked up the RAID card cheaply on eBay and 9GB drives routinely crop up at the local Goodwill Computerworks store for around $5 apiece.  I have another three waiting to be added when we hit 85% capacity, but I'll need to buy another enclosure.

For disaster backup, I use a 35/70GB DLT drive (eBay again) and store the tapes in the desk drawer at my office.

[johnp]

Has anyone tried a RAID 5 solution?
I am willing to invest a little cash and time to get a solution to my backup problems. I have over 60,000 songs and more time than I can imagine in my content.
Has anybody any warnings or encouragements on using IDE drives in RAID 5 config under Windows 2000?

For me RAID 5 is the only way to go.  You get 1/n disk waste vs. 1/2 for RAID 1 (mirrored)  The controllers are not that expensive, and all you need is 3 or more drives of the same size.  You can buy attached storage cases like the one I sell than can make multi terabyte arrays, but if you buy 3  x 250 Gig drives and a controller, you can end up with 500 gig of reliable storage for close to $1K.  I could not have imagined that price / GB ratio even a few years ago.

An important point to remember is that any RAID solution is not a backup or disaster recovery solution.  It is a fault tolerant  storage solution.  If you want backup, I say you can't beat tape and a good ol' server side automated application like Veritas Backup Exec.  You can set that to do a full backup when you start (prepare to feed the tape drive for a while) and then set it to do incremental backups after that so it will automatically add new tracks to your backup archive as they are added to your library.  When the tape gets full, stuff in another, and let it go until it gets full and store the full tapes in a fireproof box at Fort Knox or some other place safe dry and cool.

Just my 2 cent worth.

[kiwi]

Are there any reasonably priced tape solutions for backing up 400-600 gb's of data?  

I'm trying to figure out whether a tape drive makes more sense than just getting doubles of all the HDs and keeping them safe and in a different location.  Right now, 200gb drives are going for $190... and 250gb drives are going for about $260 or so...  

I think that for a tape backup system to work, it would need to have over 30gb per tape of uncompressed data. (Since I'm using APEs, I can't imagine that they can be compressed that much more.)  Also, if the size is less than 30gb, then the time it takes to backup will be quite a bit more.  I'd much rather have the 50gb of storage or more.

-kiwi

[Eccles]

Are there any reasonably priced tape solutions for backing up 400-600 gb's of data?

I guess that depends on your definition of "reasonable".  A quick eBay search for DLT shows an external 35/70GB DLT drive currently at $255, and two different DLT autoloaders for $495 apiece.  And that's just on the first page.

[kiwi]

I guess that depends on your definition of "reasonable".  A quick eBay search for DLT shows an external 35/70GB DLT drive currently at $255, and two different DLT autoloaders for $495 apiece.  And that's just on the first page.

I guess that part of it is not really knowing what to look for in terms of tape backups.  

So, they appear to be mostly SCSI (from at least a cursory glance/search at pricegrabber.com) and the 35/70 tapes are $35 or so.  So, about $1 per GB.  

Interesting, with IDE hard drive prices being as low as they are, it makes their price relative to tape backups quite reasonable.   Particularly, if you consider the time saved.  Now, for doing incremental backups... i.e. daily, with significantly less data needing to be backed up, I think that tapes would have an advantage, but I don't see media libraries changing that frequently.  (At least mine doesn't.)  

-kiwi

[KingSparta]

>> So, about $1 per GB.
I remember I Paid $650 for a drive that held 88k

[Eccles]

I remember I Paid $650 for a drive that held 88k

We've come a long way since cassette tape storage.

[KingSparta]

well I played with IBM cards also back in the late 60's (Military Computers).

one card out of order messes up your whole day

[Eccles]

one card out of order messes up your whole day

As one who has dropped more than one JCL deck, I concur.

[KingSparta]

been there done that.