Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[benn600]

I back all my media up over the internet.  This has worked quite well overall.  One issue I run into, though, is when MC rewrites tags for all a media category.  Such as when I filled the [Album] tag for all photos to match the parent folder name.  Suddenly, I have to upload 85GB of photos.  Or how MC must have rewrote all my tags back FLAC tags on April 6th.  Again, I suddenly have a ton of data to upload.  This amounts to 528GB.

I have full access to the remote server (both Windows).  Is there an application that will intelligently synchronize file changes only?  Clearly, less than 1% of the data in these files has actually changed.  If a server/client software product could run checksums on half, 1/4, 1/8, 1/16th, etc., of the file, it could break down the amount of the file to actually copy significantly.  It probably needs to be on the server itself to get full-speed reading of the files.  To perform this on one system wouldn't really work because it would have to download the entire file and scan bits, defeating the purpose.

I like not being inhibited to major library changes.  If I need to make massive adjustments to tags, I like to be able to without running into this hassle.  This update will take approximately two months of uploading, not to mention stealing my upload bandwidth during that time!

Does SVN handle this on its own?  I've thought about making the directories TortoiseSVN.  Then, I'd see when files changed and could submit changes, followed by updating on the other system.  Although I think it still pulls the whole file each time.  Deltas are merely used on the server, right?

It's a tough issue but backups are certainly essential, even with good server redundancy.

[MrC]

Backups are difficult and both time & resource consumptive, so you need to carefully consider your needs.

SVN is not appropriate for what you are doing - it is a revision control system, not a backup system.

You want a block level backup system, but these are more difficult to implement, so most backup software works at the file level.

You have too much data to use a WAN.  Instead, you want fast, local backups, which you control, and can move media offsite as required for your risk tolerance.  If my recollection is correct for your setup, you have many terabytes of data.  Plan on having at least 3 to 4 times the amount of backup storage as you have local storage.  A single duplicate may or may not work for you.

Have a good read first of the info at:  http://en.wikipedia.org/wiki/Backup, including following the links describing the different backup types, and decisions required.

Certainly you want automated incremental backups, which run daily.  And periodically you'll want a full backup.

[tonse]

I do the same every night with two synology nas servers. Synology uses rsync for backup.  As far as i know rsync backs up at byte level.  So only the different bytes are backed up over wan.  Works extremely fast.  One of the synology servers is very remote.  Only has 56 kbytes internetconnection,  and still the backup of differences goes well over night.
Maybe you can look for rsync on your servers

[bspachman]

I also use rsync (specifically the implementation for Windows called DeltaCopy) locally, and CrashPlan for offsite backups. I find that after the (very) slow initial backup, everything afterwards is very quick--even with bulk tagging changes!

brad

[benn600]

That is such a good idea.  I've used rsync a ton professionally so this would of course be a natural progression.  I like the idea of no problem quick and easy backups.