INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Reflections of digital storing  (Read 3816 times)

Blue Boy

  • World Citizen
  • ***
  • Posts: 124
Reflections of digital storing
« on: November 01, 2010, 01:43:22 pm »

As I become more custome to tranfer and enjoy my music from HD and MC some questions has come to my mind. Maybe anyone has answer or can tell how you do regarding this matter, here I go and all is asked from an audiophil point of wiev.


How many times can you copy a file from one HD to another HD without having the files coruppted or
degenerated?

I just right click copy and paste it, is this safe or is there a better way?

Do you ever/or is there need to defrag the HD you store your music files on?

In Windows 7 there is an option to optimize the (external) disc depending on what you store, why should this matter as everything is in digital format.

Every input on this matter is welcome!

Blue Boy
Logged

pcstockton

  • Citizen of the Universe
  • *****
  • Posts: 1261
Re: Reflections of digital storing
« Reply #1 on: November 01, 2010, 01:59:49 pm »

I dont know a ton about this, but yes simple copy and pasting does not "verify" that things were done without errors.

Use something like SyncToy or another back-up program to mitigate checksum or other problems.

That said i only worry about that when moving around a large amount of data.

-Patrick
Logged
HTPC (ASRock Mini PC 252B: i5 2520M Sandy Bridge/HD3000 - 2.5 GHz - 8GB RAM - 256GB Intel SSD - Win7 Home) > MF V-Link 192 > Wireworld Ultraviolet > Naim DAC > Naim NAC 102/NAPSC/HiCap (PSU) > Naim NAP 180 Amp > Naim NACA-5 Speaker Cables > Naim Ariva

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Reflections of digital storing
« Reply #2 on: November 01, 2010, 03:01:13 pm »

How many times can you copy a file from one HD to another HD without having the files coruppted or degenerated?

Infinitely.  You're thinking about it like it is analog, but it isn't.

If your computer cannot properly copy files from one place to another without errors, it wouldn't be long before it wouldn't be able to boot anymore, and you'd have other VERY SERIOUS issues in everyday use.  Windows creates, moves, and copies files around as part of just running everyday.  Temp files are created, moved, copied and destroyed.  Files are copied to and from RAM and to and from the swap files on the disk, and so on and so forth just as part of the normal operation of your computer.  Modern hard drives have substantial error correction circuitry built into the drives themselves.  Without this capability, they wouldn't be able to function at all.

While a media file may be able to still play if errors are introduced (though there would be issues with playback in most cases), an executable file would not be so lucky.  Data file copies on computers should be bit perfect every single time.  If they are not, you have broken hardware that needs to be replaced.

I dont know a ton about this, but yes simple copy and pasting does not "verify" that things were done without errors.

This is not entirely true.  The simple Windows file copy process DOES have error checking capabilities built in, though it is not extremely rigorous.  It generally relies on your hard drive to be doing its job properly, or to report read/write errors to the OS (which they do).  That is a good tradeoff generally, because the specialized error correction circuitry in the hard drive is FAR faster than anything you could accomplish in software, and the drive has access to telemetry and other information on the state of the surface of the disk itself, which helps it to do it's job.  If you try to do the same thing in software, your computer would run SUBSTANTIALLY slower in everyday use, for absolutely no benefit.  Usually fingerprint verification systems (like MD5 and SHA) are used to verify a copy operation has succeeded when the transfer has happened via an unreliable or unverifiable mechanism, usually network transfers.  So, when you download a big ISO file from the web, it might be a good idea to verify the SHA hash if you can, because transfers from the Internet can be unreliable (you can have a flaky router causing errors at any point along the transfer chain).  Unless you have broken hardware, this should not be needed when transferring from one drive to another.

If you are really worried about it, and doing mass copies from one place to another, it is best to use an application that either does the copy operation in another manner.  Sector level clone utilities are much more resilient, and much faster, if you are copying the entire contents of one partition onto another (so something like Acronis True Image or Clonezilla).  If you are instead copying only a set of files, you can use a variety of copy and backup tools that create and verify file fingerprints to confirm the files were copied correctly (I really like SyncBack SE, which has this feature).  This method will be slower than standard windows copies, but will verify that the copy happened correctly.  I do this when I'm archiving things off onto disks that are going to be put away on a shelf, just to be absolutely sure that the mass copy happened correctly before I delete the original files.

However, this is absolutely NOT necessary unless your hardware or software is broken.  If you detect EVEN ONE file copy error, you should immediately consider your system suspect.  This, generally, is exceedingly rare.  The only times when I've seen this with undamanged hardware was with external drives with very long non-active USB cables.  However, in those cases, it was obvious something was wrong because the drive format itself was damaged and would not open in Explorer after the copy process finished.

Do you ever/or is there need to defrag the HD you store your music files on?

Yes, if you write to the disk a lot, the files can get fragmented, particularly if you run close to the total storage capacity of your disk a lot.  NTFS is WAY better in this regard than the old Windows 95/98/ME FAT formatting system, so normally it isn't a huge problem.  However, running a defrag on your system disk and media disk every so often can marginally improve performance.

In particular, my DVR drives in my system often get pretty heavily fragmented, which can impact performance and cause dropped frames when playing back HD video.  It is simple to set Windows 7 up to defrag the disks automatically on a schedule.  Just set that up and be done with it.

In Windows 7 there is an option to optimize the (external) disc depending on what you store, why should this matter as everything is in digital format.

Well... Sort of, but it depends what you mean by "disc".  If you are talking about external hard drives, you really don't need to worry about this, and the defaults should essentially always be correct.  If you are talking about slow optical media (CDs, DVDs, etc) then that's another whole story.

On conventional media (hard drives), you can sort-of optimize the disk format depending on the types of files you store on the disk, by defining the cluster size used by the format.  However, this has very, very, very limited performance impacts except for in extreme circumstances, and can cause problems if you store a variety of sizes of files on the disk.  This should generally be reserved for experts who know what they're doing.  The defaults are fine.  Usually, the only time a regular end-user would see any benefit from a non-standard cluster size is if you ONLY store very large multi-GB files on a particular partition, and those files "grow" a lot over time (like a DVR drive).  In enterprise applications with certain data sets this type of optimization is common (usually for big iron database servers).

For optical media, the CD burning application should take care of this for you in most cases.  There are ways to manually optimize the burn to improve performance, but this won't impact playback quality, only read speed.  These techniques are most common in large game installation discs where they need to quickly transfer massive quantities of (often very small) files from the DVD to your hard drive at installation time.  Again, you don't need to worry about this.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

Blue Boy

  • World Citizen
  • ***
  • Posts: 124
Re: Reflections of digital storing
« Reply #3 on: November 01, 2010, 03:46:46 pm »

Well I'm very impressed for your answer. Right now I have sort of playing around to learn how everything works and to get a file structure that suits me. I do have deleted my "original" ripped file after paste and copy to another HD about 300 albums ripped to Wav. According to your answer
there should be no problem right? That said if there is no problem with the HDD's. It takes some times to listening to 300 albums and the ammount is growing bigger every day.

I got two HDD's one is 1TB and the other is 1,5 TB. They are different brands and have different rpm speed, one is running at 7200 rpm and the other 5500 rpm I think. I'm soon going to get two equal
discs and use one for the original rippings and the other as a slave for using in my HIFI system.

Thank You
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72544
  • Where did I put my teeth?
Re: Reflections of digital storing
« Reply #4 on: November 01, 2010, 03:48:53 pm »

You can save yourself some trouble and some disc space by using a lossless format.  APE or FLAC are great choices.  There is no difference in the sound produced.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42444
  • Shoes gone again!
Re: Reflections of digital storing
« Reply #5 on: November 01, 2010, 04:21:12 pm »

Lossless compression and the verify functionality is your friend if you're worried about data corruption.

APE verify saved my tail twice (that's punny!), both times due to bad system memory.
Logged
Matt Ashland, JRiver Media Center

Blue Boy

  • World Citizen
  • ***
  • Posts: 124
Re: Reflections of digital storing
« Reply #6 on: November 01, 2010, 04:35:55 pm »

Well Jim H, since I found out about taggings for Wav. and the price for space is not so high anymore I feel that I have no need for another format at least not for now, but I have concider going for a SSD HD but right now that is too much money and as they have limited space I might consider your advice.

Maybe MC in the future could have an option for secure backup copying between HHD's Integrated i the software. Right now I will copying with an software to avoid problems.

Thanks for all advice
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Reflections of digital storing
« Reply #7 on: November 01, 2010, 04:41:43 pm »

APE verify saved my tail twice (that's punny!), both times due to bad system memory.

Bad RAM has been the culprit most of the time when I have had problems with disk writes here too.  Memtest 86+ is your friend.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72544
  • Where did I put my teeth?
Re: Reflections of digital storing
« Reply #8 on: November 01, 2010, 05:46:55 pm »

Maybe MC in the future could have an option for secure backup copying between HHD's Integrated i the software.

Follow Matt's advice about "Verify" if you care about the integrity of your files.

WAV is a bad idea.  Tags are not compatible across programs and devices.  Maybe some day they will be, but I doubt it.
Logged

Blue Boy

  • World Citizen
  • ***
  • Posts: 124
Re: Reflections of digital storing
« Reply #9 on: November 01, 2010, 05:49:46 pm »

I do care very much as it takes a lot of time to rip and tag all those files. Can anyone provide links to
software that do the job?
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72544
  • Where did I put my teeth?
Re: Reflections of digital storing
« Reply #10 on: November 01, 2010, 05:51:40 pm »

I do care very much as it takes a lot of time to rip and tag all those files. Can anyone provide links to
software that do the job?

www.jriver.com
Logged

pcstockton

  • Citizen of the Universe
  • *****
  • Posts: 1261
Re: Reflections of digital storing
« Reply #11 on: November 01, 2010, 05:54:33 pm »


WAV is a bad idea. 

Fully agreed there.  My DAC has a USB input for WAV replay from a thumb drive.  Tagged WAVs do some strange things there.
Logged
HTPC (ASRock Mini PC 252B: i5 2520M Sandy Bridge/HD3000 - 2.5 GHz - 8GB RAM - 256GB Intel SSD - Win7 Home) > MF V-Link 192 > Wireworld Ultraviolet > Naim DAC > Naim NAC 102/NAPSC/HiCap (PSU) > Naim NAP 180 Amp > Naim NACA-5 Speaker Cables > Naim Ariva

craigmcg

  • World Citizen
  • ***
  • Posts: 243
Re: Reflections of digital storing
« Reply #12 on: November 01, 2010, 05:58:53 pm »

I am nowhere near the expert some of the posters on this thread, but I agree with them about using WAV.

Even though drive space is cheap, the space and tag advantages of lossless formats like FLAC or APE make them a much better option for archiving your music.  If you want to make a CD for stereo use, you can re-convert from lossless to WAV with no risk of loss.
Logged

Blue Boy

  • World Citizen
  • ***
  • Posts: 124
Re: Reflections of digital storing
« Reply #13 on: November 01, 2010, 08:10:23 pm »

I know that there is a lot against Wav. but when I have ripped a cd, I  have artist and track title
provided by FREECDB. Then I add what I need to have for my track, so my track looks like this:

All My Loving - The Beatles - Parlophone 1964 the recordlabel and year is added by myself.

As this info is in the track, there is no way I can loose it. The only thing I need for Wav. tagging is
that MC keep the coverart and can do a backup. I have tried and it works great. That is as long as
the software handles this, but I have no plans to replace MC, and I don't use Ipods or other portable
players. If I change my plans I can always make a MP3 or Flac file of my Wav. files to use on this kind of decives.

If anybody wonder why I add label and year info, it is because I have a huge collection of vintage
music a lot of them are compilations where the tracks are from various recordlabels. Most of the
companies don't exist anymore and as this is my no: one hobby I like to learn on what record company/ies a particular artist did the recordings. I also use a catalouging software to add my CD's and the info. Well this might take forever but it keeps me from the beers and booze ;D
 
Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: Reflections of digital storing
« Reply #14 on: November 02, 2010, 06:58:55 am »

If all this tags is present i MC15, you should be able to convert all your present WAV files to Flac (my favorite), and still have the meta data and cover art intact. Make sure you test it on one album, copy the album on the disk, and then take a library backup just to be 100% safe.

Also remember that you can use MC15 to automagically ripp your CD to Flac in a spesific folder structure and then download cover art and meta data. The metadata database is not THAT huge yet, but I'm sure you'll come across some hits.
Logged
- I may not always believe what I'm saying

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Reflections of digital storing
« Reply #15 on: November 02, 2010, 11:27:53 am »

Yes.  I don't think anyone is suggesting you re-rip your original media from scratch.  MC can easily convert the existing files from WAV to FLAC or APE.  We're talking mostly about ripping NEW content to FLAC or APE instead, to avoid this issue in the future.

Also, Matt is referring to a feature of the APE format.  APE embeds a MD5 fingerprint in the file itself, which can be used by supporting software to make sure the file hasn't been corrupted or damaged.

Since Matt created the APE format, he'd probably be the one to ask about how this works in detail.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/
Pages: [1]   Go Up