INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Detecting Corrupt Files  (Read 2710 times)

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Detecting Corrupt Files
« on: March 17, 2014, 08:18:00 am »

isnt there a way to run some sort of batch file like this http://www.hydrogenaudio.org/forums/index.php?showtopic=45380&st=0&p=400038&#entry400038  or this   http://www.hydrogenaudio.org/forums/index.php?showtopic=41786

in conjunction with import or??

or something like audiotester implemented maybe??? the source code is included.
http://www.vuplayer.com/files/audiotester.zip


i dunno. just know those both tell me what the bad files are.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Detecting Corrupt Files
« Reply #1 on: March 17, 2014, 08:30:14 am »

i dunno. just know those both tell me what the bad files are.

The problem with solutions like these is that they only impact one possible type of file that MC can use.

What if the problem is caused by a movie, or an image, or a borked PDF?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Detecting Corrupt Files
« Reply #2 on: March 17, 2014, 08:45:59 am »

I think there was a similar solution that checked *all* of the file types(at least the most common ones).

ive got the files  somwhere I think. 

Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Detecting Corrupt Files
« Reply #3 on: March 17, 2014, 09:52:29 am »

I don't see the point of the AccurateRip database though.
Verification. Not only does my PC think things ripped correctly, I can also see that 30 other people got exactly the same result.
Now I can rule out any possibility of some weird hardware issue causing the rip to appear successful even if it is not a 1:1 copy of the data on the disc.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Detecting Corrupt Files
« Reply #4 on: March 17, 2014, 10:31:16 am »

Verification. Not only does my PC think things ripped correctly, I can also see that 30 other people got exactly the same result.
Now I can rule out any possibility of some weird hardware issue causing the rip to appear successful even if it is not a 1:1 copy of the data on the disc.

Just because you're paranoid, don't mean they're not after you.  ;) ;D

For the record: The chances of this happening are infinitesimally small.  While the birthday problem does certainly apply to the CRC32 hash algorithm, this issue doesn't specifically apply to your use-case.  The birthday problem would apply, I suppose, if someone was intentionally trying to defeat the CRC32 hash check (by feeding it different inputs over and over and searching for one that matches), but with only two "random" inputs (the hash of the information on disc, and the hash of the file post-ripping) the chances this would happen are something on the order of 1 in 4 billion.  In other words, you're far more likely to be hit by lightning or eaten by a shark.

Plus, and more importantly, AccurateRip is just recalculating the same hashes, so the chances of this check resulting in a collision are equivalent, especially given the same inputs.  If you get a hash collision on the actual Secure Rip check, then the AccurateRip hash check tells you nothing (because it is going to result in the exact same collision).

Oh, and if the hash generation is actually failing on your hardware (and producing incorrect results), then that means that your computer can't do basic math properly, which means all bets are off.  In this case, the hashes won't ever match (neither of them) in the real world.  The chances of a "broken hardware" CRC-32 collision matching? I can't even imagine the odds.  I'd bet it is something like the chances that all the particles in the sun are going to spontaneously align their quantum states such that fusion stops, or something like that.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Detecting Corrupt Files
« Reply #5 on: March 17, 2014, 10:40:06 am »

Sorry, I don't think I was clear in my previous post. What I mean is, if the drive is not reading the disc correctly (outputting bad data) or the disc itself is damaged in some way (perhaps a bad pressing, even though it is not scratched up) then my drive could rip the disc and think everything went fine.
If the drive is only checking against its own data, then it will not report an error if it gets the same bad data back each time.
 
If you compare that against the accurate rip database, you can confirm whether or not other people are getting the same results that you are. If they are, now you know that your rips are good.
If you don't do that comparison, they are probably good, but you can't be certain about it.
 
I have had a visibly damaged disc which has verified as being ripped "correctly" after multiple passes on it, because it got the same bad data each time.
When listening to the audio that rip resulted in, there were obvious problems.
Without external verification, that disc appeared to be a "good" rip.
 
 
I don't mean to suggest that two rips with the same checksum could possibly result in different audio without intentional manipulation. (which I have actually seen people claim to be true!)
Logged

astromo

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2239
Re: Detecting Corrupt Files
« Reply #6 on: March 17, 2014, 11:04:45 pm »

Following on from the original post:
I suppose you guys might not be on speaking terms with Spoon over at dbPowerAmp (??) but if you were then perhaps you could run the imports past the AccurateRip database to see if they are good or not http://www.accuraterip.com/ (only applies to audio though).

I've made good use of PerfectTunes recently (you can find it over at dBpoweramp). I was having system hassles [very long story] and on the journey, the partition where I had my local audio files stopped being recognised. I was able to recover the partition using some recovery software but how could I be sure the audio was in a good state and hadn't been corrupted? Also, how to avoid the long drawn out process of backing up the data from network storage?

I had PerfectTunes in my bottom drawer and gave it a go. It identified broken files that I deleted and files that didn't match the AccurateRip database. I used this cut down list to back up all the broken or suspect files rather than the whole lot.

Personally, I found the software worthwhile in terms of time saving. Note my audio collection has about 10,000 tracks, so searching for needles in the haystack was made achievable.

For those wanting more background, PerfectTunes first "listens" to the audio files that it's directed to and builds a database. The local database is then compared with AccurateRip and errors or non-matches are flagged. I'm all for a one-stop software solution, i.e. MC. Sharing this to illustrate the value of addressing glynor's original question/issue.
Logged
MC31, Win10 x64, HD-Plex H5 Gen2 Case, HD-Plex 400W Hi-Fi DC-ATX / AC-DC PSU, Gigabyte Z370 ULTRA Gaming 2.0 MoBo, Intel Core i7 8700 CPU, 4x8GB GSkill DDR4 RAM, Schiit Modi Multibit DAC, Freya Pre, Nelson Pass Aleph J DIY Clone, Ascension Timberwolf 8893BSRTL Speakers, BJC 5T00UP cables, DVB-T Tuner HDHR5-4DT
Pages: [1]   Go Up