INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Does anyone know of a way to check the integrity of my library?  (Read 13501 times)

steverosspeter

  • Recent member
  • *
  • Posts: 14

Does anyone know of a way to check the integrity of the files in my library?

I've been trying to discover if it is possible to do this with no success.

I suspect I may have some corrupted MP3's or Flac's; as when I run import it says it is adding 251 files, but when finished it reports it has actually imported no new files nor failed to import any files.

Any help appreciated
Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2661
Logged

AndrewFG

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3392
Re: Does anyone know of a way to check the integrity of my library?
« Reply #2 on: April 15, 2014, 10:59:36 am »

If your files are lossless rips from CDs you could try PerfectTunes AccurateRip to check your rips against a master database.

http://www.dbpoweramp.com/perfecttunes.htm

However tools like this are not relevant for lossy rips such as MP3 etc.
Logged
Author of Whitebear Digital Media Renderer Analyser - http://www.whitebear.ch/dmra.htm
Author of Whitebear - http://www.whitebear.ch/mediaserver.htm

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #3 on: April 16, 2014, 08:39:01 am »

Thanks for the suggestions, I'd seen the bad files page, but the files can't be bad files as import tries to import them every time.

My files are about 50/50 mp3/flac but I shall give that a go.

But none of this explains why import shows 251 files being added then not importing anything, no mention of skipping bad files or anything like that.

Would appreciate any other suggestions.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Does anyone know of a way to check the integrity of my library?
« Reply #4 on: April 16, 2014, 08:55:18 am »

I suspect I may have some corrupted MP3's or Flac's; as when I run import it says it is adding 251 files, but when finished it reports it has actually imported no new files nor failed to import any files.

I believe that this is expected behavior if Update for External Changes is enabled in your Auto-Import settings.

If MC detects any changes in the files, it "re-imports" them and merges the changes back to your Library.  If, when it does this, it doesn't actually find any changes from the data already in the Library, then it won't actually do anything, so the report shows empty.  This is easy enough to test.  Disable that option and re-run the Auto-Import scan.

However, if it happens every single time you run Auto-Import, with the same 251 files, this can be a sign that all is not well with your filesystem (because it means the files are showing up as modified, when they're not).  In that case, some basic Disk troubleshooting would be the best course of action.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Does anyone know of a way to check the integrity of my library?
« Reply #5 on: April 16, 2014, 08:58:24 am »

It is also worth noting that every time you run the Tools > Import > Run Auto-Import Now scan, it tries to re-import files "previously marked as bad".  This could also result in the behavior described, if you have 251 files that are actually corrupt on disk.

The bad files wiki page linked above shows how to list these files in your Library, if needed.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #6 on: April 16, 2014, 09:51:10 am »

Hi, Firstly thanks for the replies, Update for External Changes is enabled, I just disabled it and ran auto import, same result; still say adding 251 new files but when finished it reports it has actually imported no new files nor failed to import any files. The detailed report is blank.

Ran it 3 times and got the same result every time.

Created the bad files smart list as instructed and there were no results.

I ran disk check earlier in the week before posting on here as I wondered if there were any disc issues (although it is a only a 2 month old barracuda) and there were no problems discovered.

Anyone got any other suggestions?
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Does anyone know of a way to check the integrity of my library?
« Reply #7 on: April 16, 2014, 10:45:32 am »

http://www.dbpoweramp.com/perfecttunes.htm
However tools like this are not relevant for lossy rips such as MP3 etc.
For what it's worth, I gave PerfectTUNES a try, and did not find it to be especially useful - it also made me question the worth of the AccurateRip database. (which might explain why Media Center does not use it)
 
I had a large number of discs which had been ripped accurately - and confirmed as such in dBpoweramp - but which PerfectTUNES found to have many "errors" in the rips - presumably caused by my disc being a different version from the one compared against in the AccurateRip database.
 
 
And while their "DeDup" tool has been the best tool I have used for finding duplicate files thus far, it is rendered absolutely useless by the fact that you have to confirm every single deletion individually. That's fine for one or two tracks, not for thousands.
Logged

Claude Lapalme

  • World Citizen
  • ***
  • Posts: 233
Re: Does anyone know of a way to check the integrity of my library?
« Reply #8 on: April 16, 2014, 11:12:30 am »

Not the most accurate solution perhaps, but since corrupted files generally cannot be converted, I find that dbpoweramp's file converter, set to "Test Conversion", gives a good reading of bad files as it posts a failure report when a file fails conversion. I'm not saying it's perfect, but I bet it's enough for most files and is quite easy to use. You right click a large group of files, set it to "test conversion" since you don't want an actual extra file, and wait for the results.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Does anyone know of a way to check the integrity of my library?
« Reply #9 on: April 16, 2014, 11:23:10 am »

Is ignore files previously removed enabled?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #10 on: April 16, 2014, 12:17:00 pm »

I agree about PerfectTUNES, there are too many correct but different versions of cd's out there.

I had wondered about "ignore files previously removed", but I have the same result with it enabled or disabled.

The dbpoweramp's file converter, set to "Test Conversion" is an interesting idea; just downloaded it and started it test converting my entire collection, will take a while; I've got >50,000 files to check. I'll post as soon as I get any results.

Any other ideas greatly appreciated
Logged

Claude Lapalme

  • World Citizen
  • ***
  • Posts: 233
Re: Does anyone know of a way to check the integrity of my library?
« Reply #11 on: April 16, 2014, 12:49:25 pm »

Wow! I have about the same number of files. I've done it a couple of thousand files at a time! Hopefully, you won't have a long list of failures. At any rate, I think you can easily copy-paste and print the list.
Logged

AndrewFG

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3392
Re: Does anyone know of a way to check the integrity of my library?
« Reply #12 on: April 16, 2014, 03:19:41 pm »

For what it's worth, I gave PerfectTUNES a try, and did not find it to be especially useful - it also made me question the worth of the AccurateRip database. (which might explain why Media Center does not use it)
 
I had a large number of discs which had been ripped accurately - and confirmed as such in dBpoweramp - but which PerfectTUNES found to have many "errors" in the rips - presumably caused by my disc being a different version from the one compared against in the AccurateRip database.

Hmm. This is certainly OT, but in my case, out of about 900 CDs, it found 12 CDs with ripping errors and 31 CDs not in the AR database. I bought new copies of the 12 bad CDs and I re-ripped the 31 not found CDs with dBPowerAmp on two separate PCs to ensure both machines gave the same check sum. Then uploaded those 31 x 2 rip results to the AR database. Then about one month after that, when the AR database had been refreshed, all of my CD's are now reported as good rips confirmed by the AR database..

Logged
Author of Whitebear Digital Media Renderer Analyser - http://www.whitebear.ch/dmra.htm
Author of Whitebear - http://www.whitebear.ch/mediaserver.htm

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #13 on: April 17, 2014, 08:30:13 am »

Right ran db poweramp's file converter, set to "Test Conversion" only took took 6 hours, quite cool watching the 4 CPU's individually working.
It did find some errors, when I played the associated files most appeared to have a jump in the audio, but all played. I backed up the albums and then deleted them (27 actually corrupted audio files)and ran import again expecting to see the 251 number go down, not to zero as some of the non audio files such as image and log files could also be corrupt.

Result - No change at all; still adds 251 files then nothing. AHHHHHHH!

At least I have removed some corrupted files (all copies of CD's ripped by someone else) from my library, so thanks for the idea.

Any other suggestions?
Logged

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Does anyone know of a way to check the integrity of my library?
« Reply #14 on: April 17, 2014, 11:47:05 am »

I've noticed that FLAC files with internal cue sheets will be "added" every time import runs (even if they are already in the library and have not changed at all).

(With all the other more important things going on, I didn't want to go into more details than "cue support needs to be rewritten from scratch".)

But in this case, it may be the OP's problem.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #15 on: April 17, 2014, 11:48:22 am »

If anyone would be interested, this might be a good call for a new pscriptor scriplet that tests files using whatever means might be available.  For FLAC, it would run flac.exe in test mode to verify the flac; for mp3, it could try to convert the file.  Etc.

It would update a field in MC (of your choosing) to indicate failure status.

You'd select the files, run the command, and wait for the results.
Logged
The opinions I express represent my own folly.

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #16 on: April 17, 2014, 12:34:36 pm »

The Flac internal cue sheets issue sound like a very likely scenario, by "With all the other more important things going on," are you referring to Matt or something else?

Is there any way I could check this?
Logged

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #17 on: April 18, 2014, 12:51:41 am »

heres what I use.

http://www.vuplayer.com/other.php

very easy. picks up things dbpoweramp missed.

Logged

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #18 on: April 18, 2014, 12:53:36 am »

If anyone would be interested, this might be a good call for a new pscriptor scriplet that tests files using whatever means might be available.  For FLAC, it would run flac.exe in test mode to verify the flac; for mp3, it could try to convert the file.  Etc.

It would update a field in MC (of your choosing) to indicate failure status.

You'd select the files, run the command, and wait for the results.

if you need a guinea pig, im yer huckleberry

id love to try it.
Logged

arin

  • Junior Woodchuck
  • **
  • Posts: 97
Does anyone know of a way to check the integrity of my library?
« Reply #19 on: April 18, 2014, 03:18:12 am »

If anyone would be interested, this might be a good call for a new pscriptor scriplet that tests files using whatever means might be available.  For FLAC, it would run flac.exe in test mode to verify the flac; for mp3, it could try to convert the file.  Etc.

It would update a field in MC (of your choosing) to indicate failure status.

You'd select the files, run the command, and wait for the results.

I really would like to have a try...but could it work also with ALAC?
Logged

steverosspeter

  • Recent member
  • *
  • Posts: 14
Re: Does anyone know of a way to check the integrity of my library?
« Reply #20 on: April 18, 2014, 04:11:34 am »

Right I'll give http://www.vuplayer.com/other.php a try and report back
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #21 on: April 18, 2014, 04:19:36 pm »

RE:

If anyone would be interested, this might be a good call for a new pscriptor scriplet that tests files using whatever means might be available.  For FLAC, it would run flac.exe in test mode to verify the flac; for mp3, it could try to convert the file.  Etc.

It would update a field in MC (of your choosing) to indicate failure status.

You'd select the files, run the command, and wait for the results.

if you need a guinea pig, im yer huckleberry

id love to try it.

I really would like to have a try...but could it work also with ALAC?

I have the flac testing portion of the scriptlet done (using flac -t), and have tested it in OS X and Windows.  They only difference really between running flac -t in a command line shell and using pscriptor is that you can get the error message back into an MC field (for sorting or smartlist'ing).

I looked at the decoder source code files for the audiotester (at http://www.vuplayer.com/other.php), and it does a fair amount of checking for various inconsistencies and corruptions.  I tested it - the output is a list of bad files which you then have to work through one at a time (hopefully your bad files are few).  It is probably the best option for Windows.

I wondered how MC would deal with converting a corrupted FLAC - it blindly went ahead and created an output file, despite flac -t indicating the files has a bad checksum.  Bad MC.

Unless there is an ALAC decoder that can detect and flag inconsistencies, I'm not sure there's much that can be done.  I'm relying entirely on command line tools (and any extant Perl modules), so if the tools don't exist, I'm not inclined to start working out file formats.
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Does anyone know of a way to check the integrity of my library?
« Reply #22 on: April 19, 2014, 05:54:12 am »

It's a shame that Apple devices require the use of ALAC, because this feature alone means that I would rather keep my library as FLAC files.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Does anyone know of a way to check the integrity of my library?
« Reply #23 on: April 19, 2014, 11:04:39 am »

Ffmpeg can probably do it.  ALAC files are just MP4s, of course.

http://superuser.com/questions/100288/how-can-i-check-the-integrity-of-a-video-file-avi-mpeg-mp4
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #24 on: April 19, 2014, 01:57:58 pm »

Nifty:

Code: [Select]
$ ffmpeg -v error -i good.m4a -f null -
$ ffmpeg -v error -i corrupted.m4a -f null -
Error while decoding stream #0:0: Operation not permitted

The file corrupted.m4a had a single bit change in a nibble.

@arin - do you have any corrupted ALAC files you'd want tested (to confirm results)?
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #25 on: April 19, 2014, 07:47:03 pm »

Thanks again glynor.  I updated the scriplet to test m4a's using ffmpeg.
Logged
The opinions I express represent my own folly.

arin

  • Junior Woodchuck
  • **
  • Posts: 97
Re: Does anyone know of a way to check the integrity of my library?
« Reply #26 on: April 20, 2014, 04:13:10 am »

As soon as I get one faulty ALAC I will send it to you, MrC. Thank you very very very much indeed!
Thanx a lot to glynor too.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #27 on: April 20, 2014, 01:39:10 pm »

Why don't I just post an update to pscriptor and you can test it out.

Currently only flac and m4a are supported (since they use the flac and ffmpeg binaries).  So long as ffmpeg can determine validity of other file formats, those can be easily added to the script.
Logged
The opinions I express represent my own folly.

arin

  • Junior Woodchuck
  • **
  • Posts: 97
Re: Does anyone know of a way to check the integrity of my library?
« Reply #28 on: April 21, 2014, 03:50:57 am »

I'm still a newbie with pscriptor...but this will be a good chance to get things going with it!
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Does anyone know of a way to check the integrity of my library?
« Reply #29 on: April 21, 2014, 12:07:27 pm »

Great.  Post your questions in the pscriptor thread, and lets see how it goes...
Logged
The opinions I express represent my own folly.
Pages: [1]   Go Up