Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Shady Bimmer]

I've searched the forum and found several questions regarding fix/ repair broken links on import.  Unfortunately only half have really been answered.

I like the feature but unfortunately it isn't acceptable for me.  1) It doesn't work for me, but it may simply be a matter of incorrect expectations.  2) You only get one shot - any broken links remaining after the import are out-and-out deleted

I had asked this question some time ago but never got an answer:  Is the "fingerprint" or "signature" that MC stores specific to a track or to a file?  There is a difference. 

Not too long ago I had a dual-drive failure in my raid array and I lost considerable amounts of data (Just about 6000 songs).  I still have my MC database intact as well as the original CDs and I simply want to re-rip the CDs and have the tracks "drop in place" of those missing.  My library is entirely APE (converted using MC10) but MC is unable to find matches for any tracks and imports them as new   I even tried using the exact same procedure as I had originally: EAC rip to WAV, MC convert to APE and import.  Still no go.  I had asked about recovery from this scenario back in the very early days of MC11 and it was indicated that this feature would work for this but perhaps I am asking too much?

I would also like to have the option of NOT removing broken links on import and only fix those links that may be fixed (IE: leave the rest "broken").  Because MC absolutely insists on removing broken links I would be forced to rip all 6000 tracks then do a single en-masse import, praying for success (I would much prefer to do this on an artist or album basis).   I've seen several requests that have implied similar functionality but have not seen any responses (positive or negative).

My library still has over 15,000 valid tracks and I'd rather not have to start from scratch on the remaining 6,000.  Does anyone have suggestions?

[JimH]

I don't understand what you're asking.  Maybe you could start by explaining "signature or fingerprint".

[Shady Bimmer]

Ah.

My original query (ages ago) was for a feature in MC where a fingerprint would be generated for each track and stored in the database with the rest of the track information.  This fingerprint would identify any given track uniquely and could be used in the very scenario I have:  MC Database populated with data for a particlar track.  The underlying file (.ape, .wav, whatever) gets deleted and needs to be re-ripped from CD.  A fingerprint of the newly-ripped track could be generated and matched to the previous existence, facilitating a "reattachment" of the database entry to the new file.  I don't want to have to re-enter all the track information again - that is exactly the work I am hoping to avoid 

Aside from renaming the file exactly as it had been in the same location is there a way to re-rip tracks from their original media and have them matched against the existing database entries?  When I had asked that previously it was indicated that the "fix broken links" feature in MC11's import would do this.

I originally ripped from CD to WAV and my entire database had later been converted to APE using MC10 at one point.

I was assuming (big mistake on my part) that MC was using a fingerprint or signature to identify tracks.  If the fingerprint is specific to the file, then file format, tagging information, etc would be significant.  If the fingerprint is specific to the track then file format and tagging would not be relevant for the match.

[JimH]

Have you tried the "Track lookup" feature of MC?  It uploads tag info to our database and this can be recovered later.

[Alex B]

Shady,

Could you write an example of the paths and filenames you used originally? I could try to write instructions for connecting your library information with the newly ripped music files.

On today I replaced the virtual location of my about 30.000 files that were on the D:\music, E:\Video and E:\images folders with substituted drive letters X, Y and Z. I also replaced my cover art links. I didn't move any physical files and during the process all library entries lost the file links for a while (purposely). I didn't use import dialog at all, just MC Library Tools.

[Shady Bimmer]

Have you tried the "Track lookup" feature of MC?  It uploads tag info to our database and this can be recovered later.

Hmm.  I hadn't thought of this but I'll try it.

I have used the "upload track info" feature for all of my tracks and CDs so this could be possible. . .

Edit:
I use a large number of fields (especially the "notes" field) and these aren't all populated using the "Track Lookup" feature.  If I use this to populate the basic fields, is there an easy way to then have MC match it to the DB entry with all the info (IE: use one MC database to do the lookup and populate the tags, then use my main DB to actually import these files and "fix" the broken links?)

[Shady Bimmer]

Shady,

Could you write an example of the paths and filenames you used originally? I could try to write instructions for connecting your library information with the newly ripped music files.

The problem is - how do you identify the new track freshly ripped from CD and match it to a database entry?  Filename must remain arbitrary as the filepaths in the database have no correlation to the filepaths of the ripped files.  After entering all the track information into MC I used it to rename/move files to standard locations in the appropriate filesystem.

The issue is that the files had been lost (as in irrecoverably deleted) and I am re-ripping from the original source to replace the missing files.

[Alex B]

The problem is - how do you identify the new track freshly ripped from CD and match it to a database entry? 

Filename must remain arbitrary as the filepaths in the database have no correlation to the filepaths of the ripped files.  After entering all the track information into MC I used it to rename/move files to standard locations in the appropriate filesystem.

The issue is that the files had been lost (as in irrecoverably deleted) and I am re-ripping from the original source to replace the missing files.

This would be easier If you had explained your appropiate filesystem, but I'll try:

For example:

- If the naming system used was "base path\artist\album\what ever filename.ape".

- Make a new empty library and rip a few CDs using this naming scheme: "rip path\artist\album\track number.ape". You can disable the CD database and cover art lookups.

- Change back to the original library and rename the broken library entries for those CDs using the track numbers as filenames. Use the "Rename Files From Properties" tool. MC claims that there were errors while tagging and moving files, but the Filename field changes anyway.

- Change again to the ripping library and use the "Find and Replace" tool for changing the "rip path" to the "base path"

- When you change again to the original library MC finds the files. After that you can mass rename the track number based filenames with MC Tools.

If the folder structure varies try to collect similarly located albums together.

If your original folder naming system was not based on the library information you can "virtually" move the broken library entries into a library information based folder structure even the physical files are missing.

[Shady Bimmer]

This would be easier If you had explained your appropiate filesystem, but I'll try:

Thanks for the suggestions, but again - the filename must be arbitrary (as in - the filepath of the file I want to match against the database has absolutely no relevance whatsoever in any form).  That is why I didn't bother describing it.  You are assuming that the files will be ripped to a filename that follows a pattern.

Think of it this way:

Remove filepaths from the picture entirely.  You have some file called "thisfile.wav".  You don't know any information about this file (no artist, no album, no trackname, nothing) but you know that it has an entry in the database since you had already ripped/entered track info/renamed/imported this file previously.

I do in fact know this information, but I don't want to have to re-enter all the information again.  I am going to try Jim's suggestion but unfortunately even YADB information won't match exactly what I have in my database for every track.  Files/Directories may not end up with the same name - they may be close but they won't necesarily match identically ("Old Friends Disc 1", "Old Friends [Disc 1]" and "Old Friends (Disc 1)" are three separate "albums" as far as MC is concerned but I know they are the same).

This is why I had asked about a fingerprint or signature - MC could identify a file based on the actual file contents and match it to entries in its database.  I was wondering if MC used fingerprints in its database to be able to identify unknown files for possible relinking to existing entries in the database (my exact scenario).

If your original folder naming system was not based on the library information you can "virtually" move the broken library entries into a library information based folder structure even the physical files are missing

Thanks again, I had also indicated that this is not an option.  If I were to spend the time renaming 6000 files individually (manually) I would just rename them to the right place and be done with it.

The issue is that regardless of how I name the files during ripping, if they don't exactly match the MC database I have then filename becomes irrelevant - either it matches exactly or it doesn't match at all. 

Fingerprinting solves this, which is why the concept has existed for decades.  As an example, I support 14 versions of OS distributions from five different vendors:  Tens of thousands of files, each of which may have multiple "versions" from tens of thousands of patches.  To meet audit requirements I can identify exactly where any given file in any of those distributions originated based on a signature that I can generate (without any other knowledge).

I know MC can find "duplicates" and it can also "fix broken links", but there are several algorithms for doing these.  I'm curious if the method MC uses can be used to help me in my situation.  I had thought the "fix broken links" feature would work, but I have not been successful with this.

[hit_ny]

You're asking if MC can,  using fingerprints recognise if a newly ripped file exists in the database ?

No.

Neither can YADB for that matter or there would not be multiple versions of what is essentially the same track.


Its an interesting problem, lots of systems out there use checksums for this. You would have to rip the audio file using the exact same encoder, even then i think it would not be 100% reliable.

Question i wonder is can an encoder *always* generate the same file, bit perfectly when ripped from the same source ?

my guess would be yes, what about the ripping part, now if that was done securely this (in theory) would ensure a bit-perfect rip + encode any time.

Provided of course your cd did not get damaged.

To test this, rip a track in secure mode, create an sfv or md5  for it. Rip it again (secure)and use the previous sfv /md5 to identify if its the same track. I have a feeling the previous checksum may not always match the newly ripped track.

King has a dupe master plugin that could be used in this case. It generates a checksum and saves these in the database whilst ignoring the tags. Maybe for the future you could use king's plugin to give you this. Integration with fix broken links is prolly what you want maybe they do that in v12.


BTW You don't come across this problem with the various OS files you mentioned earlier as you are never able to create a OS file (as you can with ripping a CD), just copy it, or verify if its there or whether its different from a previous OS version.

[Shady Bimmer]

You're asking if MC can,  using fingerprints recognise if a newly ripped file exists in the database ?

No.

That's what I was inquiring about.  (Thanks).

Neither can YADB for that matter or there would not be multiple versions of what is essentially the same track.

This I know, otherwise I wouldn't have this problem.  There are databases capable of doing this, though.

Its an interesting problem, lots of systems out there use checksums for this. You would have to rip the audio file using the exact same encoder, even then i think it would not be 100% reliable.

This isn't exactly true.  The difference between a fingerprint and a signature is just this:  A fingerprint is typically based on characteristics of a file that uniquely identify it.  A signature is typically based on the actual bits in a file.

A fingerprint is generally unique and is typically used to identify an unknown.  Fingerprints are generally not suitable for validation.  A signature, on the other hand, is used for validation but may or may not be suitable for identification.  Simple checksums are a perfect example of this.

Audio fingerprinting has been around for quite some time and ther are several active projects around it.   There are even systems where you can play a song over the phone and have it identified by an automated service (artist/track).

That is where the "file" vs "track" question came from in my original post:  If the signature was based on the raw bits of the entire file then it isn't possible to identify an APE and WAV as being the same track without decoding the APE.  If a fingerprint of the track itself were used, then both the APE and WAV would have the same fingerprint and be identified as being the same.  Ripping and encoding errors would not generally affect the fingerprint, while they do render signatures useless for comparison in this case.

Do a google search for 'audio fingerprinting' and you'll find details, including links to free fingerprint databases.

BTW You don't come across this problem with the various OS files you mentioned earlier as you are never able to create a OS file (as you can with ripping a CD), just copy it, or verify if its there or whether its different from a previous OS version.

Acutally you do come across this.  One case is identifying exactly which patch revision  any arbitrary file came from (you can go back and search through them all for identical binaries or simply keep a fingerprint database).  If you need to do this on a large scale (Over a quarter million desktops and over 10 thousand servers, just in north america) full-file comparisons become impossible.  Another case comes from the inherient lack of trust on the part of audit groups - from a consumer point of view just take my word that this is indeed something you want (even though I loathe it).

You can alter OS core files given sufficient privileges, and it does happen (hence the lack of trust on the part of auditors)

If only I had another 3 hours/day and another day/week I'd probably be able to start working on a plugin for fingerprinting.  Unfortunately I've already started on enhancements to the md (raid) driver in the linux 2.6 kernel to help prevent future losses similar to what I've recently experienced and this is taking the little free time that I don't have. . .

[hit_ny]

This isn't exactly true.  The difference between a fingerprint and a signature is just this:  A fingerprint is typically based on characteristics of a file that uniquely identify it.  A signature is typically based on the actual bits in a file.

A fingerprint is generally unique and is typically used to identify an unknown.  Fingerprints are generally not suitable for validation.  A signature, on the other hand, is used for validation but may or may not be suitable for identification.  Simple checksums are a perfect example of this.

Audio fingerprinting has been around for quite some time and ther are several active projects around it.   

Good description.

There are even systems where you can play a song over the phone and have it identified by an automated service (artist/track).

You are referring to shazam, using the stuff that Phillips did for them. I wonder how effective it is ?

Do a google search for 'audio fingerprinting' and you'll find details, including links to free fingerprint databases.

I did look for this some time back, sad thing not a single one of those links has a program that one can download and create their own fingerprints. Unless i missed it. Got the impression they were selling a service rather than a product.

So unless this is something JRiver can easily create, they will have to license some one else's product.   Hmm...

King did look into this some time back to make the dupe master plugin..and i think he liked the music brainz stuff, but was not able to finish it, can't recall why.

Acutally you do come across this.  One case is identifying exactly which patch revision  any arbitrary file came from (you can go back and search through them all for identical binaries or simply keep a fingerprint database).  If you need to do this on a large scale (Over a quarter million desktops and over 10 thousand servers, just in north america) full-file comparisons become impossible.  Another case comes from the inherient lack of trust on the part of audit groups - from a consumer point of view just take my word that this is indeed something you want (even though I loathe it).

You can alter OS core files given sufficient privileges, and it does happen (hence the lack of trust on the part of auditors)

Why would you need a fingerprint for these files ? assuming the filename itself has not been modified. If it has then sure.

If only I had another 3 hours/day and another day/week I'd probably be able to start working on a plugin for fingerprinting.

Thats sounds nice. But aren't the algorithms for this fingerprinting stuff patented.

[Shady Bimmer]

I did look for this some time back, sad thing not a single one of those links has a program that one can download and create their own fingerprints. Unless i missed it. Got the impression they were selling a service rather than a product.

One reference page for audio fingerprinting: http://www.icasit.org/ecommerce/audio_fingerprint.htm.

MusicBrainz is the only "free" software I found doing a "10-second" search.  It has a "tagger" that generates fingerprints, but I don't know if these are available outside of the app or they are sent solely to the MusicBrainz database.

I haven't searched SourceForge yet but it is becoming a hot technology and several music services (including "Amazon") are rumored to be looking into their own engines.  It would seem to be valuable for an application such as MC.

Why would you need a fingerprint for these files ? assuming the filename itself has not been modified. If it has then sure.

/kernel/drv/sparcv9/ip

This is a loadable kernel module for the IP networking stack.  There are over 42 updates to this same file from three different patches.  On any given system if you want to definitively identify exactly which version of the 43 (including original distribution) you either compare the binary to all 43 versions or you compare a fingerprint to a database.

I won't even bring filesystem or volume snapshots into the mix, but fingerprints and signatures solve identification issues around these as well.

If you use the builtin checksum tools you might be able to identify that this file is not what it should be but you can't identify exactly which patch it is part of without additional work.  Fingerprints (unique signatures actually, which can function as fingerprints for multi-purpose use) make this a no-brainer.

Filepaths are not suitable for identifying files.  Filepaths can't identify an instance or version.  Renaming a file loses any form of "identity" association.  There is no way to recover filepath information once it is changed without additional information.

Thats sounds nice. But aren't the algorithms for this fingerprinting stuff patented.

Perhaps - but that doesn't mean nobody can use them.  It also doesn't prevent someone else from developing alternative algorithms, and there are quite a few papers on this topic describing different algorithms.

MC is (at least in my opinion) "Best in Class" and surpasses anything else I've found for media library management.  Audio fingerprinting would be another valuable feature to further set it apart from others.
 

[JimH]

Just an FYI.  MC does do audio fingerprinting.  It doesn't do it for the exact purpose you have in mind.  Like many other things, it might be nice to do, but right now this is not even a remote possibility.

You might try to find a way within MC to get closer to what you want.  If you can talk more in terms of what the program does do than what it doesn't do, it could help.

[hit_ny]

Right it isnt going to happen anytime soon, so hopefully a custom plugin could be done.

Seems there is post  here but nothing yet. That is of course after you sorted out your linux module Hopefully avoid having to use .NET

/kernel/drv/sparcv9/ip

This is a loadable kernel module for the IP networking stack.  There are over 42 updates to this same file from three different patches.  On any given system if you want to definitively identify exactly which version of the 43 (including original distribution) you either compare the binary to all 43 versions or you compare a fingerprint to a database.

I won't even bring filesystem or volume snapshots into the mix, but fingerprints and signatures solve identification issues around these as well.

If you use the builtin checksum tools you might be able to identify that this file is not what it should be but you can't identify exactly which patch it is part of without additional work.  Fingerprints (unique signatures actually, which can function as fingerprints for multi-purpose use) make this a no-brainer.

Filepaths are not suitable for identifying files.  Filepaths can't identify an instance or version.  Renaming a file loses any form of "identity" association.  There is no way to recover filepath information once it is changed without additional information.
Perhaps - but that doesn't mean nobody can use them.  It also doesn't prevent someone else from developing alternative algorithms, and there are quite a few papers on this topic describing different algorithms.

Thx for the explanantion. They use MD5 for the signature but what do they use to make the fingerprints ?

[Shady Bimmer]

Just an FYI.  MC does do audio fingerprinting.  It doesn't do it for the exact purpose you have in mind.  Like many other things, it might be nice to do, but right now this is not even a remote possibility.

You might try to find a way within MC to get closer to what you want.  If you can talk more in terms of what the program does do than what it doesn't do, it could help.

I'm up for any suggestions on how MC might be able to help:

I have a database that includes 6000 "broken" links.  These broken links are from files that were "lost" due to hardware failure.  I had spent a large amount of time inputting data into many MC library fields, some of which is not easily replicatable.

I have the original media for these files that I can recopy/rerip, but the filenames/paths will not match.  I'm looking for a method independent of filename, for "reattaching" the 6000 MC library entries to the 6000 files restored from original media.

My MC library had originally been populated using WAVs, followed by MC10 converting to APE, renaming and moving files as well.

I am now using MC11, but can revert to MC10 if needed (I have multiple copies of the original MC10 database).

Using a sample album (Jimmy Buffett's "Bars")  have tried using MC10 to convert to APE, keeping the original WAVS as well.  I used EAC to do the rip, as I had orignally done and the EAC checksums matched the originals.  I then used MC10 to "Lookup track info" from YADB, which populated tags on all the files.  Alas, despite the fact that this was a single album there was no consistency in information across tracks.  For example, half the tracks had the artist as "Jimmy Buffett" and half had "Buffett, Jimmy"   Nonetheless, I continued on.

I had used a separate standalone library for the MC10 work.  I then moved to another PC using MC11 (My files are on a network drive) and attempted to Import with "Fix or remove broken links"  and "Include network and disconnected drives".  MC11 imported these files as new and removed the 6000 broken links (no files fixed).

Any ideas?

[JustinChase]

Let me start by saying, I didn't read all this thread.  I just read the first couple of items, but if I understand from that, this may help, if not sorry to interrupt...

If you just re-rip all of your tracks with the same folder/path information as you had originally into a slightly different folder, then move all those tracks into the folder where MC expects them to be (from outside MC), the links will suddenly be correct and all will be well again.

Use a different library that you can delete later to rip the tracks, move them with IE, then open your original MC library, and MC should see all the tracks, complete with all of your old data.

Hope it helps...

[Shady Bimmer]

If you just re-rip all of your tracks with the same folder/path information as you had originally into a slightly different folder, then move all those tracks into the folder where MC expects them to be (from outside MC), the links will suddenly be correct and all will be well again.

The problem is there are 6000 tracks to rerip and MC itself has the proper filepaths as I used MC to properly tag and rename all the tracks originally.  Reripping/renaming into the original paths will be very time intensive and could take just as long as doing the original tagging/renaming for all 6000 files.  This is exactly what I'll end up doing if I don't find another solution.

MC has all the info needed and is able to match tracks against YADB.  Unfortunately the information in YADB doesn't exactly match my library (it isn't even consistent within itself) so that information isn't useful either - either the filepaths match exactly or they don't match at all.

I was hoping the "Fix broken links" feature during import would help but I've tried a few albums (over 100 tracks) and haven't had any matches.

[Alex B]

What Justin said was about the same I tried to suggest. BTW, what is so important with your original filenames? MC preserves all library information and the filenames are irrelevant for usability. However, you can also keep the exact original filenames and paths if you copy them to custom fields before virtually relocating and renaming the broken library entries. Afterwards you can use Move/Copy Fields tool for returning the original names.

Let's assume that you have about 500 CDs to rip and you are going to rip 20 CDs at one time.

- With your broken library first make the next fields: "Filename (name) Copy" and "Filename (path) Copy".

- Select the tracks of the first 20 albums and copy the contents of the original fields to those custom fields.

- Mass rename the broken albums like this: "base path"\[Album]\[Track Number].
(e.g. the third track of an album will have this name in your broken library: "base path"\"name of the album"\3.ape.

- Rip your CDs using a different empty MC library and use that naming scheme. The only thing you must check is the album name. One possibility would be to use sequence numbers instead of the album names. The track names would look like "base path"\20\3.ape. That would make this step a bit easier, but you must give the album numbers manually at the previous step.

- Next time you open the broken library those 20 albums will have new disc files. Now you can give them the original names and locations back with Move/Copy Fields tool.

With this system you must repeat these steps only 25 times instead of tagging 6000 new tracks. The biggest job will be ripping of those 25 x 20 albums.

EDIT

You could first make 25 new playlists for the missing albums. Then it would be easier to pick up the correct CDs and go on.

[park]

I completely agree with the the the opinions of the main theme of this topic (mc being able to apply all tag info for one file to another by hcecking a fingerprint) and hope that it is serious considered to be included in a future version of MC. I'm sure this would make re-ripping a practical opportunity for those who dont know how to/dont have the time to/dont want to, carefully re-rip to one kind of folder structure and then attempt the mass "rename from files" thing to cheat MC into applying that tag info.

(brief aside: Why not have an option to "search the local library" for tag info, when ripping a CD, first. If the track durations and no.'s match, then all previous tag info for that library entry would be added to the new file without ever going near YADB.)

But, my main reason for posting on this thread is to ask about the "fix/repair..." feature as it is now. I think that its a great idea, but it's not exactly working how I would like it to. I agree with a previous suggestion of having the option to not blanket delete links that arent found. I think that the "fix" part and the "remove broken links" parts are sufficiently different enough in the possible requirements that would warrant their use, that they should be seperate tickable options.

Also, may I check that this also works with video files. On several occasions the import has re-imported some episodes of south park that i have divX'd on my drive. But I have not touched those files at all.
Moreover and worse, the carefully tagged info (synopsis etc.) is destroyed "by the remove borken links" part of the process. So, the file is being imported for no good reason, (no info about that file has changed), yet the previous tag info isnt transferred to the newly imported one, and then the old library entry for that file is deleted.

Is it that tag info not in the files themselves is not transferred? I hope not.

[Alex B]

I tried to offer a practical solution by using the current abilities of MC.

What I would like to have in future is a way to copy/paste all tags from certain items to other items. MC can copy the tags to Windows clipboard, but can't paste them back.

For example, if I select all 15 tracks of a certain album I can copy/paste all information into an Excel sheet. I would like to be able to edit the information with Excel and paste the information back to 15 selected tracks. Those tracks could be the original tracks or any other tracks. MC could check that the same amount of tracks is selected and show proper warnings before pasting the information back. MC could even have a separate "copy/paste" mode for preventing accidents.

If pasting from other applications is too complicated to program MC could at least be able to copy/paste the information inside MC. It could have "Copy all library information from the selected items"and "Paste all library information to the selected items" tools. MC could warn if the file formats are not compatible (e.g. audio vs video) or if the number of the items doesn't match.

[park]

yeah. it could all be done in a little popup window and you could choose the order of tracks to be copied on the left and to be copied to on the right.

Personally I'm still holding out for the ability to find tag info for just one field. I have a lot of albums missing "date" info, but dont want to do a search on YADB etc. because all the other info I've collected has been customised by me.

[Shady Bimmer]

What Justin said was about the same I tried to suggest.

I am aware of that.  It still involves quite a bit of work.  You are making assumptions that I've already stated are NOT valid.  I know how to use these features of MC (I've been using since MJ).

You are making assumptions about how the tracks are ordered.  I have quite a few multi-disc sets as well as live performances.  Many are not coming from CDA and are in fact CD Data with lossless files (wav, flac, shn) already named with their own directory and file naming schemes.

Since I've already said that depending on filename is not an option why keep coming back to it?  I know how to coerce filenames and brute-force them into place but this will not be as simple as you seem to make it out to be.

Edit:
I have "Submitted track info to YADB" for all of my tracks and I do plan on using "Lookup track info" to help get naming roughly inline with where it needs to be.  Since many tracks had been updated by others when YADB was accepting updates to existing tracks a bit of information won't be entirely correct but I'm hoping it won't be too bad.  A bunch of tag fixes with some massive renames done patiently looks like it will be my ultimate solution.

I do plan on using temporary libraries to do all the work on getting the files in the proper places where my library expects them.

[Alex B]

I am aware of that.  It still involves quite a bit of work.  You are making assumptions that I've already stated are NOT valid.  I know how to use these features of MC (I've been using since MJ).

You are making assumptions about how the tracks are ordered.  I have quite a few multi-disc sets as well as live performances.  Many are not coming from CDA and are in fact CD Data with lossless files (wav, flac, shn) already named with their own directory and file naming schemes.

Since I've already said that depending on filename is not an option why keep coming back to it?  I know how to coerce filenames and brute-force them into place but this will not be as simple as you seem to make it out to be.

A:

I tried to offer a practical solution by using the current abilities of MC.

Usually if someone expresses a specific problem the goal is to find a solution together by discussing about different possibilities. I thought that during a constructive discussion your specific problems could have been helped at least to some extend by gathering ideas from many users. I thought you would be happy of any instant reduction of your workload, but obviously that is not your goal. If you are more interested about future MC development it's OK for me. Sorry, if I disturbed you.

[JustinChase]

Not trying to feed any fires here, but...

If you have to re-rip all the files anyway (because they were all lost in the drive crash), how exactly are you think you might avoid all that work.

Again, I haven't read everything, do sorry if I'm being ignorant.

[hit_ny]

If you have to re-rip all the files anyway (because they were all lost in the drive crash), how exactly are you think you might avoid all that work.

Again, I haven't read everything, do sorry if I'm being ignorant.

My guess is that there is a lot of library only information that pertains to the lost files ie, tag information that is not saved to tags (in the case where the format does not suport tags).

So the idea is to try and get the re-ripped files back to where the library "thinks" they are and all will be cosy. Its very tedious to do this manually, so he was hoping there was some fingerprinting system in place that will instantly identify the newly ripped files and associate all the pertinent library info to them.

It's a tricky problem. People make use of the library to store all kinds of meta-data. If you don't have a backup, and by that i mean library backup+backup drive. You might spend ages entering back in (again). This was not supposed to happen and he did have a RAID setup but then he lost 2 drives, the one thing that a RAID can't protect against

Is this case enough to inspire JRiver to build in some sort of fingerprinting ... maybe

[Shady Bimmer]

Usually if someone expresses a specific problem the goal is to find a solution together by discussing about different possibilities. I thought that during a constructive discussion your specific problems could have been helped at least to some extend by gathering ideas from many users. I thought you would be happy of any instant reduction of your workload, but obviously that is not your goal. If you are more interested about future MC development it's OK for me. Sorry, if I disturbed you.

I appreciate the help and did say thank you.  I did mention more than once that depending on filenames is not suitable but the suggestion kept getting made  (You shouldn't have felt any need to reply with the same suggestion again if you felt I was being unnapreciative).  Yes this will help reduce the load, but we're still talking about a significant amount of work - not just hours or even a dozen hours but my estimates are over 50 hours of work fixing tags and renaming files to the proper location.  That may be peanuts to you but is almost a full time work week for me.  Estimating that I'll only be able to spend a few hours/week doing this I'm looking at months before I get my library restored to where it was before my loss.

I am not looking for future work on MC but rather how MC can may be able to help me now.  As I indicated more than once I had been told previously that MC's "fix broken links" feature should help in this situation.  I tried many times and it does not work for me.  I have heard no suggestions on how that might in fact possibly help me (IE: a hint as to how MC determines a particular file belongs with a particular library entry to "fix" the broken link in that entry so perhaps I can do the needed "massaging" in my work to get this to happen).

I am also curious if anyone has other ideas.  Jim's suggestion of using YADB to help populate tags is usefull and I do plan on using this to help reduce the amount of manual work in renaming files.

My livelihood is leading a team of engineers and being proactive/preventative is something I put value in.  For my life hobbies/interests I have similar goals, especially since I would actually like to be able to use and appreciate the fruits of my labor rather than spending every bit of my free time recovering to that point.  After all, shouldn't the end result be an enhancement to our lifestyles?

[Shady Bimmer]

My guess is that there is a lot of library only information that pertains to the lost files ie, tag information that is not saved to tags (in the case where the format does not suport tags).

Essentially that's it in a nutshell.  In my case the original files never had tags prior to being imported to MC, so the original media doesn't have the information needed to even get back to that point

So the idea is to try and get the re-ripped files back to where the library "thinks" they are and all will be cosy. Its very tedious to do this manually, so he was hoping there was some fingerprinting system in place that will instantly identify the newly ripped files and associate all the pertinent library info to them.

Exactly.

It's a tricky problem. People make use of the library to store all kinds of meta-data. If you don't have a backup, and by that i mean library backup+backup drive. You might spend ages entering back in (again).

Right again.  In my case, I have backups of the MC library (on tape in fact) and counted on the original meda as the backup for the actual files.  The correlation between the two is what got lost and is exactly one of the roles fingerprinting can play.

This was not supposed to happen and he did have a RAID setup but then he lost 2 drives, the one thing that a RAID can't protect against

Exactly.  I have actually had seven failures among six of the same model drive in 1.5 years.  Dialogues with the drive manufacturer have proven fruitless and I have since abandoned these drives (No I won't publish the manufacturer).

The risk was known to me (I am all too familiar with various types of RAID failures) and I got bitten before I was able to do my best effort at protecting against this failure.

[MAPgeekdad]

Just throwing something out here....

What about the library function "export to XML"?  Would this would any more flexibility for matching of the tags?

BTW, I will be going out tonight to buy another backup drive : )

I have been a little lax lately and only have about 60% of my 20k files backed-up.  Haven't lost anything yet in 2 years (reaching for some wood to knock on), but I don't want to push my luck any longer.

[Shady Bimmer]

What about the library function "export to XML"?  Would this would any more flexibility for matching of the tags?

If I had some way to match the properties & attributes in the xml to the actual files it would, but I'm not sure otherwise.

For example:  If MC was able to export its fingerprint for each track into xml, I could probably use a temporary library to import the re-ripped tracks, then export the xml from both libraries.  It would probably take me 15 minutes to put together a perl script that could parse both XMLs and rename the files from the temporary library to the names in the correct library - my job would be done.

I have been a little lax lately and only have about 60% of my 20k files backed-up.  Haven't lost anything yet in 2 years (reaching for some wood to knock on), but I don't want to push my luck any longer.

In the end it all comes down to managing risk and balancing cost with that risk.  Sometimes you end up on the wrong side of the balance

[Shady Bimmer]

Jim / Matt / John

Would it be possible to have the fingerprint exportable in XML with the rest of the fields?

That'd be golden but I don't know what the effort would be or if there's some other reason it needs to be kept internal.

[JimH]

No.  Sorry.  It took a lot for us to work out how to do it, and we only use it internally.

[hit_ny]

Thanks for the suggestions, but again - the filename must be arbitrary (as in - the filepath of the file I want to match against the database has absolutely no relevance whatsoever in any form).  That is why I didn't bother describing it.  You are assuming that the files will be ripped to a filename that follows a pattern.

Think of it this way:

Remove filepaths from the picture entirely.  You have some file called "thisfile.wav".  You don't know any information about this file (no artist, no album, no trackname, nothing) but you know that it has an entry in the database since you had already ripped/entered track info/renamed/imported this file previously.

It looks like you will have to find an alternative way to fingerprint your files (in the future). Or even have a separate program that will do this.

Your situation is much more complicated to recover from than usual, as you say you can't use filename and paths etc.

I tend to think in units of album artists & albums or by radio show. So if such a thing were to happen at worst i would know which albums to rip etc. I never let MC organise the path based on Properties. Since MC can easily display this in panes, i prefer a simple way to represent how data is stored on HD.

But this might not apply in your case, so some way of fingerprinting is required.

[Shady Bimmer]

It looks like you will have to find an alternative way to fingerprint your files (in the future). Or even have a separate program that will do this.

MC already has a fingerprint, as confirmed by Jim, and it works very well with YADB.  Perhaps a future enhancement could be for MC to use this during import to solve this problem.

Your situation is much more complicated to recover from than usual, as you say you can't use filename and paths etc.

The issue of using filepaths isn't really unique to me and it really isn't complicated - there just needs to be a single piece of information that maps a piece of data (the track) to the database entry.

Centralized online databases such as YADB, Freedb, etc can't use filenames either and need some other method to identify tracks.  Fingerprinting is one method of doing this.

Depending on filepaths for recovering from loss is also not dependable, but it may work for most home users.  If you depend on a central database such as YADB or FreeDB then there's no guarantee that the path you originally used will continue to exist and be returned for the same track.

Every ripping program needs some way to name files.  You can choose to use 'track01', 'track02', etc and store files that way, but then you are entirely dependent upon applications to do runtime identification.  If you lose their databases you've lost all metadata and can't identify those files.  If you rename files outside of the app (move to another drive, etc) then you've again lost all metadata and can't identify those files (that's where the "fix broken links" feature of MC arises, but I don't know how it identifies files as matches for broken links or I might be able to make this work)

I look at the problem wholistically based on knowns and unknowns.  The only part of a media file that is guaranteed is the content itself.  Everything else is transient.  It isn't a new concept and absolutely is not uncommon - there has been a lot of research on this subject with a large number of papers published around this very topic.

I tend to think in units of album artists & albums or by radio show. So if such a thing were to happen at worst i would know which albums to rip etc. I never let MC organise the path based on Properties. Since MC can easily display this in panes, i prefer a simple way to represent how data is stored on HD.

I do know which albums need to get replaced and can rerip them easily.  Matching the ripped files to the database is where the issue comes in.

How I name my files isn't entirely relevant - the ability to reproduce it is what is key.  CDA doesn't have a concept of files or filenames, so ripping applications compensate.  If you simply follow a schema [Artist]/[Album]/[track#] then the actual text you use is still relevant.  Track01, track1, track01 are all different.  When artist or album names contain characters not valid in filenames, what did you use in place?  Or did you leave the characaters out altogether?  Then there are tracks from data CDs with naming schemes already applied.  These are generally not unique and some additional facility must be used for naming (an additional directory level for instance) to help ensure uniqueness.  Again reproducability is the issue.

Computers aren't "fuzzy".  While we all know "Mr. E's Beautiful Blues" and "Mr Es Beautiful Blues" is the same track, as far as your computer goes they are not.

In my case, MC has been dead-on in using YADB for identifying unknown tracks.  I just wish it could do the same using its own local database rather than YADB since the actual text in YADB doesn't match my database.

I think the point may be getting missed by some - It isn't that I can't get back to where I was, it is that it will take a significant effort over a long period of time.  I know quite a few people that have had significant losses and most end up starting from scratch without ever saying a word.  If only I had the time for that! 

[hit_ny]

that's where the "fix broken links" feature of MC arises, but I don't know how it identifies files as matches for broken links or I might be able to make this work)

I think all it does is "remove" broken links. But i see how one could understand it might...just might reassociate.

I look at the problem wholistically based on knowns and unknowns.  The only part of a media file that is guaranteed is the content itself.  Everything else is transient.  It isn't a new concept and absolutely is not uncommon - there has been a lot of research on this subject with a large number of papers published around this very topic.I do know which albums need to get replaced and can rerip them easily.  Matching the ripped files to the database is where the issue comes in.

When artist or album names contain characters not valid in filenames, what did you use in place?  Or did you leave the characaters out altogether?  Then there are tracks from data CDs with naming schemes already applied.  These are generally not unique and some additional facility must be used for naming (an additional directory level for instance) to help ensure uniqueness.  Again reproducability is the issue.

I left the illegal chars out, and used the tags. The filenames are not important as MC is what i use to browse with.

Agreed, fingerprints sounds like a good way to uniquely identify a file. I imagine a pretty good dupechecker. Musicbrainz has a plugin, now all we need is a MC wrapper for it.