Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[chrisjj]

When I send tracks to a device, MC falsely marks some as Duplicate, preventing them syncing.

As requested by AlexB at http://yabb.jriver.com/interact/index.php?topic=56222.msg383787#msg383787 , I have done a test on a small library with logging. With about 1938 tracks in the library, I send 67 by the same artist to a device (hard disc folder), and I get 3 false-marked as duplicate. Here is the log. http://wiki.tango.info/mul/User:Chrisjjj%5CJRMC14_False_Duplicates_log

Any idea what is up? Thanks.

[Alex B]

I think the log does not expose the filenames or any other details of the individual files.

I wonder if MPL files could give us more information.

Could you do the following?

1) Clear the device and delete the Database.mpl file if it is stored elsewhere (It's location can be defined by setting a database path in the Handheld options).
2) Create playlist of those 67 files. Export the playlist in MPL format.
3) Sync the playlist.

Pack the exported playlist and the newly created Handheld database file (Database.mpl) in a zip package, upload it somewhere and post a link here or attach the zip file to your reply if you have that option (check the additional options in the Post Reply form). Also list the fles that didn't sync in your reply.

EDIT

Matt posted immediately after me. Most likely you can find the cause of the problem by doing as he instructed.

[Matt]

Duplicate detection is based on creation of a 'match key' for each file.  By default, the match key is built on (basically) this expression:
[Name]-Clean([Artist], 1)-[Album]-[Genre]-FormatNumber([Track #, 0])-[Media Type]

You could add that expression as a column to any file view and sort by it.  Can you see duplicate files?  If so, can you see why?  Perhaps 'Clean' is making the artists the same when you don't want that?

[chrisjj]

> the match key is built on (basically) this expression: [Name]-Clean([Artist], 1)-[Album]-[Genre]-FormatNumber([Track #, 0])-[Media Type]

Thanks Matt. It would have been useful to know that three weeks ago when I asked.

I'm amazed MC thinks that Duplicate marking without even inspecting the audio is ever going to work reliably. Yes, all the false duplicates have non-unque values for that expression.

How does one change from this default to e.g. Filename?

[Matt]

You need to change tag information to make the files unique.  You can't put two "different" files with identical tags on a device.

Match keys for a handheld are based on tag information since this is immutable across systems and devices.  Filenames are not suitable match keys, since devices use different filenames than the source system and moving between multiple systems would make a mess.

[chrisjj]

> You can't put two "different" files with identical tags on a device.

Since I can put them in the library, I see no sense in being prevented in putting them on a device.

> Match keys for a handheld are based on tag information since this is immutable across systems and devices.

Fine - except that it causes false matches.

You said the current match key is "the default". How do I change it?

[Matt]

> You can't put two "different" files with identical tags on a device.

Since I can put them in the library, I see no sense in being prevented in putting them on a device.

You can copy anything you want to a device using My Computer or Rename, Move & Copy files inside Media Center.

But if you use the handheld engine, you are using a sync engine with intelligence.  It has to be able to know when a device is synced.  It has to work across multiple machines.  It does not look at filenames, because they are not a technically viable solution for this purpose.

So again, if it messages you that there are duplicates, you need to change the files to not be duplicates.  It could be as simple as adding a 1 or 2 to the end of the name.

[chrisjj]

> if you use the handheld engine, you are using a sync engine with intelligence.

But not the intelligence to consider the audio. Even Windows Media Player duplicate detection does that. I suggest JRiver makes MC do that too. Or at least documents in Help its failure to do so.

> It does not look at filenames, because they are not a technically viable solution for this purpose.

The default does not look at filenames. The workaround will do... if you will please tell me how to change from the default.

> you need to change the files to not be duplicates.  It could be as simple as adding a 1 or 2 to the end of the name.

Thanks, but the track titles are currently accurate and I need them to remain so.

[Alex B]

Out of curiosity, I checked a library of about 70000 audio tracks against the expression Matt posted. I created a calculated library field and a smartlist that lists only the duplicates in that field.

I found a bunch of real duplicates that need to be fixed (this didn't surprise me), but only one occurrence in which all tags are correctly filled (names exactly as on the CD cover, track numbers correct, etc) and still a duplicate is listed. These two tracks are from the special three disc edition of the "Dimitri From Paris - In The House" album.The tracks "disc 1: track 5" and "disc 3: track 5" have identical artist and name values. Including the disc number in the expression would fix this problem.

Probably I will never try to sync both tracks, but I may add some additional info to the name tags because it would be good to know what is different in these two tracks.

[chrisjj]

and a smartlist that lists only the duplicates in that field.

That's neat. Can you tell me the definition of that smartlist? I cannot work out from the docs how to get a smartlist to show only duplicates in an expression.

[Alex B]

It's easy.

1. Add a calculated library field with the expression:




2. Create a smartlist that lists the duplicate values in that field:




You must to create a new library field because the smartlist system is not designed to be able to access the more or less temporary expression columns that exist only in some of the list views.

[chrisjj]

>> By default, the match key is built on (basically) this expression:
>> [Name]-Clean([Artist], 1)-[Album]-[Genre]-FormatNumber([Track #, 0])-[Media Type]

> You said the current match key is "the default". How do I change it?

I'd really like an answer to this. Thanks.

[chrisjj]

I found a bunch of real duplicates that need to be fixed (this didn't surprise me), but only one occurrence in which all tags are correctly filled (names exactly as on the CD cover, track numbers correct, etc) ..

Probably I will never try to sync both tracks, but I may add some additional info to the name tags because it would be good to know what is different in these two tracks.

Note that the tags will not then be "correctly" filled as per your definition. I would dearly like a workaround that does NOT require tags to be kluged.

still a duplicate is listed. ... tracks "disc 1: track 5" and "disc 3: track 5"

You've got to wonder why duplicate detection ignores Disc#...

[Neco]

I kind of have this same problem..   So far its just 3 duplicates in the sync list that I can tell.

 the ONLY similarities in the tags are the  Album Name, Artist/Album Artist, Genre Tag,  etc..   Pretty much stuff that is supposed to be the same across files in a set.

File Names are all unique,  Tagged track names, disc numbers, and tracks numbers are all unique.

What's puzzling me is there is not a real easy way to see what the other file it is matching, is.   I.E   if you're going to mark a file as a Duplicate,  would be nice to have the popup give the filename of the "other" file it is supposedly a duplicate of, and its location on disk.

I'll try and find some time to setup this  special  Filter and View  posted above and see if I can figure out the cause... maybe I'll find some useful info.

[chrisjj]

File Names are all unique,  Tagged track names, disc numbers, and tracks numbers are all unique.

Filenames and disc numbers are ignored. But Track names and track numbers are not and should be sufficient to avoid duplicate status.

[chrisjj]

In case it helps other users, the workaround I'm using for this false duplicate status is to kludge the track number, this being a field that is included in the match key but not important to my device setup. I do this using this Mp3tag action:

 Format value "TRACK": $replace(%_filename%,-,)

Since my filenames are unique and of the form 00008637207120-1-1.wma, this action sets the track number to a unique value e.g. 0000863720712011. Media Jukebox mis-displays this as -2147483648 for all filenames here, but does seem to use the true value for the match key, since this does cure the false duplicate status.

[MrC]

The real solution for this is item 11 on my wish list (see below).

[chrisjj]

The real solution for this is item 11 here:

   http://yabb.jriver.com/interact/index.php?topic=68410.0

Thanks, but that says:

The topic or board you are looking for appears to be either missing or off limits to you

[MrC]

I forgot, its in the beta board.  Here's the line item:

11. Sync: Include [Disc #] in MatchKeyExpression for audio

Same-named tracks at the same track # on multi-disc albums prevents synchronization on HH.  Include [Disc #] in MatchKeyExpression, or allow it to be modified so users can amend.

And there is a link to a thread:

The current sync expression fragment used for duplicate detection of non-images (eg. audio) is:

  [Name] Clean([Artist], 1) [Album] [Genre] FormatNumber([Track #, 0]) [Media Type]

Some albums have second (or third, ...) disc's that are identical to the first disc in terms of Track #/Name (and of course Album, Genre, Artist and Media Type), so MC marks all but one track as Duplicates and does not transfer them.  Consider Fleetwood Mac's Tusk special edition:

   Disc 1 Track 3: Think About Me
   Disc 2 Track 3: Think About Me

I can see that the converted files (mp3's) have a [Discnumber] tag, but am not sure if MC is able to compare [Discnumber] against [Disc #]

Would it be possible to include [Disc #] as part of MatchKeyExpression ?

[chrisjj]

11. Sync: Include [Disc #] in MatchKeyExpression for audio

Same-named tracks at the same track # on multi-disc albums prevents synchronization on HH.  Include [Disc #] in MatchKeyExpression, or allow it to be modified so users can amend.

Thanks. Including [Disc #] won't solve the instance I reported here - the 'duplicates' had no Disc #. I really hope you'll take the second option - "allow [MatchKeyExpression] to be modified so users can amend" since that will solve my and every other instance of false duplicates. And I think it would be good to take the first too - add [Disc #] to the default.

[MrC]

Agreed.  For clarification, I'm a user like you - not a JRiver employee.

[chrisjj]

For clarification, I'm a user like you - not a JRiver employee.

Thanks for the correction. But one difference is that you're on the beta test team . Could you be so kind as to link that beta forum post to this thread? I hope it might strengthen the case for the full solution. Thanks.

[MrC]

Your post is being read, and the requests have been made.  When JRiver gets around to it or feels the time is right, it will be updated.

[chrisjj]

OK, thanks.