INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Managing Duplicates -- flexible criteria  (Read 1836 times)

al1947

  • World Citizen
  • ***
  • Posts: 162
Managing Duplicates -- flexible criteria
« on: December 18, 2014, 10:15:01 am »

There was a pretty good discussion of this at http://yabb.jriver.com/interact/index.php?topic=91034.0

However, I am wondering whether there is any way to modify the duplicate search parameters described in that string to allow for a little variance?

Most especially -- duration. It is very common for the exact same recordings of a song to vary by two or three second from one album to another. Is there anyway to manually edit the search parameter to allow for such a discrepency?

Similarly, song names. I am pretty sure a solution is not possible, but I'll still ask. Online databases sometimes will modify a song name with a phrase such as "[album version]" and the like, which defeats a duplicate search. Or famously, the Rolling Stones classic is variously named "Satisfaction", "(I Can't Get No) Satisfaction", or "I Can't Get No Satisfaction". I figure there is no alternative to a manual search, but maybe someone  has a useful idea.
Logged

BartMan01

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1513
Re: Managing Duplicates -- flexible criteria
« Reply #1 on: December 18, 2014, 11:06:03 am »

Give this a read:

http://yabb.jriver.com/interact/index.php?topic=76294.msg516833#msg516833

Now if your track names are different, you have a bigger problem that will require some manual clean up.

One change I made to this process was to create a custom field called Name (base). In that field I put the track name without the 'modifiers' to get to the base track name and then use that for my duplicate search.

Example:
My Great Song (live version)
My Great Song (album version)
My Great Song
My Great Song (recorded in my basement)

All have a Name (base) of 'My Great Song' and all appear as possible duplicates until I deal with them using the post above.

I also have a smartlist that finds all Name (base) entries with modifiers - this is currently looking for anything that ends with a ')', but I change it as needed.  I also have a static play list of files that are 'correct' as-is and exclude that playlist from the smartlist used to find the modifiers.
Logged

al1947

  • World Citizen
  • ***
  • Posts: 162
Re: Managing Duplicates -- flexible criteria
« Reply #2 on: December 18, 2014, 05:25:48 pm »

Interesting stuff. I am testing and making adjustments to suit my needs. However, the .jvi view file attached to the post you reference does not open for me. Do saved views not work on MC20 for Mac?
Logged

BartMan01

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1513
Re: Managing Duplicates -- flexible criteria
« Reply #3 on: December 18, 2014, 09:02:36 pm »

I just tested and was able to load the view scheme in Windows.  How are you trying to 'open' it?  It should just appear in your saved views list once it is in the right place.
Logged

al1947

  • World Citizen
  • ***
  • Posts: 162
Re: Managing Duplicates -- flexible criteria
« Reply #4 on: December 18, 2014, 09:46:20 pm »

As I said, in Mac alternate views do not appear. Just a blank view that can be customized.
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9139
Re: Managing Duplicates -- flexible criteria
« Reply #5 on: December 19, 2014, 12:21:18 pm »

Bartman01, great to see that you found practical use for that workflow. Thank you for linking al1947. I had forgotten I posted that!

al1947,
I wonder if the problem is because a MAC cannot import a Windows generated .jvi file? We need to get someone to share a MAC generated view and see if that works. I don't know if that would be considered a bug or not. Probably not. As requested, here are manual set up steps:

Steps for manually recreating the view scheme below:

(Read this post for a full explanation of what this is)

The assumption is made that you have already set up the playlist group described in the link above.

Start with a new, blank view. Once finished, 'Customise View' should look like so:


Adding The Categories (Panes)

Playlist: Click on the "Add" button, select "Playlist group", choose the duplicates one you made earlier, press OK.

I have over 2000 artists listed, so, to make it easier to get at what I want, I created three artist panes. One with the first letter, one with the first two letters and one with the full artist name. The first two are entirely optional.

Artist (First Letter): Click on the "Add" button, select "Expression". Put Artists in the name field. Copy the code below and paste it into the "Expression to group by" field, press OK.
Code: [Select]
if(isequal(mid([artist],0,4),the/ ,1),mid([artist],4,1),if(isequal(Mid([artist],0,1),1,6),0 - 9,Mid([artist],0,1)))
Artist (First Two Letters): Click on the "Add" button, select "Expression". Put Artists in the name field. Copy the code below and paste it into the "Expression to group by" field, press OK.
Code: [Select]
if(isequal(mid([artist],0,4),the/ ,1),mid([artist],4,2),if(isequal(Mid([artist],0,1),1,6),0 - 9,Mid([artist],0,2)))
Artist: Click on the "Add" button, leave "Library Field" selected and choose "Artist" from the list of fields. Press OK.

Status: This is where the second part of this dual purpose view scheme begins. For this, a different set of sorting instructions is prefferable, and to achieve that, I have used a search list, so...
Click on the "Add" button, select "Search list", put "Status" in the name field. Click the "Add" button on the search list dialogue. Type "Albums With Exclusions" in the name field. Click on the "Import / Export" button and paste the code below into the presented field:
Code: [Select]
p=hidden ~a ~sort=[Album Artist (auto)],[Album],[Track #]Press "OK" twice to get back to the customise view dialogue.

Artist: Click on the "Add" button, leave "Library Field" selected, pick Artist from the list. Down the bottom, select grouping, set the grouping size to three. Press OK.

Album Artist (Auto): Click on the "Add" button, leave "Library Field" selected, pick Album Artist (Auto) from the list. Press OK.

Album: Click on the "Add" button, leave "Library Field" selected, pick Album from the list. Press OK.

Excluded?: This is another search list.
   Click on the "Add" button, select "Search List" and enter Excluded? into the name field. Click on the "Add" button to add a search. Enter "Available" in the name field and press the import/export button. Paste in the code below:
   
Code: [Select]
[Media Type]="audio" ~sort=[Album Artist (auto)],[Album],[Track #]
   Press OK, once. In the "Rules" section, click the "Add New Rule" button. Use the drop menus to create the rule. The rule will be: Playlists | Is Not Any | use the dropdown list and select the "hidden" and "recycle" playlists from the "Duplicates" playlist group you created earlier. Press OK.
   You must add this last rule manually as all of our internal playlist id's are unique.
   
   Included?: Click on the "Add" button to add a second search to the list. Name it "Excluded". Press the Import/Export button and paste the code below into the field:
   
Code: [Select]
p=hidden,recycle ~sort=[Album Artist (auto)],[Album],[Track #]
Press OK three times to get back to the customise view dialogue. That's all the categories added.

Settings

Click on "Set rules for file display", press import/export and paste in the code below, and press OK twice.
Code: [Select]
[Media Type]=[audio] -[Genre]=[podcast] ~sort=[Artist],[Name]Note that I have chosen to exclude podcasts here. You can remove that rule if you would like podcasts to be included in your list, or add exclusions if you find other stuff getting in the way in the list.

Remove the tick from "Use parent scheme rules..."
Allow tree selection = Yes
Allow tree browsing = No
Expand "Advanced" and tick "Filter in both directions"

That's you done. Press OK. Full explanation for using the view can be found in this post.

Hope that helps,
Regards,
-marko

al1947

  • World Citizen
  • ***
  • Posts: 162
Re: Managing Duplicates -- flexible criteria
« Reply #6 on: December 19, 2014, 07:56:35 pm »

Marko,

That was awesome work, which I greatly appreciate your sharing. Enormous help.

Only tinkered with it a little: excluded classical from possible duplicates (too many tracks named "allegro" or whatever); added a sort criteria to the Duplicate Manager View that I alter between bitrate/descending or album/ascending (tend to use one or the other to decide which to keep).

Thank you!
Logged
Pages: [1]   Go Up