INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: playlist to detect duplicates...  (Read 2855 times)

pamillet

  • Recent member
  • *
  • Posts: 25
playlist to detect duplicates...
« on: February 02, 2019, 05:55:04 am »

Hi,

context: I have many issues regarding duplicate files, not due to jriver, but to my backup/sync tools, with many time conflicts that create copies of files... That's not the aim of that post, but just to explain the context

So i have to manage duplicates, and i found the parameter "remove duplicates of..." in playlist setup

that seems fine, and when i use "only duplicates of..." the result seems correct, list of files duplicated... (2 lines with the same file as in this figure...)
see attached files "jriver find duplicate.png"

but when i use "remove duplicates of"...the result is not the same and false !
For example, see jriver remove duplicate.png

the song Pannasie Stomp from Count basie is in the list and it's not duplicate as you can see in the search

see jriver search.png

the setup of the duplicate playlist seems correct...

see jriver setup duplicate.png

If you have an idea where i make an error...

thanks in advance

pam

PS..I have attached image, but dont know how to include in the message... tag img don't works... ?

Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9143
Re: playlist to detect duplicates...
« Reply #1 on: February 02, 2019, 07:52:44 am »

only duplicates will list all the duplicates.

remove duplicates will leave one of each duplicate in the list.

Your search starts off listing all music under D:\hubic-pamcb\musique

It then removes one of any duplicate entries, leaving you with a list of unique tracks.

It's not clear to me what you actually want to do about the duplicates in your library.

If you want to exclude all duplicated files, start off with your 'only duplicates of' list.
Select all files and flag them somehow... you have many options here... you could add them all to a new playlist, or use an unused tag and fill it with "Duplicate" or something similar.

Then, you have...
[media type]=[audio] [filename (path)]=[D:\hubic-pamcb\musique" -[unused tag name]=[Duplicate]

This, on the other hand, will list all your duplicate entries, then remove duplicates leaving you with one of each...
[media type]=[audio] [filename (path)]=[D:\hubic-pamcb\musique" ~sort=[Artist],[Name] ~dup=[Artist],[Name] ~nodup=[Artist],[Name]

If you need any more help or suggestions, just ask.

-marko

pamillet

  • Recent member
  • *
  • Posts: 25
Re: playlist to detect duplicates...
« Reply #2 on: February 02, 2019, 08:13:50 am »

Marko, thanks for your answer,

I have perhaps to explain a bit more my question.

In my example, "only" and "remove" seem not consistent...
- The "only duplicates" give me the correct list of all duplicates files (so i see at least 2 occurences of each file..), I've checked and it's correct, all files in this list are really duplicated...
- But, the "remove duplicate" have others files which are in fact not duplicated !

I gave the example (song of count basie) of a file which is only one in the search, not in the "only duplicate", but which is in "remove duplicate"...

I do not see an explanation in your answer...

other point: in your answer, i do not understand the last

Quote
This, on the other hand, will list all your duplicate entries, then remove duplicates leaving you with one of each...
[media type]=[audio] [filename (path)]=[D:\hubic-pamcb\musique" ~sort=[Artist],[Name] ~dup=[Artist],[Name] ~nodup=[Artist],[Name]

you set ~dup and ~nodup with the same tags... ?

Logged

pamillet

  • Recent member
  • *
  • Posts: 25
Re: playlist to detect duplicates...
« Reply #3 on: February 02, 2019, 08:20:31 am »

marko
another way to explain my questions

using "only duplicates", i have 2976 files, that means because they are duplicated, at max, 1488 songs...
using "remove duplicates", i have 19691 files !!! Of course, most of them are not duplicated !

(I have a total of 27826 audio files in my base...)

pam

Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: playlist to detect duplicates...
« Reply #4 on: February 02, 2019, 08:47:22 am »

Pam:  Marko has asked a really important question:  What do you want to do with your duplicates once you find them?  You might want to remove them from MC, but leave them on disk.  Or you might want to remove them from disk too.  Or something else.

Finding duplicates correctly is actually much more complicated than it seems.  There are a lot of weird cases to deal with.  Like different versions of the same album and stuff like that.

I've built a few different duplicate finder views over the years to try to help.  Once you understand how saved expressions work, in Panes Views, you can REALLY do some powerful stuff!

Here's a thread I posted with some screen shots of one duplicate finder view I made.  You can download the view too if you'd like.  The download link is near the bottom of the thread.

https://yabb.jriver.com/interact/index.php?topic=103098.0

Having a plan before you start, for how you want to handle your duplicates, will help drive the process and tools you build to get this task done.  Good luck!

Brian.
Logged

pamillet

  • Recent member
  • *
  • Posts: 25
Re: playlist to detect duplicates...
« Reply #5 on: February 02, 2019, 11:51:12 am »

many thanks for your help,

more and more i use JRiver, more i'm impressed by the rich functionnalities, specially for library management, find tags in file names, move/copy usings tags... I have more than 50 000 files in JRiver and it's a great help for me. (the weakest point in my experience is lookup info from online database...!!)

in my case, i want really to delete all duplicated files, i know that the issue of duplicate is complex, i have songs versions with different recording dates, and i pay attention to be clear on what is a duplicate. I try a duplicate list using album, artist, name AND duration to have a a strict duplicate criterion, but i do not understand why some files are in "remove" list and not in "only" list...

but in fact, in my context, it's really duplicate created by a syncrhonizing tools i used for backup... and the file name is always the same with (1) or (2...) added
(It's perhaps not a good idea to use sync for kackup, but it's with a low cost storage service hubic)

I study your tools Duplicate Finder.jvi to understand where i'm wrong...

thanks again
pam.
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9143
Re: playlist to detect duplicates...
« Reply #6 on: February 02, 2019, 12:25:54 pm »

So, any reason not to list the duplicates, then filter to keep all the filenames that end in brackets, and delete them all?

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9143
Re: playlist to detect duplicates...
« Reply #7 on: February 02, 2019, 02:13:39 pm »

This is easier when I'm not on my phone...

Quote
I try a duplicate list using album, artist, name AND duration to have a a strict duplicate criterion, but i do not understand why some files are in "remove" list and not in "only" list...
Think of smartlists as progressive filters...

You start off listing all audio at "D:\hubic-pamcb\musique"
This gives you all files, including duplicates and non-duplicates.

Next, you add the modifier "Only duplicates of"...
This removes all files that are not duplicates, and leaves you with a list of duplicates. That is pretty straightforward, and this, you understand.

However, when you add "Remove duplicates of"...
The way this works, is for each set of duplicates, all are removed from the list, except one.
As your initial search of the musique directory also returns files that are not duplicates, when you instruct MC to remove duplicates, it will not remove those, as there are no duplicates of them.

Quote
you set ~dup and ~nodup with the same tags... ?
Yes. The first gets rid of everything except duplicate entries, the second further filters the list by getting rid of all but one of each set of duplicates.

Quote
in my case, i want really to delete all duplicated files <snip> and the file name is always the same with (1) or (2...) added
Build your "Only duplicates of" list...
Then, in your search field, top right of MC, enter [Filename (name)]="1/)"

Now inspect the list and delete as appropriate.
When you're done, clear the search field. The refreshed list will reduce.

Enter [Filename (name)]="2/)" in the search field, inspect and delete as appropriate.
Clear the search field... and repeat.

When you're finished, your smartlist should be empty as there will be no duplicates left.

** As you're housekeeping here, you will want either the recycle file or permanently delete file option. If you just remove from library, the file will be left on your hard drive.

-marko.

pamillet

  • Recent member
  • *
  • Posts: 25
Re: playlist to detect duplicates...
« Reply #8 on: February 02, 2019, 05:31:35 pm »

list with only duplicaste and [Filename (name)]="1/)" is perfect !

thanks again

pam
Logged

pamillet

  • Recent member
  • *
  • Posts: 25
Re: playlist to detect duplicates...
« Reply #9 on: February 03, 2019, 02:52:47 am »

Yes !
That's done, with [Filename (name)])="1/)"

and "cerise sur le gateau", as we say in french, i understand this morning the error i made at the beginning with "Remove duplicates of"

in fact, i tested first "only duplicates", checked that this was correct, and then i replaced "only duplcates" with "remove duplicates"...
and that do not works...
But if, as marko writes, i add  "remove duplicates" to only duplcates"...that's fine...

From an ergonomic point of view, that's not so clear... in fact "remove" has no sens without "find duplicates"...so i was a bit stupid ?  ;) or the tools is a bit tricky...  :-\ ?
perhaps add a comment on "remove dup" to notice that there is no sens to use this without "only dup" before...

pam
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9143
Re: playlist to detect duplicates...
« Reply #10 on: February 03, 2019, 03:12:59 am »

Glad you're all sorted now. I only wish my french was half as good as your english :)
I'm going to run this through Google translate for you just in case anything gets lost. You could let us know how well it does with the translation :)

Quote
so i was a bit stupid ?
Definately not. You are simply just beginning to realise, and release, some of the power under MC's hood. I'm pleased you posted your questions despite feeling stupid as it gave us the chance to help you, and in the process, reassure you that you are not stupid :)

"Remove duplicates" on its own does in fact have very valid uses...

Imagine, for example, that you wanted to listen to all tracks under 'musique', but didn't want to play any duplicate tracks that might legitimately exist... Here, the "Remove" function would be perfect.

Quote
Heureux que vous soyez tous triés maintenant. Je souhaite seulement que mon français soit à moitié aussi bon que votre anglais :)
Je vais utiliser Google Translate pour vous, au cas où vous perdriez quelque chose. Vous pouvez nous dire à quel point la traduction est efficace :)

Citation

     donc j'étais un peu stupide?

Certainement pas. Vous commencez tout simplement à réaliser et à libérer une partie du pouvoir sous le capot de MC. Je suis ravi que vous ayez posté vos questions malgré le fait que vous vous sentiez stupide, car cela nous a donné l'occasion de vous aider et, ce faisant, vous rassurez que vous n'êtes pas stupide :)

"Supprimer les doublons" à lui seul a en fait des utilisations très valables ...

Imaginez, par exemple, que vous vouliez écouter toutes les pistes sous «musique», mais que vous ne vouliez pas reproduire de pistes dupliquées pouvant exister légitimement… Ici, la fonction «Supprimer» serait parfaite.

-marko
Pages: [1]   Go Up