INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Search for and eliminate duplicates?  (Read 4855 times)

Jakester

  • Regular Member
  • World Citizen
  • ***
  • Posts: 175
Search for and eliminate duplicates?
« on: July 26, 2004, 02:46:26 pm »

Can MC search for duplicates?

It would be nice to be able to tell MC to search for duplicates based on defined criteria - say identical artist and song title (with other options such as ignore case) - and have it offer up the duplicates one set at a time.  MC would then allow the user to highlight one of the duplicates and, via a single button press, eliminate the rest of the files and update the library to point all of the other entries back to the one file that was kept.

MC must allow the user to select which one to keep so that if a certain file is known to be of higher quality the user can select it as the one to keep.

MC has a lot of features - however a search of its Help did not find this type of feature - is it available?  This could save a lot of space!

Thanks all.
Logged

bebop

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 610
Re:Search for and eliminate duplicates?
« Reply #1 on: July 26, 2004, 03:25:56 pm »

Have you tried the smartlist "possible Duplicates" (right click playlists. add smartlist,smartlist rule)?
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20063
Re:Search for and eliminate duplicates?
« Reply #2 on: July 26, 2004, 03:28:58 pm »

>>Can MC search for duplicates?

Yes

~nodups

~dups

Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

Jakester

  • Regular Member
  • World Citizen
  • ***
  • Posts: 175
Re:Search for and eliminate duplicates?
« Reply #3 on: July 26, 2004, 03:49:39 pm »

I am relatively new to MC - have ripped ~200 of my 800 CDs so far and am currently configuring another PC to use as HTPC which will pull info from my office PC upstairs.  I have not played with the smartlist feature yet, although I've looked at the help which shows all of the search criteria.  

KingSparta, bebop - Can you spoon-feed me a bit?  If I create a smartlist using ~dups how do I then use that list to eliminate duplicates and have the library point back to a single remaining file (for each set of duplicates) that I pick?
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #4 on: July 26, 2004, 04:20:27 pm »

You can use for example this: ~dups:[Artist],[Name] in Search. You will get a view to duplicates, which have same Artist and Name. Then you can just the delete the redundant files.

Quote
... and have the library point back to a single remaining file (for each set of duplicates) that I pick?

A file can exist only in one album. You have to retain some duplicates, e.g. when you have an original album and a best of collection by the same artist.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

LonWar

  • Citizen of the Universe
  • *****
  • Posts: 2874
Re:Search for and eliminate duplicates?
« Reply #5 on: July 26, 2004, 04:23:12 pm »

Correct me if I am wrong...

You have 3 albums...

Album1
Album2
Album3

YOu have 1 sone that is the same on both : Song3

You want to remove the 2 copies of this sond, and still be able to access that song from either of the 3 albums...

Correct? If so, this has been a requested feature in MC, but not yet possible.


If misunderstood your question, I appologize.
Logged
-

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Re:Search for and eliminate duplicates?
« Reply #6 on: July 26, 2004, 04:37:47 pm »

Use the MD5 Hash created by King's Plugin (www.spartasoft.com)

It will work for songs across multiple albums .. and is the best (using MP3 Hash) at finding exact dupes!

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #7 on: July 26, 2004, 05:23:42 pm »

As far as I see MD5 Hash is good for finding the duplicates, but that's all.

It is already possible to use APL files with APE files. Physically they are only shortcuts to an APE file. Maybe something similar could do it? MPL? (For single tracks and all file formats.)

Another thing is that quite often different albums have also different versions of the same songs (length, overall sound quality, mixing, etc).
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20063
Re:Search for and eliminate duplicates?
« Reply #8 on: July 26, 2004, 05:35:52 pm »

the reason I made md5 hash is really to find dups and some that may have a shorter run time thats why the program will not only  test the whole file but will also test out only the first so many bytes. and if it matches and the run time is shorter for one then delete that one (after you make sure it is a dup) and keep the longer one.

this happens allot on files that have been downloaded from the internet via p2p
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

jleerigby

  • Guest
Re:Search for and eliminate duplicates?
« Reply #9 on: July 26, 2004, 05:42:46 pm »

King.  Your dupe finder is the number one plug in for MC bar none.  (And I'm bragging too 'cos it was my idea).  It's just that I wouldn't have the faintest idea how to implement something like this.  

I just used it to rectify over 500 dups.  I could never manage without this plug in.  Thanks Bud!
Logged

Jakester

  • Regular Member
  • World Citizen
  • ***
  • Posts: 175
Re:Search for and eliminate duplicates?
« Reply #10 on: July 26, 2004, 05:57:26 pm »

I am using Monkey's Audio codec in Secure mode with no normalization and will use the $23 Chaintech card to get bit-perfect playback using instructions here http://www.hometheaterforum.com/htforum/showthread.php?s=&threadid=190009  O.K. - I guess this has nothing to do with this topic - just popped up because this thread now mentions MP3, APE etc. and the $23 Chaintech thing is great for those looking for bit-perfect playback so I just wanted to spread the word ::) ...

Anyway, the posts by Alex B and imjustagamer are right on track.  Thanks for the info, although that sure surprises me.  It is definitely true that quality varies, which is why I want control over which version of the song to keep.  I thought, once I found the duplicates (which you all have now shown me how to do), I could manually change the filename field for the songs I don't want to keep to point to the one I do want to keep, and manually delete the rest.  Not pretty, but wouldn't that work?!
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #11 on: July 27, 2004, 06:24:12 am »

... I could manually change the filename field for the songs I don't want to keep to point to the one I do want to keep, and manually delete the rest.  Not pretty, but wouldn't that work?!

No. One disk file can exist only in one MC Album. You can make your own tags for the other album names, but MC don't handle them as separate albums. The only way to keep them together is to do a playlist for them.

Currently it is possible to have one huge APE file for the whole Album (one CD disk). Monkeys Audio can do APL files for linking a certain position of that APE file to a logical track. An APL file includes also tag information for an individual track. APL files can be imported to MC and used as regular audio files. It is possible to make a copy of an APL file and change e.g. the album tag and keep both in the library.

What we need now is a similar system for individual tracks. A sort of Media Center link file (could be named as .mcl). That would include tag information and a link to the main file. It should support all audio formats. The mechanism to make them could be in Convert Format (of course "Send original files to recycle bin" should be disabled).

There is already support for the APL files. Perhaps just a slight modification is needed.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #12 on: July 27, 2004, 07:31:05 am »

If that is difficult to implement for all audio formats, it would be useful just for single track APEs. I don't mind having MP3 duplicates, but it is different with APE.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Jakester

  • Regular Member
  • World Citizen
  • ***
  • Posts: 175
Re:Search for and eliminate duplicates?
« Reply #13 on: July 27, 2004, 11:13:09 am »

Thanks for the explanation Alex B.

I second the vote for a link file.  And it would be nice if it was automated in the manner I described in the original post.

Matt? ...  If not V10 then V11 at some point? ...
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #14 on: July 27, 2004, 11:34:49 am »

It is already possible.

At first I wrote this cue file for a single track:

PERFORMER "Sinead O'Connor"
TITLE "Am I Not Your Girl?"
FILE "01 - Why Don't You Do Right.ape" WAVE
  TRACK 01 AUDIO
    TITLE "Why Don't You Do Right?"
    PERFORMER "Sinead O'Connor"
    INDEX 01 00:00:00


I saved it as "01 - Why Don't You Do Right.cue" (text only)

Then I converted it to APL with MakeAPL tool (included with Monkey's Audio).

Then I made a duplicate of the APL ja imported both APLs and the APE to MC.

This is the final result after tagging:



Though, some automation would be appreciated.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Jakester

  • Regular Member
  • World Citizen
  • ***
  • Posts: 175
Re:Search for and eliminate duplicates?
« Reply #15 on: July 27, 2004, 05:51:44 pm »

Hey, great!  Now we just need automation.  That would be too much work to do for a bunch of duplicates.

Thanks for the experimentation Alex B.
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Search for and eliminate duplicates?
« Reply #16 on: August 02, 2004, 12:52:24 pm »

Quote
No. One disk file can exist only in one MC Album. You can make your own tags for the other album names, but MC don't handle them as separate albums. The only way to keep them together is to do a playlist for them.

I'm curious about this Alex..

lets say i rip one album as 1 APE file + cue. call it album1.ape

i then use MakeAPL and get 5 (say)  tracks, or APL files.

Which would i import into MC ?

Just the APLs ?

Reason i ask is that the album tag for the APE file and the APLs will be the same.

What is the preferred way to go about this ?
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #17 on: August 02, 2004, 01:31:27 pm »

Import just the APL files. An APL file holds tags and a link to a certain position. All files must be in the same folder. You can't rename APE files later. APL files will loose links.

You can also import APE files, but that is not needed. APE files can have different tags. APL files are not using them.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re:Search for and eliminate duplicates?
« Reply #18 on: August 10, 2004, 08:47:11 pm »

I just noticed that it is not possible to use APL files with Media Server. Whenever the song changes the client PC will obviously download the whole APE file. I think it can work only if you have a gigabit LAN.

I tried it in my office and downloading a new APL song took minutes, because the connection is a 10 mbit homePNA network (in my case the effective speed is only about 400 - 500 kilobytes/s).
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755
Pages: [1]   Go Up