INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Improved MC10 Dup Checker  (Read 3438 times)

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Improved MC10 Dup Checker
« on: December 23, 2003, 06:23:54 pm »

I was wondering if the dup checker could be enhanced

the same method for YADB finger printing could be used to check for dupes in a file listing.

this would really come in handy for some of use because tags are wrong some of the times. even more so if the media file came from the net in spme point in your life.

Edit:

I Just tried This, this sort of works: if you add this string to "Add Modifier", "Duplicates"

[artist],[name],[duration],[replaygain],[peaklevel],[bpm],[intensity]

at least it works better
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Re:Improved MC10 Dup Checker
« Reply #1 on: December 24, 2003, 12:46:50 pm »

I agree,  with the amont of incorrectly tagged files .. it would really be nice if it found dupes based on other properties than file/name, etc ... a file sample would be ideal :)

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #2 on: December 24, 2003, 01:23:42 pm »

well i was thinking that if J river would allow the YADB tags to be saved to the data base (for finger printing) that this could also be used for a better dup checker

after that a dup compair could be done with the fingerprint data.

i have found that compairing the replay gain etc... may help but it is not a good way.
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #3 on: December 24, 2003, 05:13:13 pm »

I did find a program that is a TRM generator

aval here
http://www.musicbrainz.org/products/trmgen/download.html

they also have a tagger

http://www.musicbrainz.org/tagger/download.html

As A Test:

File 1:
G:\Pub\My_OTR_Files\P\Perry_Mason_1953_-_1954\1954-01-07 (2694) - Perry_Mason_1953_-_1954 - Dr Hall Concerned - OTR - Old Time Radio.mp3

TRM: cab0f469-b0e6-4a52-80e9-7e43e525933a

File 2:

G:\Pub\My_OTR_Files\P\Perry_Mason_1953_-_1954\1954-01-08 (2695) - Perry_Mason_1953_-_1954 - Dr Hall Concerned - OTR - Old Time Radio.mp3

TRM: cab0f469-b0e6-4a52-80e9-7e43e525933a

this seems to be a better way of finding dups than the current way of just compairing tags
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

ThatAdamGuy

  • Regular Member
  • World Citizen
  • ***
  • Posts: 163
  • I'm a llama's iguana!
Re:Improved MC10 Dup Checker
« Reply #4 on: December 25, 2003, 12:05:48 am »

MusicBrainz seems VERY cool, but I was quite upset to learn that it doesn't support WMA files! :(

Does anyone know of a sound-based tagger that *DOES* support WMA files?
Logged

zevele10

  • Guest
Re:Improved MC10 Dup Checker
« Reply #5 on: December 25, 2003, 12:01:53 pm »

There is something i do not understand:

Many of your posts tell about the many things ,many programs you cannot use because of the wma format.

So ,why do you stick to this format?

Not a judgment ,just curious.

Listening to: 'Child In Time' from 'In Rock - Anniversary Edition' by 'Deep Purple' on Media Center 10
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #6 on: December 25, 2003, 01:34:49 pm »

Here is some info i found that is saved to the ID3v2 tags

would be nice to be able to tap into this info

Quote
 TXXX (MusicBrainz TRM Id): 7fbca0a4-5f9f-4d33-a291-ce8ae3b0dd5a
  TXXX (MusicBrainz Artist Id): bd66f7df-645c-4cc3-9f0a-b1011338a671
  TXXX (MusicBrainz Album Id): 4f32ed63-ba64-4b43-a1dd-a0549f8777e1
  TXXX (MusicBrainz Album Arti..):

If the MC10 SDK supported Generate TRM this could be usefull
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

ThatAdamGuy

  • Regular Member
  • World Citizen
  • ***
  • Posts: 163
  • I'm a llama's iguana!
Why I chose WMA
« Reply #7 on: December 25, 2003, 05:07:24 pm »

There is something i do not understand:

Many of your posts tell about the many things ,many programs you cannot use because of the wma format.

So ,why do you stick to this format?

Not a judgment ,just curious.

Listening to: 'Child In Time' from 'In Rock - Anniversary Edition' by 'Deep Purple' on Media Center 10
Hi Zevele,

That's a thoughtful and interesting question, and I must assure you that I did indeed put a lot of consideration into my music choice.

Here's why I decided to settle upon WMA, after waiting nearly 2 years to rip my collection of 350+ CDs:

- WMA sounds FAR better than MP3 at equivalent bit rates.  I've not relied upon MS' word on this, but rather conducted my own listening tests.  After much testing, I concluded that for moderate file sizes and bandwidth and excellent audio quality, WMA VBR Normal/High offered the best balance.

- Other 'possible' formats that provide an excellent balance of listenability and file compactness, such as Ogg and MP3Pro, are not supported by anywhere near the number of Windows music players and portable players.  I'm sure I could have found my own setup (using specific software and hardware) that would have worked for ME, but I'd have been unable to easily share music files with my friends, most of whom are not geeks, and can play primarily MP3 and WMA files.

- Services that I already use and appreciate -- such as MusicMatch and Napster -- already offer downloads exclusively in WMA.   They're not gonna offer Ogg anytime soon :(

There ya go ;)
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #8 on: December 25, 2003, 05:37:25 pm »

I agree Windows Media And MP3, I prefer MP3 Due To It Is More Of A Standard Than The Others To Include Windows Media.

Wth MP3 VBR High Or 320 CBR MP3 The Quality Is Very Good And With My Ears I Could Not Tell The Dif Between That And A CD (Maybe Others Can).

It seems the TRM finger printing is done by http://www.relatable.com/tech/engine.html

I sent them an E-mail requesting information, since i do not have any money i am sure i will not hear from them on using a free SDK, But you never know.

Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Improved MC10 Dup Checker
« Reply #9 on: December 26, 2003, 03:00:03 pm »

Quote
s A Test:

File 1:
G:\Pub\My_OTR_Files\P\Perry_Mason_1953_-_1954\1954-01-07 (2694) - Perry_Mason_1953_-_1954 - Dr Hall Concerned - OTR - Old Time Radio.mp3

TRM: cab0f469-b0e6-4a52-80e9-7e43e525933a

File 2:

G:\Pub\My_OTR_Files\P\Perry_Mason_1953_-_1954\1954-01-08 (2695) - Perry_Mason_1953_-_1954 - Dr Hall Concerned - OTR - Old Time Radio.mp3

TRM: cab0f469-b0e6-4a52-80e9-7e43e525933a

How bout pushing this test a bit more.

try trimming 30secs/1 min of the beginning & end of the second track.

Run yout test again...how similar are the scores, cant he software tell they are the same track or not ?
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #10 on: December 26, 2003, 03:04:37 pm »

good idea, i will try it tomorrow

i do know if it is a second or so it still works because i have been running thru some of my song files and it Id's them if it is missing a second or so from the orginal
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Improved MC10 Dup Checker
« Reply #11 on: January 03, 2004, 01:00:18 pm »

Well !! did it work or not ? I'm guessing, prolly not.

was reading up on other stuff and came upon this article

interesting quotes are

Quote
variations in the MP3 encoding process will usually cause two different "rips" of a single tune from the same CD on a single computer to have two different MD5 signature.

and

Quote
Indeed, Ward of NetPD admits that the investigative service has identified nearly 90,000 different MD5 signatures on Napster for just 34 Dr. Dre tunes.

i'm not sure how musicbrainz works, but my guess is it depends on a similar hashing mechanism. So if tracks that are otherwise similar were encoded differently, they would be not be detected as dupes at all.

That article is nearly 3 yrs old. I wonder what the musicbrainz ppl have done to get around this problem.
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #12 on: January 03, 2004, 01:14:03 pm »

no it did not find the TRM if it was cut too much, a few seconds was ok but thats about it.

and encoder does matter as you said.

there still must be a better way.
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

jleerigby

  • Guest
Re:Improved MC10 Dup Checker
« Reply #13 on: January 04, 2004, 05:20:53 am »

I did a massive job on my media library recently to identify and tag duplicates.  First I used artist,name using MC's built in dups feature.  Then when these were done I found loads more by exporting my library to excel.  I did this by using a macro to remove all characters other than a-z and 0-9 from the artist and file name.  

Then I used a formula that concatenated half the artist name with half the filename e.g. =LEFT(A2,LEN(A2)/2)&LEFT(B2,LEN(B2)/2).  So:

Fleetwood Mac - You Make Lovin' Fun  becomes FleetwYouMake and
Fleetwood Mac - You Make Loving Fun also becomes FleetwYouMake.

Then in another column I do a count of how many times the new name (e.g FleetwYouMake) appears and filter the results to show anything with a count greater than 1.

It would be great if someone could build a plug in that could do something similar.  The plugin would add a tag to a custom field e.g. possible dup.  Better still would be if MC could introduce this sort of logic as an 'Advanced Duplicates' feature.
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #14 on: January 04, 2004, 09:04:09 am »

Quote
I did this by using a macro to remove all characters other than a-z and 0-9 from the artist and file name.

thats how some of my plugins work.

Quote
Better still would be if MC could introduce this sort of logic as an 'Advanced Duplicates' feature.
Yep
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

jleerigby

  • Guest
Re:Improved MC10 Dup Checker
« Reply #15 on: January 04, 2004, 05:47:45 pm »

..but you're so good at these plug ins King  ;)
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20062
Re:Improved MC10 Dup Checker
« Reply #16 on: January 04, 2004, 05:52:48 pm »

Well that’s why I and others would like more options in the SDK

Or some way to fingerprint the first 30 seconds of a file maybe it could be done.

But what I have done is just a matter of data manipulation nothing this complex for sure.

Maybe a combo of fields compair and fingerprint compair

[artist],[name],[duration],[fingerprint]
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA
Pages: [1]   Go Up