Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Mirko]

Hi,

I have developed a simple plugin to find duplicates (and delete them).

You may want to try it (Download).

Not functional:
- deleting

If you have ideas or request, please post them.

Thanks,
Mirko

P.S. You have to copy the ocx into the plugins folder and start "registry.reg" to register it.

P.P.S. The plugin finds the duplicates but does not correctly play or show the filename, if you use one of the buttons... I don't have time to repair it yet, I do it tomorrow after work.

[zevele1]

Mirko.
Will wait until you fix it to try.

Do not know the features you want to put.
But,here some who would beat the 2 other softwares existing:
-Play
-send to
this can have 2: send to everywhere in your system
and -the must- send to INSIDE Media Jukebox.I mean kind of send to playlist.

Do not know if it is hard to built.
But if you do it,many here will praise you.

And i will send you some money to have beer

listening to 'A lot of people would like to see Armand Schaubroeck....DEAD
by Armand Schaubroeck X3pls on my turnable

[KingSparta]

Mirko

Got Picture?

Does It Delete On File Names?

Does It Delete On tag Names?

Does It Check The Kbps, Length And Compare?

[drosoph]

My $0.02 ....
There should be checkboxes for:
MATCH IF: kbps, type, length (duration), tags

Then setup delete options, like DELETE SHORTER DURATION ... or DELETE LOWER KBPS !!!

Something like that ..

[Mirko]

Zevele:
> Do not know the features you want to put.
> But,here some who would beat the 2 other softwares existing:
> -Play

Sure. This is functional already.

> -send to

I think I can do it.

> this can have 2: send to everywhere in your system

you mean copy? I will do it, if you like it.

KingSparte:
> Got Picture?

Not yet. It's quite save to install ;-)

> Does It Delete On File Names?

No. Would you like this? Then you'll get it.

> Does It Delete On tag Names?

Yes. In the future. Now there now actual deleting (but you can see a messagebox which file would be deleted).

> Does It Check The Kbps, Length And Compare?

It checks duration. No Kbps yet, because I encode to VBR all files (an the Kbps is not relevant here).
I compare on:
1 Title |PLS| Artist - Combination (including similarities)
2 Duration
3 Size

Point 2 and 3 are not exactly "duplicate"-matching, because many songs duration is about 3:23 (I test with about 10.000 songs, it takes about 7mins to analyze...). Size is also not very matching. Do you have an idea how to implement those in a more "telling" way?

I will do an automatic "delete list" which optionized categories (like drosoph suggested). I think this would be the easiest way to handle this.

Thanks.

[zevele1]

Mirko

I'am really dum at computer.But if you do really a MJ pluging.,in this case with a send to playlist fonction ,you do not need to work on 'play' inside your pluging.We can play it from the playlist.

Send or copy to somewhere on your computer will not work.The second indentical file will delete the first one[or maybe i'am wrong,i just do not know].My idea was to be able to ckeck with a program if the 'duplicates' are ok or with problems.And to keep the good one.

To just delete the shorter one or the one in lower quality look to me a little 'far out'.
If it toke you 2 years to find a song,better to keep the one at 92 kps.or the one with some music less if the 'full-192 kps' has artefacts,noises and so on.


Another question
It is possible to check by folders,or just your hard dive are checked in an automatic scan?
It is possible to ckeck from inside Media Jukebox?
You search music for artist X,and you send the songs to pluging 'Duplicate Finder'?

[drosoph]

Interface Plug-in 'DubFinder' must implement IUnknown interface.
Looks like some of your calls are diff in the new interface.

MJ 9 (9.0.0.50)

[Doof]

That's the exact same message I got. It's also the same message I got when I tried to install King Sparta's chart finder plugin under MJ8. But this is only on my work PC (and I can't install any plugins on this thing - I think it's fairly horked).

Whatever delete options you choose to implement, can you have an option to not do any deleting automatically? This probably goes without saying, but I just wanted to make sure. There are some duplicates in my collection that I actually want.

[KingSparta]

Doof

I provided a link to the VB runtime files this might fix your problem.

[Mirko]

First of all: There will be absolutly _now_ automatic deletion of files. Only "automatic" movement to the "deletion view", where all the files suppossed to be deleted are moved. So don't worry ;-)

> It is possible to check by folders,or just your hard dive are
> checked in an automatic scan?

At this moment I only scan MJ-Database. No plans to expand this.

> It is possible to ckeck from inside Media Jukebox?

Yes. And only this.

> You search music for artist X,and you send the songs to
> pluging 'Duplicate Finder'?

It may be possible, if you add the results to playing now or to a playlist. Then I have to add a switch to scan a playlist (or playing now, which is a playlist I think) only. Should be no prob.

Thanks,
Mirko

P.S. Yes. You need the VB6-Runtime including Windows Common Controls 6 (mscomctllib). I will provide a setup if this thing is "ready" (what ever that may mean for software...).
P.P.S. I actually did implement some of the options above. You may download again, if you like (right click on a deletion button to see some more options).

[zevele1]

Thank you.
To be able to 'scan' from a search result would be great-even if there is a need to send it first 'somewhere' in MJ.

The fonction send to... playlist or other INSIDE Media Jukebox after the scan a great plus as well.
As i said,like it you can play it,before deleting.

The top would be a send to ...all duplicates found

Are you ready to built a suite for MJ?
dulicates-check quality- and some other things?

I do not mean a free one

[Mirko]

I don't know what "built a suite for MJ" means.

But about the quality thing... I thought about it last night, but I'm not sure how to do this. Quality is mainly in the listeners ears, isn't it? I really would like such an option, but I believe that I'm not able to do it (logically and skill-side).

I have some ideas to add today, maybe in about 10 hours.

@drsophos:

I did some testing about duration-matching yesterday. It seems like many files in my testing environment (about 10.000 songs) have the duration of 3:32 or similar. So the matching would be... errr... not exactly usefull I think. But I did test some testing about the "mark for deletion with less duration"-option. This is more or less usefull I think.

[zevele]

It is no quality that i mean.

It is checking for defaults in the mp3.Clicks,pops,silence and so on.
There is few programs doing it.
I must go out now and do not have the time to give you links.
Will put them in few hours

[Mirko]

@zevele: If these utilities are commandline (or source-code), then I might be able to integrate them. If the have a COM-automation-object, too.

I will search for them myself, too.

There will be no charge at all. I just do it for myself and will share the results with you.

I forgot all the time to warn you: This is alpha software, so there might be bugs that damage you files. I don't believe there are, but I think I should warn anybody anywhere (to make sure no one sends his lawyer to my home *g*).

[Mirko]

The plugin is now functional. It doesn't delete at all and does only export to "Duplicates"-playlist. Try right clicking on the buttons to see more options.

Screenshot:


Thanks for any reply.

Mirko

[zevele1]

Here we go

First 2 duplicate checkers you can have a look to
-mp3check-very primitive
-mp3Match much more friendly

Another nice program:
-EncSpot--www.querillasoft.com/encspot

This program tell you the quality of the encoding process by finding wich encoder used.And tell you if a song is complete or no.

Concerning programs checking mp3 for problems,i know only 3.

-The Ashamppo one,a VERY BIG S--T.I use many Ashampoo programs and i am very happy with it.But this one is a total disaster.

-Mp3Utiliy--www.geocites/mp3utility

-Mp3Test--www.maf-soft.de

I understand better the results with Mp3Test.But i have a problem with they site[in german,so for you not a problem] and an error 'zu wenig arbeit sspeiche'.Of cause i as well do not understand.

Mp3Utility gives less easy to understand results to 'idiot average computer user'[to make short,MYSELF].

As a rule none of this 2 programs work 100% fine if you put thousands MP3 to check at once and can be very very sloowwww the more they work.

To tell you what i dream about,let me take an exemple.

I have all the Sonic Youth records.But all in lp vinyl,not in cd,include the last one.

I would like to have it on my computer.
I use P2P programs.
After a while,i have number x Sonic Youth songs in Imesh folder,and in Audiogalaxy folder,and the same in SoulSeek,Kazza,,Filetopia,Quarex and more folders.
I cannot put all this songs in the same folder on my hard drive.The new one will replace the old one and so on,and i may get at the end a 96 kps -95% defect in my folder.

With Media Jukebox pluging,i would be able to do like it:
seach music=Sonic Youth
send to play list 'checking duplicates'
And i check using your program.

I send the duplicates[i mean ALL the songs the same,not only songs |PLS|1,|PLS|2.. to playlist 'check for problems'

with one of your other pluging,i ckeck for problems.
According to the resuls and the number of times i have the song,i delete the bad ones,listen to some to have an idea.

AND NOW I AM HAPPY!!!!!!!!!

The top would be to have another pluging like EncSpot to check encoder quality and incompletes songs.
You delete incompletes,the ones encoded with a bad encoder,and you keep the good ones.

After this,you can put songs in artist\album folder,check for songs you still need to get.

And you start again with Giant Sand ,Dream Syndicate,Green on Reed,Long Ryders ...songs


-

[nila]

Unknown interface error - where do I get the VB runtime files from mentioned in this post?

[Mirko]

I'm trying mp3check and encspotconsole at this moment. I cannot include the utilities with GUI, because I don't know how to automatically read outputs from other GUI-apps. Console apps is no prob theoretically.

mp3check delivers a format I have to figure out. It's quite weird, if you try to find a reasonable structure in there (I guess it could be the ":"). encspotconsole can easily identify the sh... from eMusic encoded with xing (I _knew_ it... they simple sound to bad).

Anyway. I'm working on this. But I will try to include it in the duplicate-plugin to help the user decide, which file to remove.

Did you try the plugin?

Thanks.
Mirko

[Mirko]

KingSparta said, this URL May help:
http://www.microsoft.com/downloads/release.asp?releaseid=28337&area=top&ordinal=5

Maybe I should to an install version right now...

Here you'll find the downloadable setup. Right now it does not copy the ocx to the plugin-directory of MJ. Please to this yourself ;-)

Download

[zevele1]

Mirko

Very sorry,but when i said i am a total idiot concerning computer.....

What i do with the download?
I unzipped it-yes this i know...-i told the program to install in c\program files\jriver\..pluging and- as you had understand allready when reading me-..nothing happens.The plugin is not listed in MJ when you go 'plugin manager'

[Mirko]

Please look for a file "MJDubFinder.OCX" and copy this to the plugin-directory of MJ (maybe it is already there, if you installed into this directory). Then you'll have to use the register.reg file from the ZIP I originally posted (this is very small, about 70k). Doubleclick on that and you'll see the plugin (it's the way I "install" here).

Sorry for not providing a real setup. I will be coming in a few days, but for now I will concentrate on developing it.

If you have more question, please ask.

[Doof]

Ok, I now have this installed and it runs without giving me the IUnknown Interface error message. The only problem now is that it gets about halfway through scanning my library and then reports a Runtime Error 11, Divide by zero, error.

Any ideas?

[nila]

I still get the interface problem even after running the install program (which I have no idea what it's saying due to it being in german) and the vbruntime files.
Any ideas??

[zevele1]

Going away until monday.Will test it when back

Thank you for the work

[Mirko]

@doof:
How many files do you scan? It's not very likely that the error is descriptive here, because there is not division in that function...
Please give more info.

@Nila:
I have no idea. Sorry for only providing a german version, but I don't have the money for buying an international version of some setup-toolkit.
Anyway. Did you correctly install? This should be obvious to notice. If this is not, send the message here and I will try translating it.
Maybe you should contact me by mail and send some screenshots of the installation-procedure (mwalter@2via.de).

@Zevele:
Nice weekend!

Thanks,
Mirko

[Doof]

I have about 4500 files in my library. I told Dupfinder to only scan Audio files, and told it to Go. It started scanning, and down below in the status area it shows the files going by and beneath that is a counter of sorts that shows:

Processing file ###/3,592 - 1(2)(3),000/3,592

The ### represents the current file it's on, and the 1(2)(3),000 represents the fact that this secondary "counter" is just going from 1,000 to 2,000 to 3,000 and back to 1,000 where it starts all over again. That's all it does. At some random point in the process (once it was after 600 files, another time after 1600, and another time after 2100) it just bombs out with that divide by zero message.

[Doof]

Okay, I got it working. I think there's something wrong with my library somewhere. I created a new library, imported all of my music, and now it runs fine.

And now that I can see it in action, I have a couple of suggestions.

1. In the Difference Information box, Title is spelled "Titel". There was some mention of German (since the setup was in German) so I'm not sure how you spell Title in German, or if you're aiming for an English version, or if it's just a plain typo, but there you go.

2. How about adding in columns for Album and Filename? That way I can decide if the duplicate is a file belonging to a complete album and a loose track, or if they belong to two separate albums.

Other than those two thing, I love this! It's much easier than manually scanning the library.

A couple of questions though. There was talk of a Duplicates playlist. Where is that? I don't see it listed under Playlists. Also, (this may sound stupid) what exactly does the Delete button do? I only ask because there is also and Undelete button, so I wasn't sure if Delete really meant delete the file from the harddrive, or what... and I'm not quite brave enough to just try it.

[Doof]

Sorry, a couple of more things.

1. What does the Bitrategraph do?
2. I was thinking of the layout and I was wondering if it might not be cleaner if you could use a tree structure like MJ's Tree View.

Like... list the Artist and Name that's common to all of the matches. (For instance Metallica - Enter Sandman) and then have a plus sign next to it that you click and it expands to show you the possible duplicates beneath that. So in my example you would see something like this: (the spacing is messed up because of Interact - but the entries with |- in front of them should be indented)

|PLS| Metallica - Enter Sandman
|- Metallica - Enter Sandman
|- Metallica - Enter Sandman (Live)
|- Metallica - Enter The Sandman
|- Metallica - Sandman
|PLS| Metallica - Nothing Else Matters
|PLS| Metallica - One

and have it all alphabetized. And then for the branches only, show the information of Album, Filename, Bitrate, Size, and Length. That way you wouldn't have to double up on the number of columns.

[Mirko]

Okay, I got it working. I think there's something wrong with my library somewhere. I created a new library, imported all of my music, and now it runs fine.

Interesting. Nice to hear!

And now that I can see it in action, I have a couple of suggestions.

1. In the Difference Information box, Title is spelled "Titel".

I'm german ;-) It's corrected.

2. How about adding in columns for Album and Filename? That way I can decide if the duplicate is a file belonging to a complete album and a loose track, or if they belong to two separate albums.

If I add columns for album, I need to add 2 of them (Album 1 and Album 2). For the filename the same. Do you have a 25" screen? Ok, I will add them.

Other than those two thing, I love this! It's much easier than manually scanning the library.

Thanks.

A couple of questions though. There was talk of a Duplicates playlist. Where is that?

It will be created if you actually "delete" the songs on the deletion list using the delete-button. If it exists, it will be emptied first. The name is "Duplicates".

what exactly does the Delete button do? I only ask because there is also and Undelete button, so I wasn't sure if Delete really meant

The delete 1 and delete 2 buttons move the corresponding song down to the deletion list. Using the delete! button it copies the files to the duplicates-playlist (and does _not_ delete them!).
Undelete: If you select a song on the deletion list, it will be removed from there.

Thank you very much for the help! I might work on the GUI now :-/

Edit:
The bitrategraph shows a graph of the bitrates from VBR-files if you play them using the plugin-buttons. I know that this is not very usefull yet, but I will add support for mp3check, which displays the VBR-changings. This could be usefull in deciding which one is the best one. I plan to add support for encspot also. Tomorrow is a working free day for me, so I expect something happening. Today is not excactly free ;-)
About the tree-view... I don't know yet. Maybe I will try something.

[nila]

I've tried this with v8 and with 9 both with no luck.

Here are the messages I get when trying to use your setup program.




I've also installed the vb files found at the link given in this thread.

I'm still getting the: must implement IUnknown interface error.

I'm running WinXP by the way.

[Mirko]

@Nila.

You canceled the setup. If you see a dialog box with one bigger button left/top and two (or more) smaller buttons on the bottom, please you the big one. This is quite unusual and I guess you hit the rightmost small one.

Then you have to locate the MJDubFinder.OCX (download it from the URL given above) and copy it to the "plugins"-directory underneath the mediajukebox-folder. Then double click on the reg-file (in the MJDubFinder.zip) to register it. This should do the trick.

If this doesn't help you, please ask again.

@all
I just updated the plugin. See above link for new version. I will integrate an automatic version checking; maybe tomorrow. This is the last one for today (I think *g*).

[nila]

That worked.
Unfortunately now I'm getting: Run-Time error 93 - Invalid String Pattern after pressing: Go

[Mirko]

@Nila.

May you download it again? I updated some of the error-routines, so that they provide more information to me. I decided to first do some errorchecking, so that if you encounter a problem you can give me more information than only a number (I personally don't find think they are always telling). It's not finished yet, so it might dump again...

Thanks for the help.

[Mirko]

New Website Look here.

[nila]

Same error again

[bennyd]

Hi Nila,

I had the same 93 error and it appeared to be a musicfile
that had a square bracket ( [ ) in the tags.
Removing the bracket in the tag AND filename solved the problem.

Maybe this helps

[Mirko]

That helped me quite much (I don't have a single file using "[", "]", "#", "*" or "?"). these characters all would give the same error.

thanks a lot. a new version is uploaded.

[nila]

still not working for me

I guess in order for me to work out what's causing it to crash I'd need you to add some debug code that basically wrote down the name of every song into a file just before it started to process it.

It could either add all the songs to a big list one by one just before it starts processing them or it could just replace the name each time so that it just had the song it was about to do.
Either way I could find out that way what song is causing it to crash.

[Mirko]

I have done what you suggested. A debug version is available here. Send the file "c:\debug.lst" to mwalter@2via.de, if you crashed again.

Man. I really don't understand why this is so. I scan about 10.000 files and no error at all. Why do all system have some other issues to handle?

Thank you very much for your patience. I'm still working on providing support for mp3check (which finds quite a lot of interesting things and can repair some of them, too). But as I have to handle with pipes and stuff, I don't think that this will happen very soon. I tried some things today, but they were not exactly successfull.

[nila]

Hmm, it's not creating the debug file.
I gotta run now also, going home for the night.

Sorry - good luck.

Cya

[SeanC]

I am getting an Error 4 on files that contain [ or ]. Unfortunately, there's no way to break out once this error occurs, and so I have to shut MJ down with Task Manager.

[Mirko]

SeanC:

This should not be happening in the newest builds. Did you download it again before you tried? I explizit filter these filenames and change some special characters _internal_ to space.

This all is because of the way I look for similarities. I use a function called "like" and "[]*?" (and more) are special characters for this function. So I have to replace them just for the comparison.

Maybe I have some time this afternoon (its about 06 in the morning here) to do a detailed error-routine. So those errors might occur again, but I might have a little help in finding them. I haven't find a way of debugging them, so please be patient.

Nila:
Are you sure that you downloaded it again? It creates the debug-file in my test-environment...

I will post if I have something to test with. It will take a little time, because I mostly can't reproduce the errors and that's a little difficult to handle ;-)

[nila]

I definitely downloaded the latest version. It's still crashing and still no logs.

Did you get my copy of my database i sent you so that you could use it for testing?

I'm hoping you'll be able to fix it so I can use this because I'm sure I have a lot of duplicates.

[Mirko]

@Nila:

I got your database, but I can't use it (maybe I'm too stupid, who knows). I'm currently working on a better debugging-support. This will take some time, sorry for that, but I think in the long run it will help improve the quality. I also have to make the plugin MJ9 compatible (this shouldn't be a problem at all).

I will post here, if I have the version ready. I'm currently quite involved at work, so there very few spare time (and I have to share some of it with someone else than my computer *g*). I might have time today, if everything works out as it does in the past few hours.

Thanks for your interest. The plugin did find many duplicates in my database, by the way.

[Mirko]

I've been at work for over 13hrs today, so I'm quite tired. But I uploaded a version with debug. It definitly creates a debug-files in the "c:\"-directory. If anybody want to try it, please give it a try. It's not compatible with MJ9 now (it can't read the contents of playlists e.g.). And I will provide a better debug-version later this week. But not today.

Going to bed now...

Mirko

[David]

I've noticed other posts about this error.  Is there a fix for it?

I've installed the latest version from http://www.2via.de/mjdubfinder.htm and added the registry entries as well as the VB runtime files.

I receive the error: "must implement IUnknown interface"

I am able to run Chart Finder OK which is also a VB application.

Media Jukebox PLUS 8.0.370
Install Path: C:\Program Files\J River\Media Jukebox\

CPU: Intel Pentium 4 2.8GHz MMX
Memory: Total - 392 MB, Free - 204 MB
OS: Microsoft Windows XP  Workstation 5.1 Service Pack 1 (Build 2600)

Internet Explorer: 6.0.2800.1106
ComCtl32.dll: 6.00.2800.1106
Shlwapi.dll: 6.00.2800.1106
Shell32.dll: 6.00.2800.1106
wnaspi32.dll: Internal ASPI Layer

[KingSparta]

David

this maybe because when building the Program, Mirko has not included some of the OCX Or DLL files that are needed.

Mirko

Attempt To Build A Project, The files that the Microsoft Package and deployment wizard tells you you need you need to includ them in your ZIP, and also include them In the Plug-In's Actions Here is what I am using. it might not be perfict but it seems to have fixed the problems some if not all users were having running my plug-in.

File1=ChartFinder.ocx
File1Action1=COPY_PLUGINDIR
File1Action2=REGISTER
File2=Argentina Top Hits.csv
File2Action1=COPY_PLUGINDIR
File3=Rick Dees Top Hits.csv
File3Action1=COPY_PLUGINDIR
File4=U.S.A. Country Countdown.csv
File4Action1=COPY_PLUGINDIR
File5=Top 100 Billboard.csv
File5Action1=COPY_PLUGINDIR
File6=U.K. Top 40.csv
File6Action1=COPY_PLUGINDIR
File7=COMCAT.DLL
File7Action1=COPY_SYSDIR
File8=mscomctl.ocx
File8Action1=COPY_SYSDIR
File9=TABCTL32.OCX
File9Action1=COPY_SYSDIR
File10=Registry.reg
File10Action1=COPY_PLUGINDIR
File10Action2=RUN DONTWAIT

[Mirko]

@KingSparte:

I tried that. I noticed, that I got the interface-error if the tabctl3d.ocx was not registered (it was in fact unregistered by one uninstall of an application I tried -> I reinstalled using my setup-package and the error vanished). So I think you are right.

I tried making an MJP-Package. You may download it here.

Man is it difficult. I thought I might concentrate on developing :-)

[KingSparta]

Mirko

the check for updates button tells me there is an update but the version is older than the running version.

besides got source for this? I could use it to check to see if there is new database files on my web site.

Seen a few more problems dealing with play times of the second dup, and the summery box needs to be extended down a few lines it seems to be cutting off some info if you have many files but it seems to have installed fine.

I was thinking that maybe if there was a dup that maybe the list should have a [X] check box and check the worst file (the best file to delete).

I would like to see a Dup is only found when a match is made on the song name, the artist, and the play time is the same.

It may also be time to close this Thread and start a new one due to the length of it.