Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Mirko]

@KingSparte:

I did a new thread. But I'm not able to close the old one. Could you?

Mirko
the check for updates button tells me there is an update but the version is older than the running version.

Yeah I know. I have to update this manually, for now. So I did not do this. I have to do some automation.

besides got source for this? I could use it to check to see if there is new database files on my web site.

I have created a file on my webserver (mjdubfinder.ver) where I store the newest version number. I have to do it a better way.
But I only used the functions provided by MJ. I can send you the source, if you like.

Seen a few more problems dealing with play times of the second dup,

What do you mean play times?

and the summery box needs to be extended down a few lines it seems to be cutting off some info if you have many files but it seems to have installed fine.

Thanks for checking. I will make it bigger. Right now ;-)

I was thinking that maybe if there was a dup that maybe the list should have a [X] check box and check the worst file (the best file to delete).

Hm. I thought about that one. So I did a "dup-quote" (the percentage value on the left side). And if you sort by artist or title, you can see the different songs... But I think the checkbox-idea is good. You mean: Check the file to delete and don't use explizit delete-button up on the "button bar"?

I would like to see a Dup is only found when a match is made on the song name, the artist, and the play time is the same.

At this moment, the dupfinder checks for artist and name. The matchquotes evaluates artist, name, duration, filesize and kbits. So I think you are right for adding duration the the first checkings. I did that.

Thanks a lot for helping me out.

I tried it with MJ9. It's _slow_. I have uploaded the new version to this location.

Mirko

[David]

Mirko,
I am able to open Dup Finder using the new install that you posted.  However, as soon as I run it I get an "error -4" message.  Each time I click OK another error pops up.  The only way to exit is tro kill the application in Task Manager.

The debug log has only one entry:
1 E:\My Music\Various Artists - Nipper's Greatest Hits - The 20's\Maurice Chevalier - Louise.mp3

Thanks,
David

[Mirko]

Hi David,

is this an error box with my mailadress in it (mwalter@2via.de)? How many files do you have in the listview in the upper part of the plugin?

Sorry for that unfriendly errorhandling. As you might have read, I don't have too much sparetime to spend on this. And I even can't reproduce the errors. But I have an idea, what may go wrong here (maybe "'" is some sort of special character for the function I'm using; maybe I should even through out that compare-function alltogether...).

Thanks for reporting,
Mirko

P.S. I found out why the plugin seems to be so much slower than before. It's the debugging-file. If you compare a couple of thousand files, like I did for testing purposes, you end with a (many) multi-mb debug.lst-file. But I think this helps finding the error...

[zak326]

the same error just came up for me. And it sure is persistent it conmes up everytime I try to do anything/.

[Mirko]

@zak326: Is the error-msg with my emailadress or without? Could you please send the file "debug.lst" in "c:\" to mwalter@2via.de?

Thanks a lot.

Mirko

[zak326]

It not onlye came up with your address it came up for every song in the playlist is was analyzing. I finally ad to do a ctrl-alt-del to stop it.It was not a fun experience.

I don't know how to send a bug report other than to say it bugged me a lot.

[Mirko]

@Zak326: Thanks for the info. The main problem is: I cannot reproduce this and have no idea, where the error comes from (but I do now, because you provided the info, that my mailadress is there -> so I can find where I brought in the errorhandler and may find the actual bug).


Edit: I found the error. The new version is uploaded, if anyone wishes to try it.

[David]

Mirko,

The new version works great!

Thanks,
David

[David]

I noticed one tiny spelling error.  The plugin is listed as "DubFinder" in the MJ Plug-in and "DupFinder" within the program itself.

The program is chugging away right now without any errors so far.  Only 7,500 more entries to search.

Thanks,
David

[Mirko]

Thanks for the positive reply. I needed that :-)

There is a new version uploaded, which speeds up searching _very_ much (from about 2hrs for 10.000 files to 15mins I think).
So you maybe you would like to update.

Question: Should the plugin save the results for later usage? I think so. But I guess I have to do little check routine then, which finds deleted files during two sessions of the plugin. Would like to know how you think about it.

Hint: The play-buttons (err... the corresponding "stop") don't work with MJ9 yet. Don't know if it's my fault :-/

[KingSparta]

Mirko

This version seems to work better, and don't have the problems i did with the other.

I Did notice "Lable1" that is your Dup Counter is shown, you might want to set this to "0" on Program Init

Also I guess what the program does is add all the dups to the playing now and the user needs to Delete them his self. since you can't delete them from Playing now without using A Key Combo (i forget what it is) on the message that pops up you might want to let them know what the key combo is.

If you are going to allow a check of the play time i was thinking maybe another nice option would be to allow the user to set the seconds it can be close to.

so if Song1 was 3:30 seconds and song 2 was 3:30 that would be a match. and allow the user to set the seconds off it could be match so if he set it to 1 second the results would match if it was 3:29, 3:30 and 3:31.

the other thing to look at would be what is the KBPS if they matched the best out of the two might be the one with the higher KBPS and if the list had a check box. the program could automaticly select the lower KBPS file and place it in the "To Be Deleted" list.

Looking Good However...

[Mirko]

I Did notice "Lable1" that is your Dup Counter is shown, you might want to set this to "0" on Program Init

That's a debugging label and I forgot hiding it. It shows me the status of my duplicatication routine (I had many errors with too many matches and so I can cancel it prior to memory problems).
I will add a mechanism, so that only my username might see this.

Also I guess what the program does is add all the dups to the playing now and the user needs to Delete them his self. since you can't delete them from Playing now without using A Key Combo (i forget what it is) on the message that pops up you might want to let them know what the key combo is.

Actually, the program creates a playlist called "Duplicates". But you are not able to delete out of this, either. So I guess, some time in the future I will add a real deletion routine. But not in this early development stage.

If you are going to allow a check of the play time i was thinking maybe another nice option would be to allow the user to set the seconds it can be close to.

I had an internal quote of about 1.015 to match two durations (and filesizes). I changed it to 2sec by default and you might edit the value in the setup-tab.

the other thing to look at would be what is the KBPS if they matched the best out of the two might be the one with the higher KBPS and if the list had a check box. the program could automaticly select the lower KBPS file and place it in the "To Be Deleted" list.

I show both songs in one row. So what would a checkbox mean? Delete 1st or 2nd? I'm not sure how to do this.

Looking Good However...

Thanks for the help.

[nila]

Wahey!!
This works great for me now!

Here's the thing for me. I dont want to delete any duplicates if they're on seperate albums, only if I have the second copy in my collection of random songs which to make life easier for myself all have their album set to: Random Songs.
This is filled with surprise surprise, random songs that I like at the time and so get.

For me it'd help a LOT if I could filter the results of the duplicates shown so that it only showed duplicates where one of the album names was: Random Songs.

Any chance of any filter like this?

[Mirko]

@Nila:

I added an option for "only show duplicate, where one of them has album=xyz". I named it "Nila special option" ;-) You can find it in the new version already uploaded.

Please provide feedback, I can't test it with my DB (and I don't intend to make such a thing *g*).

I also did a smaller update-packages (downloadable here), where only the OCX-file is included. This should be sufficient, if you already had the bigger one downloaded.

@KingSparte:

the other thing to look at would be what is the KBPS if they matched the best out of the two might be the one with the higher KBPS and if the list had a check box. the program could automaticly select the lower KBPS file and place it in the "To Be Deleted" list.

In my archive, there are almost all files encoded using VBR. So it makes no sense with my files to check for KBits, because MJ seems to store the kbits from the first frame (I don't know, if it is the "cut through" (??) then it would be usefull).
Edit: I added this option. Me is not everyone, right?

I made the auto-deletion-options visible.

[KingSparta]

Mirko

about the check box list

since one would be 320kbps and the other 128 kbps the program should automaticly check it, meaning it is a dup that matched the same play time as file two. the user could then put all the checked songs in the delete list By [Put All Checked In The Delete Box]

I Am More Thinking Of Somthing Like

[_] The Beatles - Help 320KBPS
[X] The Beatles - Help 128KBPS

[Mirko]

@KingSparta:

Sorry, but I don't get it. You would like to have a checkbox, that "marks" the files matching a specific criteria? So you might find 500 duplicates and would like to check the ones with same duration but different kbits.

Hm. The checkboxes take quite much space in there (and there is not enough for that, is it?). Would you like to have an option (maybe try right clicking on the delete-buttons, you might find something similar) to pre-select some items matchin specific criteria? I could add something like this quite easy. It would make the same you would do by hand clicking on the listitems, selection for deletion.

That's a quite good idea, I think. No automatic deletion during processing, but after that for specific criterias. I like that. I have done that. Open issues: The delete-operations should only work on that selected files. I will add a function to this tomorrow.

Thanks a lot,
Mirko

[KingSparta]

>> Would you like to have an option (maybe try right clicking
>> on the delete-buttons, you might find something similar)
I just fingered that out, that works good too. :-)

by the way

[Shift & The Delete Key] Will Delete the marked Files In the Duplicates Play List. what i was thinking if a user Pushes "Delete" you have a message box comeup and it tells the users where the files are. you can also tell the user to mark the files and use Shift and push the delete key to delete the files in the duplicates folder.

[KingSparta]

Playing some more with it.

there seems no way to automaticly select files that are the same.

if the Kbps the size and the play times are all the same. the program will find it as a dup, but the user must then manualy delete one or the other.

I would prefer not to manualy delete and allow the program to select one if everything is the same.

I have 500|PLS| files the size and the play time is not the same on all of them but the program will not select them for deletion. it seems the files sizes are close to the other, and also the playtime but not the same.

Also

if you change this line in the registry.reg file

"PluginMode"=dword:00000000

to

"PluginMode"=dword:00000001

the plugin will stay resident, so when the user goes to the Duplicate folder and then back to your plugin the dup list will still be there, this will allow the user to continue where he left off and there would be no need to rescan the files.

if you do this you will need to make sure when the user presses "Delete" that all files sent to the dups folder is fromved from list 1 (the dup list)

by the way, I would like to know how many dups are in the dup list do do not hide it.

[Charlemagne 8]

You would have to be able to globally set preferences for automatic selection of the desired file to delete. There actually could be someone that WANTS to delete the 320 KB instead of the 128 KB file.

[Mirko]

@KingSparta:

there seems no way to automaticly select files that are the same.

There will be this evening (that will be about 12hrs from now).

by the way, I would like to know how many dups are in the dup list do do not hide it.

You mean I should show the "label1"-thing to all users? I thought I did that in the "summary box"... ok, I will add it (without the "randomly set" label).

I have 500|PLS| files the size and the play time is not the same on all of them but the program will not select them for deletion. it seems the files sizes are close to the other, and also the playtime but not the same

"It's not a bug, it's a feature". This will occur, if the difference in duration is not big enough (maybe you didn't change the value on the setup-page?). Is this the case here?

Btw: This resident-thing: I did add a "load" and "save" button to the plugin to store the search results during two sessions. I didn't know that I can change the default handling (thanks a lot for this one!). I think I will change this (and add a button to clean all memory, just in case someone doesn't want to have a 20MB heap allocated all the time *g*).

if you do this you will need to make sure when the user presses "Delete" that all files sent to the dups folder is fromved from list 1 (the dup list)

I don't think so. You can't delete a file a second time, if it is marked as "for deletion". Did you try that? It _should_ be in there.
But for viewing pleasure, it might be a good option.

You would have to be able to globally set preferences for automatic selection of the desired file to delete. There actually could be someone that WANTS to delete the 320 KB instead of the 128 KB file.

I thought about exactly that yesterday and also thought, that nobody would actually want that :-) Ok, I will make it an option.

[nila]

Yes yes yes!!

Your an angel or a god or something
That works perfectly.
I love the little name for me too even though the name might seem a little confusing to other users who just discover the plugin.
I dont need it personally because of the fact I name all the albums a certain way (ie Random Songs) but it might be worth adding another option whereby they can either specify an album that one of them needs to match as in my case, or the other choice might be to let users select a specific directory that the song needs to be in as alot of users might keep all their single songs seperate from their albums.

Thanks for my option though, I love it, it's soo pretty

When your adding the rules to allow automatic selection of which song to delete, I'm always going to want to delete the one that's in 'random songs'. If you included the path option too that I mentioned above maybe you could include a rule in the 'auto delete' option that allowed the one that matched the album name or the path dir to be the one automatically deleted.

Your plugin is a god send

[Mirko]

Update:

I developed a "little" speedup of the search-routine. I have to test it some more (it works with about 9.999 random text strings taking about 5secs), but I guess I can speed up the whole thing a lot. Yesterday the plugin took about an hour for my collection. That's too much for me.
I may do a new alpha version for this testing. If someone is interested in this, I would really like some volunteers. But I have to say: It may not function properly, but it doesn't make any weird things with your database. I will post the link here, when I have included it. This will be in about 8 or 9 hours, because I'm currently at work and have to do something ;-)

@Nila: Thanks for the positive feedback!

Mirko

[nila]

np,
and I'll test it for you. I have around 7000 mp3's

[KingSparta]

Mirko

It might helped also if I would have checked for an update

> You mean I should show the "label1"-thing to all users? I thought
> I did that in the "summary box"... ok, I will add it (without
> the "randomly set" label).
yes but it does not tell you as the process is working.

Also if you do another search, the summery for the last search does not clear. if you could make the summery add files as it finds dups that would be good. I am not sure if this would slow the search down allot.

I still do not see how to delete the same exact file. if you take a file and make a copy of that file and load it into MJ that is the one i want to delete, not the one off 5 seconds.

Since the program only will select it to be deleted if it is off so many seconds how about revirsing that so if it is off so many seconds do not select it for deletion.

[X] Reverse Delete Selection

This will select all files under the Default "2 Seconds" but leave the other files that are over 2 seconds or what ever a user selects.

[Mirko]

@KingSparta:

Is the file found as a duplicate? Did you order the listview by matchquote ("%") by clicking on that column?
There seems to be a missing option for automatic selecting same files. But I'm curious, if it is found...

Thanks,
Mirko

[KingSparta]

>> Is the file found as a duplicate?
YES, if you have two files and one is a copy of the other it finds it as a dup, there is no way of deleting it unless you do it manual.

% is 058% on all dups like this, all fields are the same, Size, Bytes And Kbps

-----------------------------------------------------------

if i am not mistaken if you set the seconds off to lets say 2 it must be off 0-2 seconds to match? or More Than 2 Seconds To Match?

If i was to set this to 2 seconds i would think from 0 to 2 seconds the program should find a match and put it in the auto delete list.

-------------------------------------------------------------------

I would like to delete all files that are exact matches, give or take 2 seconds (anything over my setting, it may be another version of the same song), also the one with the lower KPBS, also the one with a smaller file size.

[Mirko]

@KingSparta:

You should get a "100%"-quote for exact matching files. I checked it with my database (I did copy some files for testing this) and they all got 100%. And if you get 100%, you can auto-select them using the right mouse button on the delete-button.

I also did the tuning you suggested with the reg-file. Now it stays ;-)

@Nila, @all:

I included the new routine for searching duplicates. It works quite good, so good in fact, that I integrated it in the "normal" version.

[KingSparta]

>> You should get a "100%"-quote for exact matching files.
I will take a picture later today of it and show you they are the same. at least the data shown is.

[Mirko]

@KingSparta:

I believe you. But I'm interested in a screenshot. Maybe you should send it to my private adress (mwalter@2via.de).

Thanks a lot,
Mirko

[KingSparta]

Will Do, By The Way How Do You come Up With The Percent?

If It Is By File Name That Maybe Why The Percent Is Off.

I Will Scan Now And Send It To You, I Also Had A Problem With The Song Play Time Was Wrong On Some of The Files, I Will Try To Recreate That Also. It basicly was the seconds were correct but the mins were 00:14 when it should have been something like 03:14

I Just did a run this time I got 75% but the file size, bytes, and Kbps match. Picture on the way.

that was for version 1.0.251 i see you have a new version out.

i will see if the problem is still there

[KingSparta]

Just checked the new version problem is still there

Now I have 75 files that = 100%

Besides deleting them manualy how do I get them Automaticly into the Dupster?

It seems the problem happens when you have "files with less duration" checked.

[Mirko]

@KingSparta:

Did you try using the right mouse button? It opens a contextmenu where you can choose "autodeletion..." or something similar. There you can select "delete same files". This should do the trick I think.

HTH,
Mirko

P.S. The mail is not relevant anymore, is it?

[KingSparta]

>> It opens a contextmenu where you can choose "autodeletion..." or
>> something similar. There you can select "delete same files". This
>> should do the trick I think.
yep, no go

they say 75% but they all match all areas Bytes, Kbps, and File Size

playing with it some more.

Try this, set the Duration from the Default "2" Seconds To "0" thats where it seems to go bad, and then It gives the wrong Percent

[KingSparta]

After I set it back to 2 seconds I get 100% for all of them but one.

that one seems to have a slight dif in the song name

Love story & Love Story (Where Do I begin)

so i do not understand why that Precent should change if i change the seconds in duration if the Bytes, Kbps and duration is the same it should be 100%.

On a happy note:

By the way, this version is screaming with speed!

[Mirko]

KingSparta:

As far as I know you develop yourself in vb. So here's the coding:

   If Trim(A1) <> "" And Trim(A2) <> "" Then
       bA = (InStr(1, A1, A2) > 0 Or InStr(1, A2, A1) > 0)
   Else
       bA = False
   End If
   If Trim(T1) <> "" And Trim(T2) <> "" Then
       bT = (InStr(1, T1, T2) > 0 Or InStr(1, T2, T1) > 0)
   Else
       bT = False
   End If
   
   qFS = IIf(FS1 > FS2, CDbl(FS1) / CDbl(FS2), CDbl(FS2) / CDbl(FS1))
   qD = IIf(D1 > D2, D1 - D2, D2 - D1)
   qK = IIf(K1 > K2, K1 - K2, K2 - K1)

   Q = IIf(bA, 1, 0) |PLS| IIf(bT, 1, 0) |PLS| IIf(qFS < cMatchDiff, 1.5, 0) |PLS| IIf(qD < Val(txtDurationOff.Text), 1.5, 0) |PLS| IIf(qK < Val(txtKBitOff.Text), 1, 0)

As you might see, I calculate the quote by Artist, Name, Filesize, Duration and Bitrate. The "cMatchDiff" for Filesize is a constant about 1.015. The rest are the values entered into the textboxes in the setup-tab. If you have differences in filesize e.g., then you would not get a 100% quote.

Any suggestions?

Edit: I updated the plugin with a new autodeletion-option "delete same files matching in size,duration,kbits". Maybe this will do what you like.

[KingSparta]

>> Any suggestions?
Looks Complex

What i do know is if i change the duration in setup from "2" to "0" that should have nothing to do with the Percent between the two files your trying to match.

The setting in setup should be a tolerance

if it = 100%duration to Duration or |PLS| or - set seconds

So if the file was 3:30 to 3:32 or 3:30 to 3:28 the file should match. and it should have nothing to do with the percent or the total Precent of the file.

[Sam.]

Mirko,  This is awesome!!
826 dups out of 15,179.  In about 6 minutes.
To think, I was going to try to find them all manually!

The user interface is great too.  You should work on MJ9.


a couple things...
In the summary, it says 826 dups.  At top, it says Duplicates found: 825

The pop-up note when the mouse is over the bottom box says, "Doublicates selected for deletion".  I know this community spans the globe, so I won't assume that this is not how to spell it elsewhere...

Out of a few hundred 100% matches, there were 4 pairs that had non-matching album names.  In each of the 4 pairs, one of the files had a blank album name.

Clicking on "Delete!" brings up a dialogue box.  The title says "Move duplicates to playlist."  The text says "Remove the files from the duplicates-playlist?"

When I play a song in the dup list and I click anywhere in the dup info window, the window goes blank.

The instruction that pops up after "Delete!" says that you can delete songs from the playlist by pressing [Shift]|PLS|[Delete].  I remember being able to delete from a playlist by pressing [Shift] and right clicking on "Remove".  Well, either way, I can't seem to delete the song.  It'll just be removed from the playlist.


That's all for now.  And again, I love this.

Sam

[Mirko]

@Sam:

Out of a few hundred 100% matches, there were 4 pairs that had non-matching album names. In each of the 4 pairs, one of the files had a blank album name.

Bug-Alarm! Albums were not included in the match-quote-calculation. I have fixed that.

When I play a song in the dup list and I click anywhere in the dup info window, the window goes blank.

I have the same problem here. And have no idea how to fix that. I thought it might have to do with skinning... I already did some pictureboxes around the questionable elements, but nothing (I read this reading ChartFinder-threads).

@KingSparta:

I had a little error in the match-quote-calculation. The match was "qD < val(txtDuration.Text)" but should have been "qD <= val(txtDuration.Text)". I corrected this. Could you please check it again? I have no files matching those criterias or maybe I'm not smart enough to figure them out :-)

Thanks for the feedback,
Mirko

P.S. Updated :-)

[KingSparta]

Mirko

Should you not be Sleeping? or did you just get up?

I will need to download some test files since the last version I was able to Clear things up around this maze of files I have.


'I Can See Clearly Now' Charted At 01 In 1972

Listening to: 'I Can See Clearly Now' from 'The Best of Johnny Nash' by 'Johnny Nash' on Media Jukebox

[Mirko]

@KingSparta:

I should have been sleeping... got up about 04:45 and went to bed about 00:45 (that are 24h-time... I think 04:45 am to 00:45 am may be the right translation). And got up 04:45 today :-/ Yes, maybe I'm a little tired today ;-)

I did another little update before I went to sleep. The "label1"-thing still showed "label1" on startup. Today I will try to optimize some more, I think there is some potential in the way I modify the listviews (I show them to the user during processing, maybe disabling update will speed up a little).

How long does the plugin take on your collection? If I remember correctly, you have about 40.000 songs, so I would be interesting, how fast the new algorithm is. I think I can speed it up a little, if I do a second index (this will complicate things dramatically, but will help searching collections that big... will think about it).

Bye,
Mirko

@Sam:
I repaired all things except the "turning white"-thing. I will care about that later.

[nila]

Mirko - could I possibly have another Nila Special.
For autodelete options I'd like: Album name = Random Songs
Basically for it to just delete whatever is set under the other Nila Special Option

Thanks

[Mirko]

@Nila:
Did you recently look into the new dialog? ;-)
The option is (was, will stay) there.

Edit: I will update the plugin this evening. I think there is a bug in the memory-index-function, that may lead to not finding duplicates with articles ("The", "A") in the names. The routine for stripping those things is ready for action but not included yet.
(e.g. if you filed "Beatles" and "The Beatles" the plugin will not evaluate it as matching).

[graham131]

Mirko,

Hi I am very interested in this plugin - however whenever i download it from the web via MJ it downloaded but fails to show up in my Plugins list.

Any ideas?

Graham

[Mirko]

@graham:

Did you download from the interface-plugins page? Did you start the plugin (look in the plugin-dialog and press the "start" button -> this occurs after a restart of MJ automatically)?
Did you click "ok" the prompt which asks you do integrate something into the registry (you have to)? If this prompt doesn't show up, try downloading it from
here.

[graham131]

downloaded your resistry link and ran it.  This now recognises the Plugin but whenever MJ starts or i click on the plugin i get the following error message.

"Interface Plug-in'DupFinder' must implement |Unknown interface"

Thanks

Graham

[Mirko]

Graham:

Do you have the VB6-runtime files installed? These are necessary (please take a look at the interface plugin chart finder description, KingSparta provided a link to these).
If this all doesn't help, please be patient. I'm currently at work and can't look into it in detail. But I will.

[graham131]

Thanks I will try that, however i am already running the king's chartfinder plug in sucessfully so would assume that I have these necessary files already

Cheers

Graham

PS I appreciate the help and prompt responses

[Mirko]

Graham:

I don't know how experienced you are with computers. But in case you are: There are some files in the ZIP-archive for my plugin that end on .OCX. Maybe some of them are not registered correctly. You could test it by running "regsvr32 [name].OCX" on them (doubleclick and choose the regsvr32.exe from the system32-directory below \windows to run with).
If that doesn't help either, I think the last build is not working. Did anybody else got this error during the last few builds?

Thanks a lot for helping finding errors.
Mirko

[nila]

Mirko,
the option is there, but what I meant was an auto delete option too so that for all the songs that it found using my special rule, it would automatically select the one that had the album name set as 'Random Songs' as the one to delete.

[Mirko]

Nila,

aaaaaaahhhhh... *bang* (did you hear the cent falling?)

I will do this. Later ;-) (btw. why don't you make a smartlist for that?)