INTERACT FORUM

More => Old Versions => Media Center 12 (Development Ended) => Topic started by: gummbah on August 21, 2007, 01:19:51 pm

Title: Duplicates: where do they come from?!
Post by: gummbah on August 21, 2007, 01:19:51 pm
Not sure what happens but Media Center continues to fill my database with virtual duplicates, i.e. the exact file in the same location appears twice in the database. Removing helps, but after (a few) rescans there are new duplicates. Anyone experiences this as well?
Title: Re: Duplicates: where do they come from?!
Post by: eba on August 21, 2007, 01:25:57 pm
I have the same problem whenever I convert from Ape to mp3 (happens with about 70% of the converted files)
The only way to get rid of the duplicates permanently is to manually move the actual files somewhere else (out of MC's import folders), do an import to get rid of the broken links, then put the files back again.
Title: Re: Duplicates: where do they come from?!
Post by: KingSparta on August 21, 2007, 01:31:49 pm
It is known this may happen when a user imported playlists.

I have seen this without imported playlists however
Title: Re: Duplicates: where do they come from?!
Post by: Matt on August 21, 2007, 01:55:11 pm
If anyone can provide a step-by-step to reproduce this, we'd appreciate it.

Obviously it shouldn't happen.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 21, 2007, 03:09:02 pm
I wish I could replicate. Somehow I have the feeling it has something to do with my Ipod.
As if like Media Center scans (part of) my ipod and somehow makes them duplicates.
But not all if my ipod songs become duplicates.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 23, 2007, 12:39:37 am
This is becoming really annoying. I just removed all my duplicates and within a day I have 220 duplicates back!
Does anyone have any suggestions how to solve this?
Title: Re: Duplicates: where do they come from?!
Post by: benn600 on August 23, 2007, 12:46:18 am
Make definite sure that you are not importing any playlists that you exported.  I use iTunes to sync my iPhone as an intermediate (MC syncs files to folder).  At least three times I imported the files and kept noticing duplicates.  It took me about 10 minutes each time to realize iTunes was finding the playlist files and importing those files, too.  Look for a playlist folder or search for the playlist extensions.  It was troubling for me at the time.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 23, 2007, 12:57:30 am
I am not sure I am following you on what to do. Indeed I have recently switched from XP to Vista and used a script to export all settings and playlists in XP and import them in Vista. What do you mean 'look for a playlist folder or search for playlist extensions'?
Title: Re: Duplicates: where do they come from?!
Post by: ThoBar on August 23, 2007, 07:01:30 am
Seeing as nobody seems to be about right now, I'll try to help. If I understand the other guys correctly, under your (auto) import options make sure no playlist file types are selected to be imported. There is a playlist subtree right down toward the bottom of the import files dialog window, in case you didnt notice it there.

If you import a folder with a playlist as well as music files in there, you will get a duplicate if the option to import playlists is selected.

Example: a folder contains
  c:\music\1.mp3
  c:\music\2.mp3
  c:\music\3.mp3
  c:\music\musicplaylist.m3u  (which contains the 3 files above)

 If you import this folder with both the options to import the mp3s (as normal) AND the playlist, MC will import the files twice.

I think this is what they're getting at, and I think that is how it works. ;)


Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 23, 2007, 08:09:22 am
Ah ok, thanks. Yeah I checked it but I only have mp3's selected in the options, so no luck there.
But... I deleted two static playlists (one for music and one for movies), which I made to sync my Ipod with. Since then I haven't seen any duplicates yet. This is only for about 6 hours now, but before duplicates appeared within the hour after deleting them. Could this help in identifying the bug?
Another strange thing I noticed is that when doing an auto-import scan MC says it is analysing the library existing of 4150 files, while there are actually 18199 files in my library. I remember that on XP MC would mention the correct number of files. Not sure if that has anything to do with this.
I'll keep you posted on what happens.
Title: Re: Duplicates: where do they come from?!
Post by: Matt on August 23, 2007, 09:32:43 am
If you import a folder with a playlist as well as music files in there, you will get a duplicate if the option to import playlists is selected.

Example: a folder contains
  c:\music\1.mp3
  c:\music\2.mp3
  c:\music\3.mp3
  c:\music\musicplaylist.m3u  (which contains the 3 files above)

 If you import this folder with both the options to import the mp3s (as normal) AND the playlist, MC will import the files twice.

You should not get duplicates with this scenario.  I just tested it and couldn't trip MC up.  If you can, please let us know the step-by-step.

Thanks.
Title: Re: Duplicates: where do they come from?!
Post by: ThoBar on August 23, 2007, 10:09:44 am
You should not get duplicates with this scenario.  I just tested it and couldn't trip MC up.  If you can, please let us know the step-by-step.

Thanks.

Another theory bites the dust, but I will test tomorrow to confirm. I thought I did see this behaviour at one point, but can't be sure.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 23, 2007, 11:33:44 am
After deleting my playlists that were synced with the ipod I have not yet seen duplicates.
I have now reimported the playlists (which I exported to a txt file). Still no duplicates, but wait and see.
Title: Re: Duplicates: where do they come from?!
Post by: ethereal on August 24, 2007, 03:49:50 am
i can confirm that certain types of playlists in the same folder as the mp3s when imported cause the mp3s to be displayed twice within media center 12.0.301.

i have used playlist creator 3.5 using relative paths to the file - i.e. -

#EXTM3U
#EXTINF:229,A Touch of Platinum - A Life In Music - Volume 2 (CD2) - 03 - I'll Be Home On Christmas Day (Alternate take 4)
EL34EC~1.MP3
#EXTINF:165,A Touch of Platinum - A Life In Music - Volume 2 (CD1) - 01 - Tiger Man
Elvis Presley - A Touch of Platinum - A Life In Music - Volume 2 (CD1) - 01 - Tiger Man.mp3

this works both as a playlist and when imported (it plays the correct tracks and the files are not shown as duplicates)


when i use The Godfather 0.71b2 using custom filename playlist format (relative) - i.e. -

#EXTM3U
#EXTINF:229,Elvis Presley - I'll Be Home On Christmas Day (Alternate take 4)
\EL34EC~1.MP3
#EXTINF:165,Elvis Presley - Tiger Man
\Elvis Presley - A Touch of Platinum - A Life In Music - Volume 2 (CD1) - 01 - Tiger Man.mp3

this works as a playlist (it plays the correct tracks) but the files are shown as duplicates.
the problem could be the \ at the start of the file name - i have not had a chance to edit a playlist and import it and obviously this is impractical as most people have hundreds, if not thousands, of playlists


i like playlist creator 3.5 as this works with media center but i have to drag and drop each folder to make a playlist - obviously this is time consuming.

i prefer to use The Godfather as i can use only 3 or 4 clicks to create an unlimited number of playlists (it works recursively - takes the name of the playlist from the folder name and drops it into the individual folder with the mp3s all with the click of a button)

i understand that the godfather has not been supported for a long time now but i still find that there is nothing to replace it's speed and ease of use.
i don't know if the playlists created are in the correct format or whether the problem is with media center - i'd obviously like to see media center address this problem as i think it should be a simple matter to fix the bug -if it is a bug or add support for this type of playlist.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on August 26, 2007, 04:59:42 am
Ok, I have not found a solution to this problem. The only way to fix it was to delete the whole library file and then rescan everything, including audio analysis on 18000+ songs  >:(
I think the problem occurred after importing the database from xp to vista. Not sure if that helps.

Cheers
Title: Re: Duplicates: where do they come from?!
Post by: KingSparta on August 26, 2007, 08:00:53 am
Quote
Ok, I have not found a solution to this problem. The only way to fix it was to delete the whole library file and then rescan everything, including audio analysis on 18000+ songs

Me Too
Title: Re: Duplicates: where do they come from?!
Post by: tunetyme on August 26, 2007, 10:52:34 pm
I have been seeing a few duplicates as well.  I thought that I did it accidently. 

This has occurred before I have developed any playlists in MC and I have not imported any.  My normal method is to rip CD's with EAC and convert to an APE file.  I import the folder and update fields such as year, genre, etc.  I hope this helps isolate the problem.

Tunetyme
Title: Re: Duplicates: where do they come from?!
Post by: glynor on September 01, 2007, 04:32:49 pm
I'm having some pretty serious "duplicates" issues too.  Right now my MC library has duplicate copies of every single episode of Seinfeld and every single episode of Scrubs (so that's a whole lot of duplicates).  Unfortunately, I have no idea how it happened.

On thing I've noticed is that capitalization seems to have something to do with it.  My duplicates seem to be copies of the same physical files on disk, differing in the MC database only by the capitalization shown in the [Filename] field inside MC.  I can't do it right now, but I'll try to post a screenshot later today or tomorrow.  In the examples I mentioned above though, one filename will be m:\Video\TV\Blah\Season Bleh\Blah blah blah.avi and the other copy will be m:\video\TV\Blah\Season Bleh\Blah blah blah.avi.  Differing only by the capitalization of the V in "video".

These particular files absolutely are not listed in any playlists, so that couldn't be the cause.
Title: Re: Duplicates: where do they come from?!
Post by: Lasse_Lus on September 01, 2007, 05:04:09 pm
...i also have duplicates  :-\
Title: Re: Duplicates: where do they come from?!
Post by: Jim L on September 02, 2007, 03:50:39 am
I used to have a lot of duplicates when I had an auto-import folder set up which pointed to the location of all my music files (\\nas\media\music). Removing the folder from the auto-import list cured the duplicates but is a bit inconvenient.
Title: Re: Duplicates: where do they come from?!
Post by: Mr ChriZ on September 02, 2007, 11:01:21 am
My library has just duplicated itself =(
Title: Re: Duplicates: where do they come from?!
Post by: hit_ny on September 02, 2007, 12:03:56 pm
Going through the replies so far, it appears ppl with duplicates
- use vista
- may use auto-import.

I'm on XP & don't use auto-import and have not noticed any duplicates.
Title: Re: Duplicates: where do they come from?!
Post by: JimH on September 02, 2007, 01:20:55 pm
What do you have in common?  Which build of MC?  iPod?  Third party plug-ins?

Other software in use?

If your copy of MC is newer than 308, please post on the beta forum.
Title: Re: Duplicates: where do they come from?!
Post by: Jim L on September 02, 2007, 02:41:39 pm
Vista Ultimate 32 bit.
MC Build 308 but was happening with previous builds.
I have an iPod video but I don't think that's relevant here.
Files are all on a NAS, accessed via UNC path.
Title: Re: Duplicates: where do they come from?!
Post by: KingSparta on September 02, 2007, 03:19:31 pm
I Highly Doubt It Is A Vista Problem.
Title: Re: Duplicates: where do they come from?!
Post by: MrHaugen on September 03, 2007, 05:26:39 am
I also had the same problem with some avi files. I tried to remove the files, move it, import again, rename (you name it), but no mather what the file appeared over and over again. I have recently been using Auto Import. I think that is the sinner in this case.
Disabled it, and the problem went away.

I'm running MC build .294 on a XP pro 32 bit, no pugins or anything. I use a file server for my videos.
Title: Re: Duplicates: where do they come from?!
Post by: hit_ny on September 03, 2007, 03:17:30 pm
OK ...there is only one instance i have noticed with the duplicates

..when importing CUE files

...for some reason the recently imported smartlist will show the CUE file imported twice. You have to tick CUE in the import dialog or the cue items won't import.

I don't keep the CUE file itself in the library so its a trivial delete.
Title: Re: Duplicates: where do they come from?!
Post by: Matt on September 04, 2007, 09:08:36 am
We're still hoping for some more details from anyone that has this happening.

MC should always ignore upper vs lower case in filenames.
Title: Re: Duplicates: where do they come from?!
Post by: glynor on September 04, 2007, 09:49:16 am
I haven't been able to reproduce it since I went through and deleted all the duplicates... The problem is that I don't really know "when" it happened.  Just one day I went in to watch an episode of Scrubs, and there were a million duplicates in there.

I know it should ignore capitalization, but in my case at least, that's what happened.

When/If it comes back, and I see it, I'll be sure to let you know.
Title: Re: Duplicates: where do they come from?!
Post by: ThoBar on September 05, 2007, 09:38:37 am
I have a stack of movie duplicates that occurred about a month apart. The only difference I can see in the MC library is z:\xyz\abc vs Z:\xyz\abc. I've only just noticed these in the last few days, and so haven't fully investigated as yet. It may be when I initiated the auto-importation of files. If I find anything further, I'll let you know.
Title: Re: Duplicates: where do they come from?!
Post by: JimH on September 05, 2007, 09:40:49 am
I have a stack of movie duplicates that occurred about a month apart. The only difference I can see in the MC library is z:\xyz\abc vs Z:\xyz\abc. I've only just noticed these in the last few days, and so haven't fully investigated as yet. It may be when I initiated the auto-importation of files. If I find anything further, I'll let you know.
That's the second one that seems to be case sensitive.

Is it also on a network drive?

I'm guessing that it might be related to samba on linux running on a NAS drive.  If so, it may be a bug in the drive software.
Title: Re: Duplicates: where do they come from?!
Post by: ThoBar on September 05, 2007, 09:54:51 am
Quote
Is it also on a network drive?
Sure is.

Quote
I'm guessing that it might be related to samba on linux running on a NAS drive.  If so, it may be a bug in the drive software.
No such luck. Windows XP SP2.

The date it happened for me was 2007/05/10, was there a particular build close to that day causing the issue? (My day is 12-16hrs ahead btw)

Just so you have the full details of my setup...

WinXPsp2 acting as file server.
  I have d:\shares mapped throughout my house as Z:
  d:\shares contains Mount points and Junctions to 10 or so physical drives and folders respectively (so they appear as one via the share)
  MC on the Server also sees these as Z: via a share back to itself (yes I know I could use subst)
  The MC libraries are similarly shared as Y:, once again back to the server also. (no mounts or junctions here though)

What is interesting is that not all of the files are duplicated. Only about 50 movies seem to be duplicated, nothing else... out of a library of 65K files....

[Edit] Just checked Date Created and Date Modified, these seem to be the same between duplicates, as do all other non-editable or non-generated fields
Title: Re: Duplicates: where do they come from?!
Post by: JustinChase on September 05, 2007, 06:41:58 pm
I first reported seeing this in Media Center 12.0.267.  I reported that I saw it after trying to sync to my 60gb iPod.  Unfortunately, I don't remember what exactly I was doing, just that I had some problems, and ended up with duplicates, in some cases 3 duplicates of songs, all 'exactly' the same; path, name, etc.

My files are stored on an external hard drive, that is mapped as Drive M:/ in Windows.

I do not use Vista.

Media Center Registered 12.0.294 -- C:\Program Files\J River\Media Center 12\

Microsoft Windows XP  Workstation 5.1 Service Pack 2 (Build 2600)
Intel Pentium M 705 MHz MMX / Memory: Total - 1047 MB, Free - 136 MB

Internet Explorer: 6.0.2900.2180 / ComCtl32.dll: 5.82.2900 / Shlwapi.dll: 6.0.2900 / Shell32.dll: 6.0.2900 / wnaspi32.dll: N/A
Ripping /   Drive D: HL-DT-STRW/DVD GCC-4243N  Mode:ModeSecure  Type:Auto  Speed:Max
  Digital playback: Yes /  Use YADB: Yes /  Get cover art: Yes /  Calc replay gain: Yes /  Copy volume: 32767
  Eject after ripping: Yes /  Play sound after ripping: No 

Burning /  Drive D: HL-DT-ST RW/DVD GCC-4243N   Addr: 1:0:0  Speed:24  MaxSpeed:24  BurnProof:Yes
  Test mode: No /  Eject after writing: Yes /  Direct decoding: Yes /  Write CD-Text: Yes
  Use playback settings: No /

Portable Device Info
  Removed devices: Chris

Device info dump for 'Chris iPod Video (F:)':

WMDMName: CHRISTOPHER (F:)
DeviceUID: $$?$storage#removablemedia#8&1b50517f&0&rm#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}$0
CanonicalName: \\?\storage#removablemedia#8&1b50517f&0&rm#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}$0
VendorName: ; VendorID: 1452
ProductID: 4617; FirmwareVersion: 1.62
WMDMSerialNumber: NULL
DeviceType: 3; DeviceID: JQ5420C5TXL; DeviceName: Apple iPod; UnsupportedDevice: 0
IsDrive: 1; DriveLetter: F:; DrivePath: ; MainDriveLetter: F:
IsJanusCapable: ; IsIpod: 1
SupportsAlbumArt: 1; AlbumArtEnabled: 1
SupportsFolders: -1; PlaylistFormat: 0; EjectMode: 0; AlwaysPumpCOMMessages: 0
SupportedFileTypes: mp3;wav;mp4;aa;m4a;m4b;m4p;aiff;aif;bmp;jpg;jpeg;gif;tif;tiff;png;sgi;psd;mov;m4v
ContextMenus: Delete
MemoryTypes: Internal Memory
InstallURL:
ApplicationName: Media Center
ImageName: Apple iPod Video (black)
RootBasePath:
AudioBasePath: Music\[Artist]\[Album]\
ImageBasePath: Images\
VideoBasePath: Video\
DataBasePath: Data\
PlaylistBasePath: ROOT\
DatabaseBasePath:
AudibleBasePath: Audible\
MatchKeyExpression: If(IsEqual([Media Type], Image), [Name][File Size, 0], [Name]Clean([Artist], 1)[Album][Genre]FormatNumber([Track #, 0])[Media Type])
ChangeKeyExpression: If(IsEqual([Media Type], Image),,[Rating][Composer][Date Modified])

Device info dump for 'Chris Laptop':

WMDMName:
DeviceUID: {ED1C1A3A-128A-41E0-83D7-364EDD5E881A}
CanonicalName:
VendorName: ; VendorID:
ProductID: ; FirmwareVersion:
WMDMSerialNumber: NULL
DeviceType: 4; DeviceID: ; DeviceName: ; UnsupportedDevice: 0
IsDrive: 1; DriveLetter: ; DrivePath: C:\Documents and Settings\Christopher Baber\My Documents\My Music; MainDriveLetter:
IsJanusCapable: ; IsIpod:
SupportsAlbumArt: ; AlbumArtEnabled: 1
SupportsFolders: -1; PlaylistFormat: 0; EjectMode: ; AlwaysPumpCOMMessages: 0
SupportedFileTypes: *
ContextMenus:
MemoryTypes:
InstallURL:
ApplicationName: Media Center
ImageName:
RootBasePath:
AudioBasePath: \
ImageBasePath: Images\
VideoBasePath: Video\
DataBasePath: Data\
PlaylistBasePath: Playlists\
DatabaseBasePath:
AudibleBasePath: Audible\
MatchKeyExpression: If(IsEqual([Media Type], Image), [Name][File Size, 0], [Name]Clean([Artist], 1)[Album][Genre]FormatNumber([Track #, 0])[Media Type])
ChangeKeyExpression: [Rating]


Interface Plugins:
  Library Server (Active)
  TiVo Server
  UPnP Server (Active)
  Playing Now
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 19, 2007, 01:19:51 pm
I am also plagued with this. I have duplicates of a lot of my files. I am running 12.0.329 on XP Pro SP2. I also have an iPod that I sync to. The iPod is new and I have only recently noticed this problem but I can't swear that the problem wasn't there before the iPod.
Title: Re: Duplicates: where do they come from?!
Post by: craft on September 19, 2007, 03:08:01 pm
null
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 20, 2007, 10:18:43 am
My problem is getting worse each day. Now, I have up to 4 or 5 "copies" of songs.

I feel it is definitely related to synching to my iPod as that is really all I've been doing differently lately. I am currently synching my iPod while at the same time I made some filename changes in my library and reran the import. Here is an example (and these are not the files I did a rename to)

(http://i23.photobucket.com/albums/b387/edbro9/2007-09-20_091450.jpg)
Title: Re: Duplicates: where do they come from?!
Post by: JimH on September 20, 2007, 10:28:35 am
Maybe you're importing playlists or cue files and the tracks, too?
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 20, 2007, 10:35:40 am
No, I have the entire playlist category unchecked in the import options. Plus, I thought Matt said that shouldn't matter. I have no cue files at all.

Just for grins, I deleted all the duplicates in one of the albums, reran the import and every track was duplicated again. Again, an iPod is currently syncing while I'm doing this. The iPod harddrive is not one of my import folders. Plus, from the screenshot, you can see the dups are the same files in the same folders.
Title: Re: Duplicates: where do they come from?!
Post by: mlefebvre on September 20, 2007, 10:59:15 am
I also have been seeing duplicates for months. The other day, I spent 4 hours getting rid of over 1600 songs that were dupes (library has 45,000 songs).

The next day, I had at least several hundred that were back. And they seemed to be the same songs...

While I was deleting the dupes, I did notice one thing that I did not see above: They all belonged to the smartlists that I use to synch my iPod Video 30G. Later I was wondering if it was not related to the handheld cache (which is enabled).

...Michel
Title: Re: Duplicates: where do they come from?!
Post by: JimH on September 20, 2007, 11:00:25 am
Are you auto-importing the cache?
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 20, 2007, 11:01:15 am
I don't know. Where is the cache?
Title: Re: Duplicates: where do they come from?!
Post by: mlefebvre on September 20, 2007, 11:06:38 am
Are you auto-importing the cache?

No. The cache is on a different drive than my music files (C: drive for the cache, D: and E: USB drives for the music files). I just double-checked the import settings and they do not point to the C: drive, only to my 2 music folders on drives D: and E:.

Also, I haven't enabled the Auto part of the Auto-Import. I always import manually (Run Auto-Import Now) after a tagging session. Call me old school, but I'm used to doing that since the old MJ 7 days... ;-)

...Michel.

Title: Re: Duplicates: where do they come from?!
Post by: mlefebvre on September 20, 2007, 11:07:43 am
I also have been seeing duplicates for months. The other day, I spent 4 hours getting rid of over 1600 songs that were dupes (library has 45,000 songs).

The next day, I had at least several hundred that were back. And they seemed to be the same songs...

While I was deleting the dupes, I did notice one thing that I did not see above: They all belonged to the smartlists that I use to synch my iPod Video 30G. Later I was wondering if it was not related to the handheld cache (which is enabled).

...Michel

Oh BTW, I'm also running Vista...
Title: Re: Duplicates: where do they come from?!
Post by: JustinChase on September 20, 2007, 11:15:05 am
Oh BTW, I'm also running Vista...

As I stated above, I see this also, but have never installed Vista.  I do NOT think that Vista is part of the problem.
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 30, 2007, 02:54:25 pm
This problem still exists in latest version (.332). I just ran an import while I was synching the Ipod and MC reported 1266 new files. There were actually only 16 new files and the rest are now duplicates.

This problem seems to only happen when synching an iPod.  I am not importing the iPod cache. That is in a separate directory from my Music folder.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on September 30, 2007, 04:16:54 pm
This problem still exists in latest version (.332). I just ran an import while I was synching the Ipod and MC reported 1266 new files. There were actually only 16 new files and the rest are now duplicates.

This problem seems to only happen when synching an iPod.  I am not importing the iPod cache. That is in a separate directory from my Music folder.

It has been a long time since duplicates were bugging me and still I have had no more duplicates since I deleted my database and re-imported everything. But I remember that I also saw this problem, i.e. that suddenly a lot of new files were imported while my Ipod was connected (see my post from august 21). Not sure if I was synching at the moment though, but it might very well be the case.
So maybe a question to those who have the duplicate problem: do you own an ipod as well?
Title: Re: Duplicates: where do they come from?!
Post by: mlefebvre on September 30, 2007, 04:47:42 pm

So maybe a question to those who have the duplicate problem: do you own an ipod as well?

Yes...

...Michel.
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 30, 2007, 05:43:05 pm
Okay, I just found something. I used the following smartlist to help find the duplicates:
[Media Type]="audio" ~dup=[name],[artist] ~sort=[artist],[name],[bitrate]

What that showed me is that the duplicates are in the \JRHHCache\ (my iPod cache). But, I do NOT have that folder set to import. It seems that if you are synching an iPod during an import, MC will import files out of the cache directory even though you have not asked it to do so.
Title: Re: Duplicates: where do they come from?!
Post by: Alex B on September 30, 2007, 06:18:17 pm
Okay, I just found something. I used the following smartlist to help find the duplicates:
[Media Type]="audio" ~dup=[name],[artist] ~sort=[artist],[name],[bitrate]

What that showed me is that the duplicates are in the \JRHHCache\ (my iPod cache). But, I do NOT have that folder set to import. It seems that if you are synching an iPod during an import, MC will import files out of the cache directory even though you have not asked it to do so.

Hmm... isn't that contradictory to this:

My problem is getting worse each day. Now, I have up to 4 or 5 "copies" of songs.

I feel it is definitely related to synching to my iPod as that is really all I've been doing differently lately. I am currently synching my iPod while at the same time I made some filename changes in my library and reran the import. Here is an example (and these are not the files I did a rename to)

(http://i23.photobucket.com/albums/b387/edbro9/2007-09-20_091450.jpg)

In the screenshot you have identical filenames - not filenames that contain the "\JRHHCache\" path.
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 30, 2007, 06:26:02 pm
Yep, you're correct. I did have dups from the cache directory earlier. I just ran another test import and this time I picked up 245 more duplicates and they are not from the cache directory.  I just recently set a cache directory for the iPod. I didn't have one when I first reported it.

This is disappointing because I thought I was on to something that could be a start to the troubleshooting.
Title: Re: Duplicates: where do they come from?!
Post by: Alex B on September 30, 2007, 06:38:05 pm
Yep, you're correct. I did have dups from the cache directory earlier. I just ran another test import and this time I picked up 245 more duplicates and they are not from the cache directory.  I just recently set a cache directory for the iPod. I didn't have one when I first reported it.

This is disappointing because I thought I was on to something that could be a start to the troubleshooting.

When you have e.g. four duplicates can you play all of them without problems? Do they all have the same file format & bitrate or are some of the duplicates converted according to the handeld conversion options? (Obviously you use the conversion since you have the file cache.)
Title: Re: Duplicates: where do they come from?!
Post by: edbro on September 30, 2007, 06:53:15 pm
Yes, I can play any of the duplicates. My last post was not totally correct. I still have dups from the cache directory, I just didn't look hard enough.

Of the dups, I can play any of them, even if the dups point to the same filename with the same format.

(http://i23.photobucket.com/albums/b387/edbro9/2007-09-30_175149.jpg)
Title: Re: Duplicates: where do they come from?!
Post by: Matt on October 01, 2007, 08:36:01 am
The handheld link was the clue.  Good detective work.

When MC syncs to a handheld, hidden handheld database entries are made and sent to the handheld.  These hidden entries could get imported if import was run during the sync.

This will be fixed in build 335 and later.
Title: Re: Duplicates: where do they come from?!
Post by: Mr ChriZ on October 01, 2007, 08:37:57 am
The handheld link was the clue.  Good detective work.

When MC syncs to a handheld, hidden handheld database entries are made and sent to the handheld.  These hidden entries could get imported if import was run during the sync.

This will be fixed in build 335 and later.

Fantastic  :)
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on October 01, 2007, 09:48:40 am
Ah finally say bye to duplicates. Thanks!
Title: Re: Duplicates: where do they come from?!
Post by: mlefebvre on October 01, 2007, 12:09:45 pm
The handheld link was the clue.  Good detective work.

When MC syncs to a handheld, hidden handheld database entries are made and sent to the handheld.  These hidden entries could get imported if import was run during the sync.

This will be fixed in build 335 and later.

Great!

Will the fix find the dupes and get rid of them?

Thanks.

...Michel.
Title: Re: Duplicates: where do they come from?!
Post by: KingSparta on October 01, 2007, 12:21:49 pm
Interesting...

Not sure if that was why I seen Dups in the past, but it could have been
Title: Re: Duplicates: where do they come from?!
Post by: tunetyme on October 01, 2007, 02:42:51 pm
I am running MC 12.0.318, XP and I have only created 2 short playlists and no smartlists.  Based on the number of files MC claims that I have, I know that it is at least 100 more than my actual total.  What I have observed is that in one case the file name is susie - q.ape.  I found the correct file name plus susie.ape.  Both referenced the same directory and file.  I haven't started using any of the features like syncing to an ipod or converting my ape files to mp3.  Right now everything I am doing is plain vanilla.

Hope this helps to narrow the search. 
Title: Re: Duplicates: where do they come from?!
Post by: eba on October 01, 2007, 04:35:54 pm
My main problem with duplicates remains with converting APE files to mp3 (sending the original files to the recycle bin).  Instantly afterwards I am left with several duplicates, so this cannot be anything to do with syncing to a handheld :-\
Title: Re: Duplicates: where do they come from?!
Post by: tunetyme on October 04, 2007, 08:12:59 am
I need to correct my earlier comment.  After digging into the problem I discovered that all of my duplicates are the results of copies on 2 different hard disks.  I've cleaned it up.  So from a plain vanilla point of view I don't have a duplicate problem.

Tunetyme
Title: Re: Duplicates: where do they come from?!
Post by: Taxee on October 10, 2007, 10:03:22 pm
I have the same thing.  Running XP Pro and just noticed I have >2000 duplicate entries.

I have now turned off auto-import.

Any suggestions as to how I can get rid of them?  It's too many to do manually and I don't want to rescan the library and lose my ratings.
Title: Re: Duplicates: where do they come from?!
Post by: gummbah on October 10, 2007, 11:21:43 pm
I have the same thing.  Running XP Pro and just noticed I have >2000 duplicate entries.

I have now turned off auto-import.

Any suggestions as to how I can get rid of them?  It's too many to do manually and I don't want to rescan the library and lose my ratings.

The only way I could get rid of them was rescanning...
There is an easy way to save your ratings though. Make dynamic playlists, each playlist for a separate rate (1 to 5).
Then save them as static playlists, and reimport that after rescanning, than apply the ratings again.

But I am quite suprised that you still have problems. I haven't seen a duplicate since. Are you using the latest version?
Title: Re: Duplicates: where do they come from?!
Post by: Taxee on October 11, 2007, 09:35:47 am
I am using the latest version but the duplicates may have been created before the update.  I just noticed them but as I have 23K+ files, didn't notice the duplicates until I went to reload my iPod which I only do every few months.

Thanks for the advice re the playlists.  I will try this in the next day or so.
Title: Re: Duplicates: where do they come from?!
Post by: thurston on October 12, 2007, 05:27:24 pm
All my audio listings in Media Center have duplicated.  I'm not sure exactly when this happened.  My video files seem to be fine at this point.  Has anyone figured out how to fix this yet?
I'm running XP, MC 12.0.338, autoimport from internal HD (video and audio files are in 2 seperate folders; the only two on the drive), and have never plugged in my iPod since buying Media Center (this summer).
Title: Re: Duplicates: where do they come from?!
Post by: thurston on October 14, 2007, 05:12:27 pm
More info on the duplicates issue.
The duplicate entries appear to be identical except fo the filename.  Here is an example set of filenames:
F:\\Music\Swollen Members\Bad Dreams\09 Deep End.mp3
F:\Music\Swollen Members\Bad Dreams\09 Deep End.mp3

Note the F:\ (single slash) and the the F:\\ (double slash).

Any tips for correcting this?
Title: Re: Duplicates: where do they come from?!
Post by: JimH on October 14, 2007, 05:17:46 pm
If you search for F:\\ do only those files show up?
Title: Re: Duplicates: where do they come from?!
Post by: thurston on October 15, 2007, 10:32:25 am
If I search for f:\\ the results omit the f:\ entries.  If I search for f:\ all the entries are given.
Title: Re: Duplicates: where do they come from?!
Post by: Scotteq on October 16, 2007, 05:53:11 pm
Registered today to seek help with duplicates...  Seems I picked a good time.

/observing
Title: Re: Duplicates: where do they come from?!
Post by: thurston on October 17, 2007, 01:56:45 pm
 ?
Title: Re: Duplicates: where do they come from?!
Post by: Taxee on October 18, 2007, 09:08:23 am
I got rid of mine finally but somehow lost some files in the process.  Still not sure how that happened.  Going into my back up to retrieve them.

It is, or was, a bug.  It may have got fixed but as a result of the screw up, I am currently not a happy camper as it has cost me a lot of time.  Good programs save time, they don't consume it...

Title: Re: Duplicates: where do they come from?!
Post by: thurston on October 18, 2007, 11:28:04 am
I turned off auto-import, deleted my entire audio list (remove from media library only), and manually imported (Tools>Import Folder).  The whole process only took a minute and the dupicates are gone.  I hope this duplicate issue is fixed soon though.  Having autoimport was a nice feature, and I now don't use it for fear of adding all the dups again, having to erase them, and possibly losing my media files as Taxee did.