Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[InflatableMouse]

Weird issue here.

I cleared my MC library and imported a backup from my Windows version, without settings (just the library), because it contains dozens of new custom fields with calculated values (an Ode to MrC!), and nice views.

I ran auto-import and let it clear missing files (no protect missing drives). It looked oke but then I noticed double entries appearing for the same files (ie, every song listed twice, exactly the same, pointing to the same file).

I looked at my import settings and noticed I had ticked everything, not just audio (but data and playlists as well).

I simply deleted everything in MC (not from disk  ). Reran the import but only audio ticked. Few minutes later, everything is still showing twice, even for albums which don't even have a playlist or cue file.

Basically I got less than 8,000 unique files:

Code: [Select]

$ find . -type f \( -name "*.dff" -o -name "*.dsf" -o -name "*.flac" -o -name "*.m4a" -o -name "*.mp3" -o -name "*.ogg" -o -name "*.sacd" -o -name "*.wv" \)|wc -l
7379
It seems every time auto-import runs in the background, files get added as a new entry in the database. I ran it manually and now everything is showing 3 times.

Import details:

Code: [Select]

Library now has 28968 files. Search and update took 7:41.

Imported 9120 new files.
Fixed 413 broken links. (8 fixed, 405 removed)
Updated 1161 files that had external changes.
Failed to import 1 file.
Skipped 123 files (previously determined to be bad).
Has anyone else seen this behavior? What setting would cause this to happen?

Thanks for the help!

[mwillems]

So, I saw this with a small subset of my total files (1100 out of 60k or so).  It was also an import from a windows library in my case, just like you for old custom fields. 

I have no idea what caused it for some files, but not others, but my workaround fix was to make a smartlist to detect the duplicates and then delete them (making sure to select the "remove from library but not from disk" type option).  Then, I made sure that auto-import was ignoring previously removed files.  The next import, I saw that 1100 or so files were ignored, but they weren't imported again.  If I disable ignoring the previously removed filesm they get reimported again.

I haven't been able to reproduce it at all with a native linux from-scratch library, but importing my windows library does it reliably. This never happened prior to MC25, it only started a few months ago when I rebuilt my server on Debian buster.

[RoderickGI]

Do your file names or paths include special characters, such as "á"? That could be the issue.

Search the forum for "Compound Characters" and "Special Characters". There have been some issues lately where people have moved libraries between Mac and Windows, and now Linux.
i.e. https://yabb.jriver.com/interact/index.php/topic,123158.msg853461.html#msg853461

I think it is a Code Page/Character set/Unicode or similar issue.

JRiver, if Special Characters are the problem, I think this issue needs some attention.

[mwillems]

Do your file names or paths include special characters, such as "á"? That could be the issue.

Search the forum for "Compound Characters" and "Special Characters". There have been some issues lately where people have moved libraries between Mac and Windows, and now Linux.
i.e. https://yabb.jriver.com/interact/index.php/topic,123158.msg853461.html#msg853461

I think it is a Code Page/Character set/Unicode or similar issue.

JRiver, if Special Characters are the problem, I think this issue needs some attention.

Interesting.  Several of the files in my case do have unusual characters, but not all of them.  I would say that 75% of the duplicated files have periods, slashes, brackets, or non-alpha-numeric characters in their path/file names.  However, about 25% of the files don't have any special characters (other than spaces) in the path or file name.  Lots of file paths with spaces don't exhibit the problem, so it's all very mysterious.

More mysterious, a fresh import from the files doesn't have problems, so it's something about the windows library transfer causing the issue.

[InflatableMouse]

Yes, some of the files have special characters, but not all.

I think it might also have to do with the fact that I've imported a Windows library, without MC settings and removed all the files, then re-imported them. For now I've disabled auto-import completely and use an intermediate directory outside my main music dir to import, and use rename/move to get them moved to the main music dir. This seems to prevent double entries.

Possible related is that in Files Panes view, paths are displayed with backslashes, but when using Rename/Move, the preview window contains forward slashes. Maybe it doesn't detect the current files due to the backslashes, and re-importing them gives them backslashes too so it ends up in a loop, not detecting previously imported files.

It hurts thinking about copy/pasting all the custom fields and calculated data to a textfile and recreate it all on a new install. That's going to take hours.

I might try one other thing. Clear the library before backing it up so it only contains views and library fields, not any file or playlist references.