Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[couchjr]

Conditions and procedure:

Current version of MC Mac (same behavior on last two versions). Media files stored on attached USB-3 4 TB drive (NTFS, using Paragon driver, disk has been scanned with Disk Utility and shows all good).

110 albums in this batch; ISOs extracted to stereo DFF via SACD_extract or ISO2DSD, converted to DSF using Korg Audiogate or (for a few more recent ones) DFF2DSF from 2ManyRobots. Initial set of tags transferred to files from an MPL generated for the same ISOs (but multichannel extracts) on a Windows version of MC. Pathnames edited from UNC-Windows to Mac style using BBEdit text/html editor. Missing or erroneous tags edited either in MPL using BBEdit (after which library was cleared, files reimported and "update tags from library" and "update library from tags" commands run for all files) or directly in MC's tagger while relevant tracks were highlighted (after which same two commands were run). In the library options, the "Save to tags where possible" box is checked for all allowed fields. There are two or three custom fields; the rest are standard (though more than the default set). Once I was doing most edits in the MC tagger, I exported the latest iterations of the MPL using the Mac version.

Behavior and symptoms:

On the Mac version: If I clear the main MC library (the only one I'm using), close MC, and restart it, and then use "Import from playlist" to import the 110 albums via the latest MPL, they all import (except for one !@#$ track, 2239 of 2240 files). All tag field values show correctly in MC. Cover images imbedded in files look great. All the imported files play through my laptop speakers when I double-click on them. All the DSF files play (including the one that didn't import) when I connect the HD to my Oppo 105 and pull the files via USB without using MC or a serving computer. The sound is ravishing--and more to the point, the path/track names read in the Oppo file browser correctly as I edited them.

If I clear the MC library, restart the app, and try to import the same 110 albums via the direct "Import a single folder" command (selecting the top level folder containing the album folders), MC will only import less than a third of the albums/files. Why?

On Windows:

When I copy the full set of tagged DSF albums to a portable HD, edit the latest iteration of the MPL to convert the Mac pathnames back into Windows style and include it in the folder, my colleague is able to import all but 24 files into Windows MC via direct import (not using "Import playlist" command). Those 24 files are within the Windows path character limit from root to extension, and some are quite short, so that's not the issue. The failed-to-import files are scattered one or two on an album, where all the other tracks imported fine. He says that using Windows MC he can usually play a media file without importing it by double-clicking on it in the file system, but that doesn't work for these 24 files (and as I said, those files play for me). I don't think he's tried importing via the MPL yet.

At various points there were issues with conversion software concerning non-Roman characters (diacriticals like accents, umlauts, etc.) and with MC about colons in pathnames, but those have been resolved by editing and all pathnames now match on the MPL and the actual path to the files on the HD. This is confirmed by my success importing via the MPL.

So theoretically, each of these files is thoroughly and correctly tagged via MC Mac, the filenames and paths match between MPL and file systems, and MC Mac recognizes the files when using the MPL. Why doesn't it recognize most files when importing without the MPL? And why is the importing behavior different for the same files on the Windows side? What's different about those 24 files?

I'd appreciate any help troubleshooting this. I've been able to find and correct many glitches along the way but this is stumping us.

Many thanks in advance for your help.

[MrC]

If I clear the MC library, restart the app, and try to import the same 110 albums via the direct "Import a single folder" command (selecting the top level folder containing the album folders), MC will only import less than a third of the albums/files. Why?

Quick guess - different File Types selected under Audio in the Import dialog?

[couchjr]

Thanks, MrC.

When I import on the Mac side via MPL, there is no dialog to select file types in the import sequence. MC just asks me to select the MPL and when I navigate to it in the file browser and select it, MC immediately begins importing files. (BTW, I found the culprit on the one failing track--missing comma in fllename tag. So now all 2240 tracks import via MPL). So as the user I don't know what if any file-type settings are used.

When I import via Tools -> Import -> Import a single folder, it doesn't seem to matter whether I only check DSD file type or allow several other file types. I still only get 307 files from 28 albums. And like henry2 in a thread near this one, a few of the albums import all tracks with all the tag fields populated, but many only import one or two tracks from an album, most often with missing tags (the occasional "unknown album," or just several blank fields--which *are* populated in the media files--I trust--and the MPL--I know). And 1933 tracks from 82 albums (including embedded cover image files) are not found/imported at all.

This suggests that MC cannot read the metadata saved in the media files without guidance from the MPL. I don't understand this, since the MPL is generated from tags saved to the files and back to the library, many of which were created using the MC tagger. Is it possible the DSF files cannot retain new tag data because of some frame error or lack of standards, and each time MC imports the files via MPL it has to read the tag info back into the imported files? But surely it doesn't need the tags to read the path to import the files, even if it's missing metadata? Why don't I have 1933 more tracks imported, but with empty or corrupt tag fields, when using 'Import a single folder'? (I should say also that the Oppo recognizes these all as DSF files, according to its file-type indicators when it plays them.)

If I use File -> Open Media File, MC will navigate to the file on the HD and will play (through my laptop speakers) any of the files I've randomly selected, including ones that don't import using "Import a single folder." So obviously MC can read the file paths if I use the file browser in this command sequence.   

And why can my Windows colleague import all but 24 of the 2240 files? I haven't yet learned whether he sees all the tags, but will pursue that. That's really the most urgent part of my question--why are those 24 tracks failing on the Windows side? My Windows colleagues are not used to importing via MPLs and don't want to bother with them if avoidable. I don't mind for my own use, if that's the only reliable method, but I presume the MC Mac team wants both methods to work equally well for DSF files.

I noticed when tagging that MC's tagger seems not to recognize the option-key combinations for diacriticals when entering tag values (no Götterdämmerung) but if I create them in the MPL via an editor, MC displays them correctly on import (via MPL), as it does if they were part of the metadata present in the original ISO. That would be a good feature to correct. The Oppo displays them correctly as well.

So I thought perhaps the import failures might be related to this, but among the 307 files that successfully imported via "Import a single folder" are several with é accents and ü or ö umlauts, so I presume the character set isn't relevant, at least to finding the full filename/pathname, since many more than 307 files are free of special characters in the pathnames.

It's a head-scratcher. I hope it's some user error on my part, since that would be easy to fix once known, but I fear it's something deeper.

Thanks again for thinking about this.

[MrC]

I think we'll make better progress by tackling one thing at a time.

Can you narrow the single folder import problem by selecting a folder with just a file or two?

[couchjr]

Not sure what you mean--I think the correct folders with the fewest files have three or four. I can try importing one of those. Do you want me to try importing one of those that *doesn't* import when I try to import the entire set of folders?

Or do you mean you want me to try importing a folder that has several files, of which only one or two imported when I did the batch import?

[MrC]

I'm trying to divide and conquer.  There are too many details here to sort out what are the one, two, or three root causes, so we need to narrow the focus to narrow the possibilities.

If we can isolate the import problem to a file or two, then it may turn out there is something in the file name or path that is causing the issue (some chars in the past had this problem on the Mac MC).  Use MC's file import, not the MPL for now.

We'll deal with the tagging issues later (frankly, I don't know much about the native tagging aspects of DSF, but we'll figure that out, or someone else will chime in).

An MPL import pulls in the file and that named tags into MC's library.

As far as your friends issue, that's a whole different matter.

[couchjr]

OK.

1. I tried importing the RCA Living Stereo album of Stokowski Rhapsodies (6 files, 1 folder). Imported 0 files. This is one of the 82 folders that does not import via the parent folder.  Here are the files from Finder:

Leopold Stokowski - Rhapsodies/
01 - Leopold Stokowski, conductor; RCA Victor Symphony Orchestra - Hungarian Rhapsody No. 2 in C-Sharp Minor.dsf
02 - Leopold Stokowski, conductor; RCA Victor Symphony Orchestra - Roumanian Rhapsody No. 1 in A, Op. 11.dsf
03 - Leopold Stokowski, conductor; RCA Victor Symphony Orchestra - The Moldau (Vltava).dsf
04 - Leopold Stokowski, conductor; RCA Victor Symphony Orchestra - The Bartered Bride - Overture.dsf
05 - Leopold Stokowski, conductor; Henry Schuman, english horn; - Tristan und Isolde - Prelude to Act III.dsf
06 - Leopold Stokowski, conductor; Margaret Hillis, chorus direc - Tannhaeuser - Overture and Venusberg Music.dsf
folder.jpg

No special characters in any of these names.

2. So I imported an album of two organists playing a German organ. Two files out of 11 imported--same result as in the parent folder import. Here's the result, looking at a partial tag list in MC:

Track #   Composer        Name     Source of file   Disc #   Composition   Filename

4   Bach, Johann Sebastian (1685-1750)       Agnes Luchterhandt - An Wasserflüssen Babylon BWV 653b à 5 voci con 2 tastiere   CW SACD     An Wasserflüssen Babylon BWV 653b à 5 voci con 2 tastiere e pedale doppio   /Volumes/RC-4TB/DSD-S/01/Agnes Luchterhandt, Thiemo Janssen - Arp-Schnitger-Organ Norden/04 - Agnes Luchterhandt - An Wasserflüssen Babylon BWV 653b à 5 voci con 2 tastiere.dsf

10   Bohm, Georg (1661-1733)   Thiemo Janssen - Ach wie nichtig, ach wie flüchtig   CW SACD      Ach wie nichtig, ach wie flüchtig   /Volumes/RC-4TB/DSD-S/01/Agnes Luchterhandt, Thiemo Janssen - Arp-Schnitger-Organ Norden/10 - Thiemo Janssen - Ach wie nichtig, ach wie flüchtig.dsf

As you can see, lots of special characters here. Only the ones with special characters in the filenames imported! The full set of 11 filenames from Finder is:

Agnes Luchterhandt, Thiemo Janssen - Arp-Schnitger-Organ Norden/
01 - Agnes Luchterhandt - Magnificat.dsf
02 - Agnes Luchterhandt - Passamezzo Pavana.dsf
03 - Agnes Luchterhandt - Canzona in C-Dur BuxWV 166.dsf
04 - Agnes Luchterhandt - An Wasserflüssen Babylon BWV 653b à 5 voci con 2 tastiere.dsf
05 - Agnes Luchterhandt - Toccata in F-Dur BWV 540 1.dsf
06 - Agnes Luchterhandt, Thiemo Janssen - Ascendo ad patrem meum (2. Vers).dsf
07 - Thiemo Janssen - Tiento LVIII de medio registro de dos tiples de segundo ton.dsf
08 - Thiemo Janssen - Echo ad manuale duplex forte & lene SSWV 128.dsf
09 - Thiemo Janssen - Gelobet seist du, Jesu Christ BuxWV 188 Choralfantasie.dsf
10 - Thiemo Janssen - Ach wie nichtig, ach wie flüchtig.dsf
11 - Thiemo Janssen - Toccata in d BuxWV 155.dsf

Does MC Mac look at anything besides the current path/filename to determine whether it can import? If the filename has been edited and saved via the mpl to the filename tag/media file, would MC choke on a discrepancy between the edited filename and the one it had by default when extracted from the ISO? (I don't know if some remnant of that persists in the media file; but obviously MC can browse to the file and play it via "Open media file" . . . ) Does the import command use different information to locate the file? Conversely, might MC Mac *only* import files in which I edited the filename? At this point, I can't remember which ones I've touched, but ones with umlauts are likely.

Let me know if this test is not what you wanted or there's something else to try next.

Thanks!

[MrC]

Make sure you're running the latest MC.  There were issues with comma and/or semicolons in file paths.

[couchjr]

"Media Center is up to date (19.0.118)"

Also note that most tracks in the organ folder that did not import had no commas or semicolons. True of many of the rest of the 82 failed albums as well.

[MrC]

I'll have to dig through the other reports.  Can you copy the files that don't import and change the names to something really simple, like test1 and test2?

[couchjr]

OK, this is interesting. I duplicated the original folder to make a "Leopold Stokowski - Rhapsodies copy" folder. Renamed all the files in it "01 - Test.dsf" etc. When I did a direct import, all 6 tracks imported - and all the tag values seemed to be there.

So then I copied the filenames from the original files and pasted them over the "test" names, still in the copy folder, and tried importing that. All 6 tracks imported again.

So I tried importing the original folder again, still 0 imports.

So I renamed the original folder "Leopold Stokowski - Rhapsodies.old" and shortened the copy folder name (which contained the copied files, now renamed same as the originals) to "Leopold Stokowski - Rhapsodies" (the original name, thinking that maybe MC just needed to see a recently modified file path) and tried importing that new folder named as the original. 0 imports.

So I added "copy" back to the folder name, and it imported all 6 again. When I removed "copy" from the folder name again, 0 imports.

So then I tried importing the "Leopold Stokowski - Rhapsodies.old" folder (with the original files in it), and all 6 files imported. When I removed the ".old" from the folder name, 0 files imported.

So I take from this that MC Mac seems to have a problem with this folder name, which is the one in the MPL. Any other folder name seems to allow it to import the files.

But why is this name a problem for direct import? And why does it import correctly with this name via MPL? Am I wrong in thinking MC Mac is exhibiting flaky behavior here? I'm not looking forward to renaming all 110 folders and exporting a new MPL; more worrisome still is not knowing whether the new name will develop the same problem once it's in an MPL, or what about the current name is causing the problem.

One procedural note - many of the folder names were edited in the file system and then the MPL was edited to match, and after importing via the MPL and running the "update tags from library" and "update library from tags" commands, I assumed that the new folder name was now in the media tag "filename" path so MC would recognize it. I did not use the "Rename" command, or the tag editor in MC, to do most editing of the folder names. I'm not sure how I could have prior to importing. I presume the result should be the same whether the renaming is done within MC or merely presented to MC as a fait accompli, i.e., the first name it sees. But many of the names of folders that did not import were not (I think) edited at all, so I don't think that would account for all the failures, even if it's a factor.

[MrC]

Create a smartlist, name it Bad Database, hit the Import/Export button, and enter:

  ~d=b

and see what files are listed there, taking notice of paths.  This database lists the files MC refuses to import for some reason.

Do likewise, naming another smartlist Removed Database, and instead use:

  ~d=r

If there are files listed here, these were previously removed in MC, and MC remembers them so it doesn't have to re-import by re-reading the files.

[couchjr]

OK, don't know what a smartlist is. Will swot that up later. Is the idea that I should compare the two lists, and that things appearing on both somehow explain the problem?

I'm working only in the main library. Every time I run a test while I'm working I do an import, then select all, and delete everything from the library, then restart MC. I'm assuming this erases the DB memory and I'm starting with a clean library every time. If this is not so, once we solve the immediate problem I'll need instruction on how to wipe the DB memory explicitly so these ghost conflicts won't happen in future.

Thanks again--good night for now.

[MrC]

Smartlist - under Playlists, right click and create Smartlist.

The idea here is to see if MC has indeed tried to import, and either failed (because it thinks the files are "bad", or because it has already imported them and you've deleted them from MC in the past, so it prevents reimporting unless your force them in with a manual import).

There's a lot of chicanery going on, so we have to piece together what is happening.

[couchjr]

OK, thanks.

As I said, nearly all the files have been imported (via MPL) and deleted from the library in the past, sometimes several times. I'm using the evolving MPL to bring in the files as I refine and verify the tags, either by editing the MPL or through MC's tagger. If MC prevents me from reimporting a file directly once it's been deleted, that would explain everything, and I will need a way to override that behavior.

I need to be able to instruct MC, each time I've deleted all files from the library, to erase its memory of all files previously imported and start fresh with the current versions. Otherwise I can't use the edited MPL to update tag information to save in the files, which is a mighty handicap when making batch changes that might affect more than one album.

I'll check the Smartlists as you instruct tonight, and report back.

Thanks again!

[couchjr]

MrC, this is pretty impressive. "Bad Database" yields 0 files. Which I take to mean MC has no problem reading any of the DSF files. "Removed Database" yielded 11,201 files, including several copies of most of the 2240 files in this batch and probably several thousand copies from the second batch. For many of the versions, all tags were populated, but presumably many had outdated values from various earlier iterations.

Since each file in this smartlist had a little red "minus" icon preceding it, I did a command-A "select all" and a right-click delete to delete all files from the library and from the removed-database list. Then I restarted MC. Now a direct import brings in even more files than in my MPL! Inspection showed that some scattered files had not been properly associated with their albums due to missing or corrupted tags--possibly from conflicts with previous iterations. Anyway, I have repaired all the tags with MC's tagger, deleted all files from the library (and from the removed-database smartlist, which persists and updates every time you delete files from the library). The next direct import was clean, with no straggler files. So I exported a new MPL, deleted everything from the library and from the removed-database smartlist, restarted the app, and did an import from playlist using the new MPL. Worked perfectly, and the number of files (3401) now matches exactly the number from direct import. My spot test of the files that had been stragglers showed that they all played and retained their updated tags.

So, many thanks! It is good to know that I was not imagining that MC was doing something to the database that I couldn't see. And even better to now have a tool and procedure to ensure that the MC database is tabula rasa prior to doing each direct import. This seems an important enough function that it would be nice to see it much more directly available in the user interface, but your expertise in knowing that MC kept a list blocking re-import of deleted files, and in creating a smartlist and the correct rules (and syntax for specifying them--is that part of the expression language?) to identify those files and override the blocking, has been invaluable. I could have blundered around in the software and wiki for years and never discovered those things on my own, I suspect.

[MrC]

Glad you go it working.

In Tools > Import > Configure Auto-Import, there is a setting "Ignore files previously removed from library".  If you disable this, the files you delete (that are under an auto-import folder) will not be put into the Removed database.

[couchjr]

Thanks--I have auto-import turned off all the time (I think) so that nothing can mess up my tags without my knowing it. Will disabling that setting still prevent files from being put in the Removed database if I have auto-import and auto-update turned off?