Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[chrisjj]

I reimported a large library after using an external app to empty some fields, but MC has retained the old values. How can I remedy this? I find no Import option to respect emptied fields.

Thanks.

[glynor]

You re-imported the files?  By this, you mean you removed the files from MC, did your changes, and then imported them again?

Did you remove them twice?

Because, if not, they'll retain their old metadata. MC tracks deleted files in a special Deleted Items database to speed future re-imports (handling things like moving files externally and having Auto-Import fix the broken links, and enabling the "Ignore Files Previously Removed" feature of the Import system).

[chrisjj]

You re-imported the files?  By this, you mean you removed the files from MC

No , I left them present.

Because, if not, they'll retain their old metadata.

Well, that's sad. Because ISTM the delete workaround will break all playlist references.

MC tracks deleted files in a special Deleted Items database to speed future re-imports

Hmmm. I'll have slower but accurate anytime  

Thanks for the suggestion. Any other options?

[MrC]

Just the other day I put in a feature request to be able to clear field via MPL and MCWS.  It won't help you with empty fields in an MPL, since by design an data import won't clear existing values.  But if the feature gets implemented, a converter could be created to allow you to do what you want.

[glynor]

MC intentionally doesn't remove existing fields for blank data when importing from MPL or file tags.  It is a safety feature, and I think a good one.

There was a thread about it somewhere recently.  Search for "MPL" and "empty" or something like that on the forums.  One of the JRiver folks explained the details.

[MrC]

It's in the beta forum.

[glynor]

Just the other day I put in a feature request to be able to clear field via MPL and MCWS.  It won't help you with empty fields in an MPL, since by design an data import won't clear existing values.  But if the feature gets implemented, a converter could be created to allow you to do what you want.

Oh, there you go.  That was the thread I was thinking about.

Why didn't you just clear the fields from within MC?

[glynor]

It's in the bata forum.

You sure?

 

[MrC]

Magic!

Why didn't you just clear the fields from within MC?

I need to do it programmatically, from data pulled from AMG or discogs.  I explained the issue in the Discogs thread:

   http://yabb.jriver.com/interact/index.php?topic=87032.msg603736#msg603736

[glynor]

First of all, to check to make sure you did everything you could... Did you select the files in MC and do a manual Update Library From Tags?  Or did you just try re-importing them with the Update for External Changes enabled?  Because the former could work, while the latter might not.

The reason for the "safety" feature is actually quite sound (I remember the discussion with Matt MrC talked about in the other thread).  Basically, there are two easy-to-think-of situations where this could lead to trouble:

1. Imagine a user has some other third-party application that creates MPL files on disk (for playlists, or metadata export, or whatever purpose).  This application is lazy and spits out blank fields (rather than omitting the tag entirely) for fields which contain data it doesn't understand (maybe custom fields, or who knows what).  If I remember right, Matt knew of such tools.  Maybe it was something like tools that didn't support Unicode strings in fields, and would output blank values in MPLs for anything where it found unicode?  Or some such nonsense (or perhaps I dreamed this part).

In any case, these dumb MPL files are sitting on disk in a user's auto-imported directories, and update for External Changes is enabled.  MC happily obliterates metadata that should actually be there, because the MPL told them it should be blanked.

2. A user imports all of their tag-based metadata into MC.  Then, because they are a minimalist crazy person, they intentionally "strip" all of their files of all of the metadata (removing all of the tags on purpose).  Update for External Changes is on (as, I believe, it is by default) and Auto-Import is enabled, and whamo, they lose everything.  Also, very bad.

I think this could be designed around, and that's part of the reason that I think maybe doing Update Library (from Tags) might work, where the Importing option might not.  It is an explicit user-action, rather than an implicit one.

Anyhow, if that doesn't work, and you have a bunch of manually constructed Playlists you need to retain, I'm not sure what to suggest (other than to do the stripping in MC next time)...  But, if you don't care about the playlists, and just want to preserve [Date Imported] and playstats, then I have a solution for you.

1. Find all of the affected files in MC, and add them to a particular Playlist.

2. Then intentionally break the links to these files inside MC (so that the MC record no longer points to the actual files on disk, and points elsewhere).  The easiest way to do this will be to use Rename, Move, and Copy tool in Update Database Only mode.

If it happens that ALL of the files are of a particular type (say FLAC) then just change the extensions of the files using the Find & Replace feature of Rename, Move, and Copy to some "fake" extension (replace .flac with .broken or something).  This will make the next step a bit easier.

3. Get MCFileIngester and install it.

4. If you were able to use my extension changing "trick" above, then do this version of Step 4.  Open MCFileIngester and choose the following settings:

Source Type: Playlist
Source Playlist: Whatever playlist you made in step 1.
Mode: Change Extension

New File Extension: flac (or whatever file type the files actually are on disk, the real, pre-broken type)
Type: Clone
Only clone playstats: Enabled

Then run the ingester.  This will import the real flac versions (which actually exist on disk, since they have matching filenames and are still called .flac instead of .broken) fresh into MC, and then clone over the Playstats, Date Imported, Rating, and other similar "generated by MC" fields.  Regular fields and any custom fields will be left alone (you can look at my MCFileIngester instruction post for the full list of which fields are included in that mode).

4. If your original files were of a variety of different extensions, you can't do that clever trick.  In this case, you'll have to import the original files yourself.  You can still break the links by changing them somehow generic like inserting a fake /Broken/ directory into the filename with Rename, Move, and Copy in Update Database Only mode.  But then you need to:

a. Import the "real" files still remaining on disk.  These will import fresh, because MC doesn't consider the files removed, but still ingested and yet broken.

b. Add all of these files to a different Playlist.  It is essential that the order of the files in both playlists match exactly.  You might want to try this with just a few before you do a bunch, to see how it works.  And make a Library Backup first, please.

c. Open MCFileIngester and use the following settings
Source Type: Playlist
Source Playlist: the "old files" list you made in Step 1 above.
New Playlist: the list containing the newly reimported files you made in step 4b.
Mode: Standard
Type: Clone
Only clone playstats: Enabled

See the other Step 4 above for a description of the Only clone playstats feature.  This method will systematically go through and match File #1 in the Source Playlist, and clone the playstats over to the newly imported files in the New playlist, then File #2, then File #3, and so on and so forth.  That's why it is absolutely essential that the order of files matches identically between the Source and New Playlists (and that no files are omitted or mismatched).  Otherwise it'll happily clone to whatever item happens to match on the "other side".

This works.  I tested with my Test Library just now.  If you try to use MCFileIngester in single-file mode, it checks that both paths exist.  But in Playlist Mode, it just trusts MC's Playlist and uses those (it makes sure the MC File Record in question exists, but doesn't care if the file itself exists in Playlist mode).

This won't preserve Playlists though, so it may be useless to you.

[glynor]

I need to do it programmatically, from data pulled from AMG or discogs.  I explained the issue in the Discogs thread:

   http://yabb.jriver.com/interact/index.php?topic=87032.msg603736#msg603736

I meant him, not you.

I think the explicit Clear command idea you originally proposed is a good one.  Or just making it so an explicit import of the MPL does it, while implicit ones don't.  Or something like that.

But I do worry vaguely about those cases I mentioned above, and others we didn't think about.  I just think destroying good metadata, because an external source of dubious quality says to (perhaps automatically) flashes my D-A-N-G-E-R lights.  In there be dragons.

The safe thing to do is preserve, and if the user wants to clear those fields, they can clear those fields.

Oh yeah, and COM lets you do what you want without issue.  I can easily blank fields via MCAutomation.   

[MrC]

1. Imagine a user has some other third-party application that creates MPL files on disk (for playlists, or metadata export, or whatever purpose).  This application is lazy and spits out blank fields (rather than omitting the tag entirely) for fields which contain data it doesn't understand (maybe custom fields, or who knows what).  If I remember right, Matt knew of such tools.  Maybe it was something like tools that didn't support Unicode strings in fields, and would output blank values in MPLs for anything where it found unicode?  Or some such nonsense (or perhaps I dreamed this part).

In any case, these dumb MPL files are sitting on disk in a user's auto-imported directories, and update for External Changes is enabled.  MC happily obliterates metadata that should actually be there, because the MPL told them it should be blanked.

I don't believe MC will import the data from an MPL pulled in via auto-import.  The playlist itself will be imported, as will the files, but not the data in the MPL.

The only way to get the data is to use File > Import Playlist.

[glynor]

I don't believe MC will import the data from an MPL pulled in via auto-import.  The playlist itself will be imported, as will the files, but not the data in the MPL.

The only way to get the data is to use File > Import Playlist.

Ahh, if there is no way to mass import them (a whole folder full of MPLs), short of scripting it via MCWS or COM, then I don't see any harm in trusting MPLs where the user explicitly said "here is this MPL, I trust it, use this data".

I'd worry if there was any easy way to screw up and do a Tools > Import > Import Single Folder, or File > Import Playlist and pick a whole folder full of random stuff and it recursively does them all.

Explicit user intent versus implicit user intent makes all the difference.

I still think trusting blanked TAGS in files (which could be from god-knows-what calamity) when perhaps perfectly good data is inside MC already, is a recipe for disaster.  But if that's the case (I don't use MPLs like you, and you'd probably know) then I don't see the harm here.

[MrC]

I agree.  We're on the same page re. not using empty values to blank tags.

[glynor]

Back to chrisjj's problem...

This won't preserve Playlists though, so it may be useless to you.

Thinking of this... MCFileIngester can preserve playlists when it does a Replace Type ingestion.  That's because this swaps the new file into the actual MC record, rather than just cloning stuff over to a new record.  Playlists are basically lists of MC file objects from the database, not pointers to files on disk.  I don't know if they're actual memory pointers or references to the FileID in the database tables, but it is the record that counts, not the "file", when it comes to playlists.

So, the idea of Replace Type ingesting is to Replace the filename of the existing record, while preserving all of the existing metadata.  That's not what you want, but that's what it does.

To do this, first, if the "new file" is already in MC, then it clones all of the metadata from the old one overtop of the "new" one.  This prevents any "leaking" from the new one, which I found happened in frustratingly "performance related" cases.  Then, it does the equivalent of manually using Rename, Move, and Copy in Update Database Only mode, and then replacing the current [Filename] with that of the replacement file (but it is much more flexible because it can type in whatever values for the source filename and destination filenames it wants, following no pattern). Then, after it does the replace, it flushes the tags to the files, and then "fixes" the files (file type, resolution, bitrate, etc) doing an Update Library (from Tags) type of operation on them.

It works.  I'd been doing this process by hand occasionally (by carefully crafting the file names of the new and old files so that I could easily do Find and Replace on them in Rename, Move, and Copy), and that was a big part of the impetus for MCFileIngester.  Doing it by hand stinks, so I automated it.

But... It doesn't work here because you want to KEEP the metadata in the new files, but keep the playlists and playstats of the existing ones.

The same code could do this, of course.  Above I explained how when the new file already exists in MC, it does this special pass of cloning from the old explicitly to the new first, and then doing the "replace the existing one" trick.  I could, almost certainly, add a "Only preserve playstats, date imported, playlists, etc..." mode to the Replace Type ingester.

If you can't figure out any other way, yell.  I don't think it'd be too hard.

[chrisjj]

Why didn't you just clear the fields from within MC?

It is part of a process of metadata update from an externally accessed source. But now I see from MrC that the same problem affects updates from internally accessed sources too.

Did you select the files in MC and do a manual Update Library From Tags?

No. I used Library Import.

Trying Update Library From Tags now. ... With a single test file having Name empty, that fails too, falsely reporting:

Code: [Select]

Library updated for 1 files (0 failures)
having made no update at all AFACIS.

Or did you just try re-importing them with the Update for External Changes enabled?

Neither I nor Options Search can find such an option.

Thanks for the other suggestions but yes I do need my playlists retained. Else I would just delete the library, create a new one, and reimport the tracks, accepting loss of customisations. Which I what I have to do on a different project to work playlist import failures each time I use MC to update my DAP from an external library.

If you can't figure out any other way, yell.  I don't think it'd be too hard.

Thanks, but before I pursue that, now I see another issue. InflatableMouse reports library update is unreliable: http://yabb.jriver.com/interact/index.php?topic=77942.msg530139#msg530139 or at least was in MC18. Anyone tested it on MC19?

[MrC]

Trying Update Library From Tags now. ... With a single test file having Name empty, that fails too, falsely reporting:

Code: [Select]

Library updated for 1 files (0 failures)
having made no update at all AFACIS.

Did you modify that file?  If so, then you've changed the file's timestamp, and indeed, MC updates the Library to reflect the new Date Modified.

[chrisjj]

I made the file change necessary for the test.

MC updates the Library to reflect the new Date Modified.

So yes it updated the library, but not quite "Update Library From Tags"

[MrC]

I'm not quit sure what you are saying, but it sounds like you're dissatisfied with the wording of the command.

[chrisjj]

I'm just pointing out that the update of the library entry Date Modified by "Update library from tags" is not from the tags.

[chrisjj]

... and indeed, MC updates the Library to reflect the new Date Modified.

BTW, I find MC reports an library update even if there has been no change to file or library. Reported here.