Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[glynor]

Another troublesome issue I encountered (and have encountered regularly in the past) while writing the Rename, Move, and Copy Files article, actually just caused me quite a bit of data loss this past weekend.

Rename, Move, and Copy Files does not properly copy JRSidecar.xml files to the destination.

* It does copy them where it should.
* It does not update the <Field Name="Filename"> tag in the JRSidecar document, making it useless.

To make the files work right, you have to then manually Update Tags from Library on the files. I try to do this every time, but I forget sometimes.

If you need to re-import files (because you bork something up) and the <Field Name="Filename"> does not match the file on disk, the JRSidecar.xml contents are disregarded, and the files are imported fresh, defeating their purpose.

RMCF needs to do a Update Tags from Library run on all JRSidecar.xml files after it moves or copies them. It would be convenient if it did this even for those copied in Copy-Only (no updating Library) mode, somehow, so those Filename tags could be right.

This is all wrong, but something bad happened to me, and I don't know why.  See below.

[marko]

Wow. I never realised that. Almost all of mine are wrong!

Thanks for the heads up.

[glynor]

This cost me a bunch of data this weekend.  Here's the scenario:

* While writing the Rename, Move, and Copy Files article, I improved my file-storage scheme for TV Shows, separating out Documentary style shows (like Nova, Horizon, etc) into their own parent directory.  So, I applied that.
* I forgot to select all of the files and do Update Tags from Library after applying the new RMCF template.
* A day passes.
* I'm trying to merge two special Keywords I use into one (I accidentally have both an "Exclude from Views" magic keyword and a "Hide from Views" magic keyword, and I wanted to fix it and combine them).
* I somehow accidentally apply (via tagging in the Panes) "Exclude from Views" to my entire audio collection. The Status Display says updating 63,000 some odd files.

If I let this complete, it'll hide every single one of my files from my main views.  I couldn't get undo to work (the counter kept counting).  If I just untag them all, then all of the stuff I previously had hidden, would now again be visible (I use this for sets of media I've imported, but have not dealt with yet to get into a reasonable shape to appear in my Library).  It could be troublesome to track all of these down individually.

If I kill MC, then it'll stop part way through.  And I could restore from backup.  There's probably a recent automatic backup.  But, I just did that Rename, Move, and Copy Files... The backup was from before the RMCF (darn).  But, it was on a fairly limited set of files.  I'd moved those Documentaries, but those are generally "less important" shows.  I'd also renamed a smallish set of files I'd left sitting around in my Record directory.  But, those were mostly older episodes of shows I hadn't watched yet, so they weren't really going to lose much.

So, I killed it.   Re-importing them meant I'd lose [Date Imported] for them all.  That's a bummer, but not the end of the world for the Documentaries, and the other file set was limited.  Deemed simpler than dealing with the audio file catastophe.

Now, they Auto-Import.  Great, more catastrophe. They lost everything from JRSidecar.xml files, meaning they all came in "as default".  Carnac picked up a pile, but a bunch were documentary shows with weird episode numbers (season 0 and whatnot), and they were all junked up. Also, Carnac also didn't pick up my [Series]/[Season]/sXXeXX - Name.ts style scheme for some of them, and the [Series] names were all blank (it got episode, season, and name, but not Series for a big pile.

And it didn't use the JRSidecar.xml files.  Why not?

Oh yeah, I didn't Update Tags from Library, so those were all junk.

It was relatively painless to fix them, because Fill Properties from Filename could handle the task.  But then I discovered a Padded Number problem that was annoying...

[glynor]

Wait. I appear to be wrong.  It does seem to be honoring my JRSidecar files now even though the Filenames are wrong.  Perhaps the issue is more subtle.

I tested, thoroughly, just now (recreated similar conditions, on a smaller scale), and it didn't behave in the same way.  All of my JRSidecars were moved by RMCF, and they were honored when I re-imported them with the "test restored/broken" Library backup I made.

Hmmmm... Why did I lose all that data then?  Because I did lose it. I spent an hour or so cleaning it up.

I did note, while testing, that none of the Copy modes actually copy JRSidecar files (or cover art) at all. I'd really prefer that they do.  But, the rename mode does copy them, and that's what I used. I did not, unfortunately, actually check to see if they were there.  I don't know why, but I didn't. But I just tested with the exact same filename moves I used a couple days ago, and it worked and the Sidecars were there.

I am right, that it does NOT fix the filename tag in the files when it moves them.  But, they seem to work on import anyway, at least with the test I just did. But, when they auto-imported, my data was absolutely all messed up  So... Why did it not use them?

I need to test further.

[6233638]

This, combined with some of the other RMC issues recently, could explain a lot of the "weirdness" I've seen with my movie library over the last few months.
 
The problem is that these changes happened quite some time ago and the effects seem to appear "at random" since the damage is not immediately apparent.

[glynor]

I think I smell it.

Hold on. More testing required.

[glynor]

Hmmm... No.  It won't do it.  I thought maybe because in my previous tests, the filename basically matched, that it worked (using a relative or filename only match on the sidecar tag). I was moving them back and forth to my TV Shows directory (my old scheme and my new one), and they actually came from a mix of places with a mix of naming schemes.  I just tested with a "real" file (something recorded just a day ago or so), though, thinking the data loss might have only applied to those...

1. I tagged the file with a [Comment] so I could identify using the JRSidecar versus auto-lookup smarts.
2. Moved it from Recordings over to the real destination.
3. Restored from backup, so the file is broken.
4. Waited for Auto-Import to pick up the moved file.  In the meantime, I checked, and the JRSidecar was there, and it showed the (incorrect) filename tag inside it.
* <Field Name="Filename">M:\Recordings\SiliconValley-S02E08-WhiteHatBlackHat-26648050-0.ts</Field>
* Actual location: M:\Video\TV Show\Silicon Valley\02\s02e08 - White Hat_Black Hat.ts

It imported fine, and the comment was there.

What the heck?!?  It definitely happened to me on Sunday afternoon.  I came back, did the dumb Pane-tagging mistake.  Killed MC.  Re-opened it, and restored from the last backup.  A bunch of files were broken because I'd done those RMCFs.  It re-imported them, and I had to fix a bunch of their tags. Not all of them, but a bunch.

Darn.  What was going on?  Did I maybe do copy-and-update in a frenzy of some kind (that doesn't copy over the sidecars)?  No. I'd remember, because that would have taken a long time (going from M:\Recordings to M:\Video is lickety-split, but copying is slow).

I'm stumped about the cause.

[mwillems]

Glynor, you've got a lot going on here, but could this be in any way related to the "move all files in directory" checkbox? 

That's been my number one source of sidecar metadata loss in recent times, because when the box is checked, sidecar files are moved but are not renamed so JRiver "forgets" them.  When the box isn't checked, they are moved and renamed.

That plus the weird issue with the "move all files" box getting changed to "on" on some saved presets led me to lose a bunch of data during a RMC last week.

I'm not sure if that could be the issue in your initial breakage, but I thought I'd float it.

[glynor]

Glynor, you've got a lot going on here, but could this be in any way related to the "move all files in directory" checkbox? 

That's been my number one source of sidecar metadata loss in recent times, because when the box is checked, sidecar files are moved but are not renamed so JRiver "forgets" them.  When the box isn't checked, they are moved and renamed.

That plus the weird issue with the "move all files" box getting changed to "on" on some saved presets led me to lose a bunch of data during a RMC last week.

Yeah. I wonder.  My presets were messed up.  I thought I'd fixed them before I did those moves, but I can't say for sure.