Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[nwboater]

I would like to know if there is some reason that Carnac would not be getting Metadata and covers for any of the Shows and Movies in the list below from Windows Explorer except for Frasier? They are imported from SageTV. The Sidecar File for Frasier was generated by MC.



Also is it okay for the files to be imported with the Name Field populated? Or must it be imported blank so MC can fill it?

Thanks for trying to help me solve this big puzzle. Please let me know if you need any more information.

Rod

[glynor]

Yeah, it shouldn't be like that.

More details:
http://yabb.jriver.com/interact/index.php?topic=67107.0

We'll get it figured out, Rod.  I'm crazy-busy right now, but we'll get it.

[nwboater]

Yeah, it shouldn't be like that.

More details:
http://yabb.jriver.com/interact/index.php?topic=67107.0

We'll get it figured out, Rod.  I'm crazy-busy right now, but we'll get it.

Thanks Glynor. I did answer your last post on the other thread. I posted this one here thinking it might take someone from JRiver to know if the files looked bad for Carnac to deal with. But you are saying that they are so it must be something in Sage that is writing them wrong.

It would be very helpful if I could get these two questions answered:

1. Is it okay for Sage to populate the Name Field, or must that be left empty for MC to populate?

2. What looks wrong with what I copied from Windows Explorer?

Answers to those questions might help me to troubleshoot.

Thanks,
Rod

[glynor]

1. Is it okay for Sage to populate the Name Field, or must that be left empty for MC to populate?

Carnac will not overwrite anything that is pre-filled with data from any other external source.  It is best to give the files to MC with no metadata whatsoever, basically, other than what the filesystem provides.

[nwboater]

Carnac will not overwrite anything that is pre-filled with data from any other external source.  It is best to give the files to MC with no metadata whatsoever, basically, other than what the filesystem provides.

Thanks for the explain glynor.

In frustration I have uninstalled ALL plugins from Sage. Those include Upcoming Movies, Comskip Detector, Jetty Web Server & SageTV Web Interface. MC still doesn't have good Metadata. I'm now missing some of this good stuff, but really need to get to the bottom of why Carnac is not getting good information.

There are no Sidecar files being written by Sage or MC.

I do notice that in the SageTV guide there is still Metadata showing such as show descriptions and sometimes Cover Art. Could this be somehow getting into the file information and screwing up Carnac?



A big part of my problem in troubleshooting this is NOT knowing WHAT things should look like when I bring them into MC. Glynor commented "Yeah, it shouldn't be like that." in response to my Windows Explorer listing above. But what SHOULD it be like?

Glynor I know you are "Super busy now", but I have a huge favor to ask of you when you can get a few minutes. Could you do a screen capture of your Sage files BEFORE they are imported in MC? And could you do this also in MC Drives and Devices BEFORE import? At least then I will have some idea of what things should look like as put out by Sage and it may help in my trouble shooting.

Thanks very very much.

Rod

[glynor]

Sure can, but not today.

[nwboater]

Sure can, but not today.

Thanks very much glynor - I do understand how busy you are. Any help is greatly appreciated.

Rod

[glynor]

I'm going to make something better to use as a tutorial, and I'd planned to do it tonight, but my wife needed to do some work tonight and I ended up being daddy daycare.  Maybe later, but I'm pretty tired.  Later this week should be better.

In any case, I did think of one thing, and this might very well be your problem.

Carnac, which is the part of MC that parses the filenames (and does an awesome job on Sage recordings) is part of the Auto-Import system.  If you aren't using Auto-Import, it will not work.

You keep mentioning Drives and Devices.  This makes me think you are trying to browse the directory hierarchy and look at the Carnac results either before the files are imported, or after importing them manually (from Windows Explorer or something).  For Carnac to run, you must import the files using Auto-Import.  This means either: using Auto-Import in "auto" mode, or kicking it off "manually" by using the Tools -> Import -> Run Auto Import Now option.

If you don't import the files at all, or you import the files in any other way (drag-dropping them to the top bar in MC, importing from Windows Explorer, or using the Tools -> Import -> Import Single Folder tool), then Carnac is not applied.  You must have your Sage recordings folder added to the Auto-Import folders list (and if you want automatic Metadata from "the cloud", that option has to be enabled in the same dialog).

I don't know for sure that this applies to you, but it might explain what you're seeing.


Click to embiggen

This is my newly imported list as of right now.

I did have to manually import the metadata for a few (but not all) of the Californication episodes shown.  And my most recent recording (Raising Hope, recorded tonight) parsed right correctly (as you can see), but doesn't have any other metadata (no [Description], [Actors], cover art or anything else).  Carnac did correctly parse the Media Sub Type, Series, Season, Episode, and Name tags, in all these cases, it just seems like the Metatada lookup was attempted and failed.  Perhaps thetvdb.com was offline or slow?  Or was getting too many requests on the JRiver "key" at one time?  Not sure.

All the rest, came in automatically just like that.  As you can see, the Names are all right, as are the Series, Season, and Episode tags.  And, over in the Tag Window, I have one of them selected showing that the Description and other fields are filled as well.

Incidentally, the only one showing in that screenshot for which I have a "special" tag-on-import rule is The Daily Show (which makes sure it tags it [Name]=Date if the Series and Episode are blank, so that I get some useful information in the name field for Theater View).  Sage almost never gets the Series and Episode numbers for that show, for whatever reason.

[glynor]

And, actually, I just figured out why the Raising Hope episode lookup didn't work.  Sage tagged it (in the filename) as Season 2, Episode 18.  But tonight's episode was actually episode 17.  Episode 18 hasn't aired yet (and I don't think exists in thetvdb.com's database), so the lookup failed.  That didn't apply to the Californication episodes, which were all named properly by Sage, but it explains tonight's problem.

[nwboater]

Hi glynor,

Thanks so much for showing your files and all of the explanations.

I guess I should have mentioned it before, but most of the time I have my Sage recording folder listed in Tools/Import/Configure Auto Import. I also have checked Get Cover Art, Get Movie & TV Info, and Update For External Changes. I don't see though where you do this: ".....(and if you want automatic Metadata from "the cloud", that option has to be enabled in the same dialog)."

I do realize that files must be imported for Carnac to work its magic. I disable the auto import at times when I'm troubleshooting this problem.

The reason I have used Drives and Devices at times is that for troubleshooting purposes I want to look at how MC sees the Sage Files BEFORE import. I don't know any other way of seeing the metadata - windows Explorer doesn't show it.

It just seems that it would be a big help if I could know what the files should look like from Sage before import and the Carnac treatment. So I asked if you could do this for me:

Could you do a screen capture of your Sage files BEFORE they are imported in MC? And could you do this also in MC Drives and Devices BEFORE import? At least then I will have some idea of what things should look like as put out by Sage and it may help in my trouble shooting.

With your file listing before import compared to mine perhaps something will show that is the cause of Carnac not working. I hope this approach makes sense and that you will be able to show me the screen captures before MC imports the files.

Also please keep in mind that for some reason the series Frasier keeps working properly. It's in the same SageTV recording folder as the others.

Thanks very much for your patience with me on this and for all your efforts.

Rod

[glynor]

Gotcha.

Mine are all imported.  I'd have to remove a whole swath of them to do that, which I don't want to do because I'd lose my play stats.

I can show you in Windows Explorer if you want, I suppose, but they look just like you'd expect them to look.  A bunch of files like this:

PersonofInterest-S01E15-BlueCode-8203175-0.ts
PersonofInterest-S01E15-BlueCode-8203175-0.ts.properties
PersonofInterest-S01E16-Risk-8309652-0.ts
PersonofInterest-S01E16-Risk-8309652-0.ts.properties
PersonofInterest-S01E17-BabyBlue-8529281-0.ts
PersonofInterest-S01E17-BabyBlue-8529281-0.ts.properties
etc...

To be clear, the TS and MPG files themselves don't contain any embedded tags (not that MC can read anyway, and I don't think they do in any case).  The important thing is to make sure that when MC imports the files, there are no JRSidecar.xml files there (not until MC imports them, then it creates them).  Because MC can't write tags to the files themselves, it uses those JRSidecar files instead.  Sage uses the .properties files, but MC doesn't read those and they are ignored.

So, I'm not sure what you need to see.  My webhost is down now too (scheduled maintenance), so you might not even see that screenshot above and I can't add new ones tonight anyway.  But, how would seeing a listing like the one above help you anyway?

Onto the problem...

One other thing you might not realize, that could be hitting you if you are trying to re-import a bunch of old recordings (stuff MC already imported once before way back in the day pre-Carnac)...

MC tracks files even after you've removed them from the database (deleted them and picked "Remove").  That's how it knows not to re-import previously removed files if you have that option enabled in Auto-Import (I don't).  Even if you don't, it still tracks them.  Frustratingly for testing purposes (and re-migrating a big, old, imported set of files using the new Carnac system), if you re-import a file that you already imported, EVER, but then subsequently "removed", MC won't run Carnac on those either.

There's another step.  You have to also remove them from the Deleted Items database in MC.  This is easy, make a new Smartlist or View and use this as the search:

Code: [Select]

~d=r ~sort=[Filename]
I have mine set up in a special "Deleted Items" View under my "All Media" top-level-view, with a Location category (and others) added to make things easier to find.

So, to fix these files (if this is the problem), you have to make that view and then:

1. Disable Auto-Import
2. Remove the imported files from the database.
3. Switch to your Deleted Items View or Smartlist and find the files in question.
4. Delete them from there too.
5. Then re-import them using Auto-Import (either via turning it back on and waiting, or running an Auto-Import scan manually).

[nwboater]

Gotcha.

Mine are all imported.  I'd have to remove a whole swath of them to do that, which I don't want to do because I'd lose my play stats.

I can show you in Windows Explorer if you want, I suppose, but they look just like you'd expect them to look.  A bunch of files like this:

PersonofInterest-S01E15-BlueCode-8203175-0.ts
PersonofInterest-S01E15-BlueCode-8203175-0.ts.properties
PersonofInterest-S01E16-Risk-8309652-0.ts
PersonofInterest-S01E16-Risk-8309652-0.ts.properties
PersonofInterest-S01E17-BabyBlue-8529281-0.ts
PersonofInterest-S01E17-BabyBlue-8529281-0.ts.properties
etc...

Thanks for showing me these. One thing that is different is that I don't have ANY properties files with mine. Neither in my newly Sage recorded ones or the old Sage recorded ones that did import fine. But you mention below that MC ignores them so guess that's not related to my problem.

Re showing me your Sage files before import here's how I do it (this assumes that you are making new recordings on a regular basis): Temporarily remove the Sage Recording folder from the Auto Import list. Let a couple of recordings be made and do a screen capture of those files in Windows Explorer and MC Drives & Devices. Then re-enable the recording folder in auto Import. After they have been imported and Carnac has done it's thing take the same screen shots again. We now can see what the files and any tagging look like as Sage records it, and then after MC deals with it. By seeing the differences in your files with MC working fine, and mine where MC is not it might give us some clues as to what is going on. If this is too much hassle for you, or you are concerned about something getting messed up please let me know. I will certainly understand and will quit bugging you on it.

To be clear, the TS and MPG files themselves don't contain any embedded tags (not that MC can read anyway, and I don't think they do in any case).  The important thing is to make sure that when MC imports the files, there are no JRSidecar.xml files there (not until MC imports them, then it creates them).  Because MC can't write tags to the files themselves, it uses those JRSidecar files instead.  Sage uses the .properties files, but MC doesn't read those and they are ignored.

I didn't realize that the .ts files did not contain any tags (metadata?). In fact I thought you had mentioned in an earlier post on the other thread that if the Name field contains data that MC could not overwrite it. That's why I had spent so much effort trying to determine if Sage or MC was filling The Name Field. Guess I misunderstood you.

I did a recording a little while ago and had Import of the recording folder disabled. Here is the file in Explorer and MC Drives and Devices before Import.






Two hours later (should be enough time for Carnac) the files still look the same and there have been no Sidecar or other files created.


All the files that I have been posting about are newly recorded files so the "Old recordings" issue that you mentioned should not be a problem. But thanks very much for all the good info on importing old files. I need to keep that in my 'Help file'.

Thanks again glynor for all your efforts here.

Rod

[nwboater]

Hello JRiver - Jim, Matt, someone please.

Could you please tell me if Carnac should be able to deal with the files that are in my previous post and the first post of this thread!

I've been struggling with this for quite awhile and still don't have a clue if it's the way Sage is presenting the files or it's something in MC that is not able to deal with them.

Please note in the first post that MC deals fine with Frasier. But I have recorded all kinds of other shows and movies, at least some of which Carnac should handle.

I really need to know which program is causing the problem. Please please take a look and let me know.

Thanks very mcuh,
Rod

[glynor]

I didn't realize that the .ts files did not contain any tags (metadata?). In fact I thought you had mentioned in an earlier post on the other thread that if the Name field contains data that MC could not overwrite it.

That only applies to files that have a JRSidecar.xml file or a Tag on Import rule that writes to a particular tag.  Some file types do have limited (and usually useless unless you carefully encode them yourself) embedded metadata that MC can read (MKVs) and I don't know how those work with Carnac, but it has never gotten in my way.

I'll get back to you with what you want as soon as I'm not daddy-day-care anymore.  Hopefully tomorrow.

BTW: Enabling/disabling Auto-Import is unnecessary.  When I was testing it, I would just turn the "Auto" mode off (leaving all of my folders in the list), let some stuff record, and then do the "Run Auto-Import Now" to import them.  Oh, and yes, the Properties file thing is an option in Sage.  It has no effect on or off I don't think (though I haven't tested, I can't imagine how they would).

Browsing in MC's Drives and Devices part of the tree could be messing something up (maybe it "semi-imports" them then or something?)... Try knocking that off.  Do this, just as a test:

Turn off Auto-Import (Tools -> Options -> Library & Folders: uncheck run auto import in background).  Just leave it off for the duration.
Set up your Sage recordings folder as a watched folder if they aren't already.
Record some stuff.
Check them in Windows Explorer if you like (but if your filenames look like that, then you're fine).
Do Tools -> Import -> Run Auto Import Now.

Simple is best.

[glynor]

PS.  You may want to try also (separately) running an identical test with your comskip setup turned off.

MC might import those EDLs as metadata.  I don't know how that affects Carnac.  Obviously, if that's the cause, it needs to be fixed in MC, but it would be good to know if that's what is causing it.

I don't have any comskip setup at all.

[JustinChase]

PS.  You may want to try also (separately) running an identical test with your comskip setup turned off.

MC might import those EDLs as metadata.  I don't know how that affects Carnac.  Obviously, if that's the cause, it needs to be fixed in MC, but it would be good to know if that's what is causing it.

I don't have any comskip setup at all.

I use comskip, and all of my files have an .edl file before MC imports them, and I don't have any of the issues described here, so I think the edl files can be ruled out as a cause, FWIW.

[glynor]

I'm working on reproducing a test for you now.

Give me a little bit to finish it.

[nwboater]

I'm working on reproducing a test for you now.

Give me a little bit to finish it.

Thanks very much glynor.

I have done 3 recordings per your routine in post 13: 2 misses and a hit. The movies are Barneys Version, and The Green Hornet. No Sidecar files for them. 1/2 hour Show Seinfeld got the Sidecar file with cover and show description.

So now there are two shows that it works fine with, Seinfeld and Frasier. Other Shows we watch & movies no joy. Very very strange!

Tomorrow I'll try some recordings without Comskip, but per JustinChase it works okay for him with Comskip.

Rod

[glynor]

Here's my test:



http://vimeo.com/glynor/mc17-carnac-test

Hope it helps, or is at-least enlightening to other folks.

[Matt]

First, Glynor deserves a prize for that video.  Wow.

As for nwboater's problem, I don't think I really understand what's happening.  More details might help.  Recording inside Media Center is another possibility so that Sage filename parsing isn't required.

[nwboater]

glynor - Incredible video! Thanks very much for doing it. I'm sure this will be very helpful to several people.

Here's a screen shot from your video of some files in Windows Explorer after deleting them from MC (Totally). I take it that these should appear the same as they would if they had just been recorded by Sage.



I just recorded an episode of CSI so I could compare the file naming format to yours. (Just noticed that I recorded CSI NY. Hope that doesn't matter.) So here is my Explorer listing. Please note the following:

1. The only show that gets a Sidecar generated is Frasier. It's the only one out of lots of Shows and Movies I've recorded that gets a sidecar file and shows metadata.

2. In my file listing for CSI there is no showing of the full program name, or of the episode name as there is in glynors. But there also isn't for Frasier yet it generates a Sidecar file.

3. In glynors listing the shows have a 7 digit number before the -0.ts whereas mine are 6 digit. Also his numbers all begin with 8 and mine begin with 2.




Thanks again glynor for all your efforts!

Rod

[nwboater]

First, Glynor deserves a prize for that video.  Wow.

As for nwboater's problem, I don't think I really understand what's happening.  More details might help.  Recording inside Media Center is another possibility so that Sage filename parsing isn't required.

Matt - Thanks for checking in on this thread.

My problem is very simply that out of a LOT of movies and shows recorded by Sage the only one that gets metadata and covers in MC is Frasier. Some time back I imported a bunch of Sage recorded files and they got good metadata and covers in MC. So we are trying to determine why the heck all of my new recordings are not being handled by Carnac and generating Sidecar files.

I asked glynor if he could show me what his files look like before being imported in MC. I was hoping that it might give a clue as to why his are working fine and mine are not. The above post shows an example. Do you see any reason that Carnac should not be working on my files?

Re "Recording inside MC". For the past 2 or 3 months I have been doing that since I really want one program to do everything. Since we use and HDPVR which is considered an 'Analog Tuner' by MC there are several features that we miss that Sage does quite well. I believe that they are mostly caused by the need to record in .jtv with the HDPVR in MC. They are:

1. Comskip wont work.
2. Movies listed as Shows.
3. No Cover Art - only a screenshot

There are some other features that we really like in Sage, but the 3 above are the main ones.

We had hoped that these issues would be resolved in MC so we could stick with recording there. But it looks like it may be a long time, if at all, thus the switch back to Sage.

Thanks again Matt. Would sure appreciate it if you could tell me if Carnac should be able to deal with the files I showed above. At this point I still don't know if the problem is in Sage or MC.

Rod

[glynor]

3. In glynors listing the shows have a 7 digit number before the -0.ts whereas mine are 6 digit. Also his numbers all begin with 8 and mine begin with 2.

I think you might have nailed it right there.  Matt may need to comment, but that makes me suspect...

Matt, I believe that number in Sage is an auto-incrementing "serialization" number to ensure unique file names.  Either that, or it is specific to the user's particular program guide data.  Unfortunately, I don't have any originally named really old recordings to see if I can see it tick back to a different style of numbers.  In any case, it seems like you can't rely on it being a specific 7-digit pattern.  The number string at the end may be of variable length.

But if Carnac is keying off of the exact pattern of "-XXXXXXX-X.ts" rather than a more generic "-NumberString-X.ts" then it would explain this problem.

Other stuff...

1. The only show that gets a Sidecar generated is Frasier. It's the only one out of lots of Shows and Movies I've recorded that gets a sidecar file and shows metadata.

They don't get the Sidecar files unless MC writes metadata to the "file".  Carnac does not do this.  Tag on Import rules create the JRSidecars, as does the automatic Medatada lookup.  But just Carnac doesn't cause them to get written.

Is there anything different about the Frasier file name that might explain why it is working and the others aren't?

2. In my file listing for CSI there is no showing of the full program name, or of the episode name as there is in glynors. But there also isn't for Frasier yet it generates a Sidecar file.

CSI (and CSI-NY) are probably both bad example files to look at at this point.  For me, at least, the filename for CSI comes in as CSICrimeSceneInvestigators, which Carnac translates as "CSI Crime Scene Investigators" and then thetvdb.com lookup fails (it is looking for "CSI").

To get those working, I had to create Tag On Import rules to make the Series name match what thetvdb.com uses (though this is easier now with the fact that MC will use a relational Series ID tag instead of the [Series] tag, once you manually tag just one of the files properly with the manual lookup tool).

I describe what I did to get CSI working in my epic Tag on Import thread.

Those edge cases are easy to address once we get Carnac working in-general for you.  Like I said, I suspect it might be that numbering thing you noted in the filenames.

[nwboater]

I'm really leaning towards the Sage numbering system as being the culprit, but it would be good if Matt could tell me if it should be able to deal with the numbers I am getting from Sage or if they don't look right.

I looked at my Sage files that I imported in MC a few months ago and they were all 7 digit numbers, not six. And they started with 5, 6, 7 or 8. They got good metadata and covers in MC. Those files were all recorded on my WHS machine which I am no longer using with a tuner. All recording is now done on our HTPC where the tuner now resides. So it's a whole new install of Sage.

It would be helpful if other folks recording in Sage could post what kind of file numbering system they are getting from Sage.

I've gone through all the Sage TV settings numerous times and I just don't see what could be causing this big change in the numbering system. Anybody have an idea on that?

glynor, thanks for the further info on how Sidecar files are generated & other good stuff. And thanks again for the great video!  Also do you have any plugins in Sage besides the Web Server stuff??? Just wondering if something from a plugin might be generating the good numbers?

Rod

[JimH]

Great video demo, glynor!  Really nice.  Thank you.

[glynor]

You're welcome to use it.  I set it as a CC Attribution and Share Alike license, and I meant it.  No commercial restriction.  I'm thinking I have enough raw material there in the recording to make a real demo too.  Maybe tomorrow, we'll see.

That was my first real attempt playing with Camtasia Studio.  Great product.  I didn't have to take that video into Final Cut at all.  That was all done in Camtasia.  I used it as a trial to learn the application better.

A few things would have probably been easier in Final Cut (the transitions and fine edits), but only because I know it so well.  Mostly the editor was superb.  I prefer Telestream's ScreenFlow a bit, but that's Mac-only (and if you use Windows apps in a VM on a Mac, the "smart zoom" feature doesn't work at all), but Camtasia is where it is at for Windows screencast recordings.

[Matt]

In any case, it seems like you can't rely on it being a specific 7-digit pattern.  The number string at the end may be of variable length.

But if Carnac is keying off of the exact pattern of "-XXXXXXX-X.ts" rather than a more generic "-NumberString-X.ts" then it would explain this problem.

Carnac currently requires 7 digits.

I wonder, would 6 or 7 digits work?

[nwboater]

Carnac currently requires 7 digits.

I wonder, would 6 or 7 digits work?

Thanks for answering this Matt. So it seems that is my problem.

But the biggest mystery is why does it work for Frasier? Please see Post #1 here for the files and you will see that Frasier is also a 6 digit #.

When you ask "I wonder, would 6 or 7 digits work?" I take it that you are wondering if it would be okay to revise Carnac to allow 6 digit numbers? Right?

Thanks again Matt, maybe we are now getting somewhere.

Rod