Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[greynolds]

I've been gradually working on setting up MC to be the central hub for all of my media files.  The kind folks at JRiver recently added support to read tags from Windows Media Center .wtv files which took care of one big group of files and should be useful for others who want to import their WMC content into MC.  My next goal is to do something similar with my BeyondTV recordings, which are transport streams using the .tp extension.

MC doesn't seem to import any tags from the BeyondTV recordings, so the name is just most of the filename and pretty much no other tags are populated.  In examining the files with a hex editor, most of the tags are embedded at the end of the file, though some of the tags seem to be somewhat garbled.  Here's an example from one recording:

Code: [Select]

SS-ActorsSS-Channel19"SS-Display-title0Antiques Roadshow (1/2)SS-DVDReG)ady
TrueSS-EPGIDEP2036520442SS-Genre&Collectibles / ArtSS-MPAA-Rating SG)S-Orig-Airdate20141201,SS-Series-description SS-Series-title$Antiques Roadshow(G)SS-Show-description(
A Steiff pushmi-pullyu ``Doctor Dolittle'' figure, circa 1967; Pablo G)Picasso linocut, circa 1950; a pair of Italian-carved door panels, circa 1550.SS-Show-titG)le(Junk in the Trunk 4SS-Start&130620741611149852(SS-Station-CallsignWGBHDT,SSG)-Target-DurationCNS36000000000 SS-Target-Start&130620744000000000SS-TZBias300*G)SS-Unique-Channel-ID8000020000100019000000028055SS-UserID&SS-Year-Of-Release
Notice how the name of the Show-title tag has some junk in the middle of the tag name.  Unless this is because of some standard content that needs to appear at regular intervals in a transport stream, reading the tags from the file may not be practical.

There is another option though.  Someone wrote a tool that will parse the BeyondTV server, obtain the tag info from it, and write it out in XML format.  Here's an example for the same show:

Code: [Select]

<?xml version="1.0" encoding="utf-8"?>
<episode-properties>
<property name="Recommendation">False</property>
<property name="NoTags">TRUE</property>
<property name="FullName">M:\BeyondTVRecordings\Antiques Roadshow (1_2)-(Junk in the Trunk 4)-2014-12-03-0.tp</property>
<property name="ReadOnly">True</property>
<property name="Managed">False</property>
<property name="Name">Antiques Roadshow (1_2)-(Junk in the Trunk 4)-2014-12-03-0.tp</property>
<property name="Folder">BeyondTVRecordings</property>
<property name="Length">6464013792</property>
<property name="LastWriteTime">130620784262105836</property>
<property name="LastExistsTime">130645065298143831</property>
<property name="Added">130644675583206028</property>
<property name="AddedBias">300</property>
<property name="Duration">42582399996</property>
<property name="FileFormat">MPEG-2 Transport Stream</property>
<property name="StationCallsign">WGBHDT</property>
<property name="EpisodeDescription">A Steiff pushmi-pullyu ``Doctor Dolittle'' figure, circa 1967; Pablo Picasso linocut, circa 1950; a pair of Italian-carved door panels, circa 1550.</property>
<property name="Title">Antiques Roadshow</property>
<property name="UniqueChannelID">000020000100019000000028055</property>
<property name="TargetStart">130620744000000000</property>
<property name="EPGID">EP2036520442</property>
<property name="EpisodeTitle">Junk in the Trunk 4</property>
<property name="Description"></property>
<property name="InProgress">False</property>
<property name="UserID"></property>
<property name="OriginalAirDate">20141201</property>
<property name="DisplayTitle">Antiques Roadshow (1/2)</property>
<property name="DVDReady">True</property>
<property name="MovieYear"></property>
<property name="Channel">19</property>
<property name="Editable">TRUE</property>
<property name="TZBias">300</property>
<property name="Actors"></property>
<property name="TargetDuration">36000000000</property>
<property name="ActualStart">130620741611149852</property>
<property name="Rating"></property>
<property name="Genre">Collectibles / Art</property>
<property name="SrtExists">False</property>
<property name="DisplayText">Antiques Roadshow (1/2) - Junk in the Trunk 4</property>
<property name="SortableName">Antiques Roadshow (1/2)</property>
<property name="SortableTime">130620561611149852</property>
<property name="Watched"></property>
<property name="LastPosition"></property>
</episode-properties>
The source code is available for this tool, so it should be reasonably easy for me to modify it to write out a JRiver sidecar file instead of the above XML format.  Beyond that, I would need to setup a service that watches for new .tp files in the BeyondTV recording folder and writes out new corresponding sidecar files.

So to my questions...

I manually created a sidecar file for one of the BeyondTV recordings that was already in my MC library, but there doesn't appear to be a way to get MC to automatically update the tags from it.  I let it sit for 15 minutes or so with nothing happening.  Using the Update Library (from tags) feature does the trick.  Will this happen automatically if I wait long enough (as in will MC notice the existence of the sidecar file at some point and update the tags)?

Is there any interest from the JRiver team in working on supporting using tags in the .tp files?  I suspect that the tags are specific to BeyondTV, so I can certainly understand if there's little or no interest in working on it, but figured I would ask.

A potential alternative solution is to get a subset of info from the filename, which looks like:

Antiques Roadshow (1_3)-(Anaheim)-2014-04-21-0.tp
Chicago Fire-(Ambush Predator)-2015-01-13-0.tp

The first part of the file name is the series name, the second part (after the dash) in parenthesis is the episode name, the next part is the recording date.  I'm not sure what the last part (-0) is.  Oddly, BeyondTV puts the 1 of 3 into the series name rather than at the end of the episode name for Antiques Roadshow.  I'm not good enough with them, but I suspect that regular expressions could be used to extract that info from the filename in the auto import tag setup.  Though it would provide less info than the other option, it would presumably be less work and would at least get the series name, episode name, and recorded date.

Suggestions would be appreciated, but I'd like to end up with something that will work without ongoing manual intervention.  If MC's TV recording functionality improves enough, I could consider completely eliminating WMC and BeyondTV, but until then I'll be using them.  BeyondTV is actually used primarily as a backup just in case WMC fails to record something.

[glynor]

It only reads the sidecar tags automatically at import time, I believe.

However, doing the Update Library (from Tags) function can be automated.  It can definitely be automated via COM (so, if you're going to write something to auto-create the JRsidecars, you could then, as a second step force the update).  It can probably be automated via MCWS too, though I haven't looked.

[greynolds]

It only reads the sidecar tags automatically at import time, I believe.

However, doing the Update Library (from Tags) function can be automated.  It can definitely be automated via COM (so, if you're going to write something to auto-create the JRsidecars, you could then, as a second step force the update).  It can probably be automated via MCWS too, though I haven't looked.

Thanks for pointing me in the right direction.

It looks like the web service interface is the best option.  The Search method can be used to get a list of the relevant files.  They're all in a folder named "BeyondTVRecordings", which makes the query nice and simple:

http://{serverip}:52199/MCWS/v1/Files/Search?Action=mpl&ActiveFile=-1&Zone=-1&ZoneType=ID&query=BeyondTVRecordings

This returns a mpl file that contains each file and all of it's fields:

Code: [Select]

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<MPL Version="2.0" Title="MCWS - Files - 2364">
<Item>
<Field Name="Key">12893448</Field>
<Field Name="Filename">\\{filepath}\Antiques Roadshow (1_3)-(Anaheim)-2014-04-21-0.tp</Field>
<Field Name="Name">Antiques Roadshow (1 3)-(Anaheim)-0</Field>
<Field Name="Date">41750</Field>
<Field Name="Bitrate">12046</Field>
<Field Name="Duration">4258</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">TV Show</Field>
<Field Name="Date Imported">1419997111</Field>
<Field Name="File Type">tp</Field>
<Field Name="File Size">6413092320</Field>
<Field Name="Date Created">1419994013</Field>
<Field Name="Date Modified">1398128824</Field>
<Field Name="Compression">tp video (video: mpeg2, audio: ac3)</Field>
<Field Name="Sample Rate">48000</Field>
<Field Name="Channels">2</Field>
<Field Name="Bit Depth">32</Field>
<Field Name="Width">1920</Field>
<Field Name="Height">1080</Field>
<Field Name="Aspect Ratio">16:9</Field>
<Field Name="FPS">29.9700889587402344</Field>
</Item>
</MPL>
Next, there's a method named SetInfo that can be used to set each field value:

http://{serverip}:52199/MCWS/v1/File/SetInfo?File=\\{filepath}\Antiques Roadshow (1_3)-(Anaheim)-2014-04-21-0.tp&FileType=Filename&Field=Series&Value=Antiques Roadshow

So I should be able to create a service that monitors the search results (perhaps once every 30 minutes) for new .tp files that don't have the proper tags set and fire off a series of SetInfo calls to update the appropriate tags.  Another option is to go with a command line program that gets run as a scheduled task by Windows every 30 minutes (or whatever makes sense).

The only minor snag in all of this is that it looks like BeyondTV marks its recording files read only, which causes MC to gives an error about the file being read only when it attempts to update the tags as a result of the web service call.  But that should be easy enough to resolve by removing the read only attribute from the file before making the SetInfo web service call.  If I want to keep things clean, I can re-set the read only attribute when I'm done.

Thanks again for the tip.

[JimH]

I've been gradually working on setting up MC to be the central hub for all of my media files.  The kind folks at JRiver recently added support to read tags from Windows Media Center .wtv files which took care of one big group of files and should be useful for others who want to import their WMC content into MC.  My next goal is to do something similar with my BeyondTV recordings, which are transport streams using the .tp extension.
A potential alternative solution is to get a subset of info from the filename, which looks like:

Antiques Roadshow (1_3)-(Anaheim)-2014-04-21-0.tp
Chicago Fire-(Ambush Predator)-2015-01-13-0.tp

Please take a look at the topic called "Carnac" on our wiki.  It does that kind of thing.  It might need to be extended for this case, but it also might work.

[greynolds]

Please take a look at the topic called "Carnac" on our wiki.  It does that kind of thing.  It might need to be extended for this case, but it also might work.

Thanks.  It looks like Carnac would definitely need to be extended to handle this case, but would still only pickup a few fields.

Something that just occurred to me that could be a potentially slick solution would be to add a new function that could be used in the Auto Import "Apply these tags" section in the folder configuration (Tools -> Import... -> Configure Auto-Import -> {select a folder} -> Edit -> Apply these tags (optional) -> Add -> Custom -> Value -> Insert Function -> {select the new function here}).  The new function would call an external program which would return the value to be used.

Another possibility would be to allow the user to configure a substitute/override for Carnac for an auto import folder.  The substitute would call an external program which would be expected to return XML in the same format that's used in Sidecar.  MC would then use the info in that XML to populate the various fields that are supplied and auto fill any other fields just as it does now.  So the external program could supply series name, episode name, original record date, episode description, and so on, while MC would populate things like aspect ratio, program length, etc. that aren't supplied.

For now, I'm going to pursue the option of using the web service calls as it shouldn't be too difficult to get working.  With a little luck, I may be able to get it done over the long weekend and potentially share the result with others who may be interested in using the same solution.

I'm not very familiar with what the thetvdb.com web service supports for queries, but I may also take a look there and see if there's a reasonable way to get the season / episode numbers and other data from the series name / episode name or series name / original broadcast date as I could incorporate that into my program as well.

[greynolds]

I got a solution for this written today and I'm pretty happy with the result.

To start, it gets a list of BeyondTV shows that have been imported into MC via MC's Search web service.  This is just based on a query for items that have BeyondTVRecordings as part of their path.  So:

http://{serverip}:52199/MCWS/v1/Files/Search?Action=mpl&ActiveFile=-1&Zone=-1&ZoneType=ID&query=BeyondTVRecordings

This gives me back the list of files, along with their tags, that I need to take a look at.

Next I scan through the list 1 file at a time, skipping over any that have the string "BeyondTVTagsImported" in their Notes tag.

For each remaining file, I make a query to BeyondTV's web service to get the list of tags for the file.  Then I query thetvdb (using an app id that I registered with them) to get the series id (and a few other things, such as the series description) from the series name supplied by BeyondTV and then another query to get their episode info based on series id and original air date.  I then use the MC SetInfo web service function to set each tag for the file, adding a Note value of "BeyondTVTagsImported" so that I won't do anything with this file next time.

I've got this setup to run as a scheduled task every hour, which should be adequate for getting new recordings tagged in MC.

Series on PBS presented a bit of a challenge.  First of all, the thetvdb default result for Antiques Roadshow is for the UK version.  I added code to check for a show title of "Antiques Roadshow" and replace it with "Antiques Roadshow (US)" so that the first result that comes back is now the correct series.

Next, the various Masterpiece Classic shows (Downton Abbey, Mr. Selfridge, The Paradise, etc.) all have a series name of "Masterpiece Classic" in the BeyondTV tags.  I added some code to figure out which Masterpiece Classic series it is and use that for the series lookup.  The other issue is that the original air date in the US is different than the original air date that is in thetbdb database, so I can't (easily) get season and episode numbers for these shows.  A few of them actually have the series and episode numbers in the episode names, so I may add some code to parse them out later.  Unfortunately, they aren't consistent: "Season 5_ Episode 1", "Season 2" (no episode number), "Series 2_ Episode 1", etc.

If anyone is looking for the same solution for importing tags from BeyondTV recordings, I'm willing to share what I came up with.

[glynor]

That's pretty cool.

One comment... It seems like you're doing more manually than is needed in the script.  It seems like a function of the fact that the shows are imported into MC without tags, and then you're modifying them when your script runs.  But...

Bear in mind, that MC can automatically parse tags at import using it's powerful Expression Language:
http://yabb.jriver.com/interact/index.php?topic=68462.0

So, transforming one auto-filled tag into another thing (or applying tags based on the Filename and other known factors) is simple to do within MC at import time.  The system is quite powerful and can parse the filenames and modify any of the existing tags as they come in (and, critically, before MC does the tvdb lookup that it does on its own).

Things like fixing the lookup of Antiques Roadshow for the proper version are simple from within MC, because it has a [theTVDB Series ID] field (which is relational, and keyed on [Series]) that, when it exists, is always used instead of the series name when looking up metadata for shows.  So, the workflow to fix a series like that, is to select a single, existing example in your Library in MC, and do Right-Click > Get Movie & TV Info.  Change the search that comes up as needed to find the proper show in TVDB.  Then, when you apply it, MC writes the series ID to that [theTVDB Series ID] field. Now all episodes, current and future, of the series (everything where [Series] matches the one you just tagged) are considered to be that particular Series ID (regardless of the [Series] tag at the time).  Now, even though your series is called "Antiques Roadshow" inside MC, all episodes are automatically tagged with the series ID pointing to the US version, and MC will auto-lookup their metadata for you.  If you wanted to call a show "Bee Goo Beep Bop" and have MC automatically consider that to be "Battlestar Galactica (2003)", you can, without any scripting magic at all.

So... Instead of doing so much manually (which is an impressive feat, but will require ongoing maintenance, and seems like it could be fragile).  I'd suggest maybe not working around the system so much.

Perhaps you can record to a non-monitored directory and then run your script on those, writing JRSidecar files, and then move the files to the "real destination directory" at the end of the script?  In that case, you really only need [Series], [Season], and [Episode] in the JRSidecar file, and MC will take care of the rest automatically.

Or, you could keep it like it is, and have your script write the JRSidecar file, and then remove and re-add the file to MC? (Tip, when you do this from MCWS or COM, you have to delete the file twice or MC will restore the metadata from the old removed entry, rather than re-importing the file from scratch.)

Anyway, it sounds like you have a great solution for yourself there, and I don't mean to knock it.  If you were looking for suggestions, though, I'd be thinking about some of this.

[greynolds]

One of the big problems with what you're suggesting is that I can't get the season and episode numbers if I don't have the correct series id to begin with because BeyondTV doesn't have season and episode numbers in their metadata.  I have to get the season and episode numbers based on the series id and the original air date.  If MC would modify their query to tvdb to fall back to using the original air date when the season and episode numbers are missing, that would most likely simplify things a lot.

What I'm doing may require a bit of ongoing maintenance over time, but the list of problem shows should continue to remain small and since I have the source code, it's not a big deal to make updates.  Something I could do is have my program build up a database of sorts of series info which I could modify manually to handle things like the Antiques Roadshow case.  This would have the additional benefit of reducing the number of calls I need to make to tvdb - once a show is looked up once, there's really no need to do it again (I do already cache them within each program run).

[RoderickGI]

One of the big problems with what you're suggesting is that I can't get the season and episode numbers if I don't have the correct series id to begin with because BeyondTV doesn't have season and episode numbers in their metadata.  I have to get the season and episode numbers based on the series id and the original air date.  If MC would modify their query to tvdb to fall back to using the original air date when the season and episode numbers are missing, that would most likely simplify things a lot.

What I'm doing may require a bit of ongoing maintenance over time, but the list of problem shows should continue to remain small and since I have the source code, it's not a big deal to make updates.  Something I could do is have my program build up a database of sorts of series info which I could modify manually to handle things like the Antiques Roadshow case.  This would have the additional benefit of reducing the number of calls I need to make to tvdb - once a show is looked up once, there's really no need to do it again (I do already cache them within each program run).

You may benefit from having a look at the "EPG Collector" source code. Steve Bickell is the author of EPG Collector and is quite happy for others to use his code. Of course check with him first if you like. He provides the source on the download page. See https://sourceforge.net/projects/epgcollector/files/V4.2/

I use EPG Collector (EPGC) to collect my EPG data from the OTA broadcast. As that data does not include Season and Episode numbers, I use EPGC to look those up for me. The hit rate is not 100%, more like 35% on average, but it does work for a lot of programs. Much of the problem is the source data I use which often doesn't have Episode names. I believe that EPGC uses the "Series Name" and "Episode Name" as the key to look up Season and Episode numbers, rather than rely on the "Series ID" and "Original Air Date".

But, if you take a look at the www.thetvdb.com lookup section of the code you may get some hints. Also note that EPGC creates a local database of TV shows and movies which it checks before looking at thetvdb, similar to what you are talking about doing. It also tries to differentiate Movies from TV Shows and looks up themoviedb if appropriate. Again, a source of ideas.

[greynolds]

You may benefit from having a look at the "EPG Collector" source code. Steve Bickell is the author of EPG Collector and is quite happy for others to use his code. Of course check with him first if you like. He provides the source on the download page. See https://sourceforge.net/projects/epgcollector/files/V4.2/

I use EPG Collector (EPGC) to collect my EPG data from the OTA broadcast. As that data does not include Season and Episode numbers, I use EPGC to look those up for me. The hit rate is not 100%, more like 35% on average, but it does work for a lot of programs. Much of the problem is the source data I use which often doesn't have Episode names. I believe that EPGC uses the "Series Name" and "Episode Name" as the key to look up Season and Episode numbers, rather than rely on the "Series ID" and "Original Air Date".

But, if you take a look at the www.thetvdb.com lookup section of the code you may get some hints. Also note that EPGC creates a local database of TV shows and movies which it checks before looking at thetvdb, similar to what you are talking about doing. It also tries to differentiate Movies from TV Shows and looks up themoviedb if appropriate. Again, a source of ideas.

Thanks - I'll take a look at it when I get a chance.  The goal here was to do something fairly quick that handles the shows I watch and I have come pretty close to meeting that goal.  But if I can make it better, there's no harm in doing so.