Topic: Is everyone getting the "Year" where Season "number" should be? (Read 28163 times)

CountryBumkin · « **on:** August 18, 2015, 11:33:36 am »

When I record an OTA TV Show using mc2xml data, I am getting the Show's "Year" for the Season "number".

For testing, I have one computer using mc2xml with "Microsoft" as the source, and another using mc2xml with "Schedules Direct" as the source. Both sources have this issue. I'm doing all OTA recordings.

For example, an episode of a TV series should be showing Season "1", Episode "6", but instead I get Season "2015", Episode "6". Then the TV's metadata can't be looked up since the season is incorrect.

So - could JRiver be pulling the "Season" data from the data field for "Year"?

EDITED: both Microsoft and Schedules Direct have same data issue

RoderickGI · « **Reply #1 on:** August 19, 2015, 09:11:30 pm »

Does that mean that you have run your stand alone version of mc2xml (I think you have one of those, the donateware version, correct?) and looked at the XMLTV file?
Or maybe you don't have the donateware, based on your question elsewhere. But could you run the integrated version directly outside MC, so that the XMLTV file created is retained. Just for testing of course.

As I'm sure you are trying to work out, the problem could be in the source data (possible) or in the field mapping from mc2xml to MC (less likely). MC correctly maps the Season numbering for me using the output from EPG Collector and the xmltv_ns numbering system.
<episode-num system="xmltv_ns">39 . 4 . 0/1</episode-num>
Season 40, Episode 5, Part 1 of 1

For that matter, does the integrated version of mc2xml, or previous versions, output Season and Episode data in the xmltv_ns format? There doesn't seem to be a way to tell it to do so on the mc2xml web site, or any mention of Season or Episode numbering.

CountryBumkin · « **Reply #2 on:** August 20, 2015, 04:51:36 am »

I'm only using the MC built-in mc2xml now. But with different data sources (Microsoft and Schedules Direct). I did not have this problem when I ran the "stand-alone" mc2xml and imported that data into MC.

If it was a field mapping problem, everyone should have this issue. So far no one else has reported this as an issue on their systems.

I can't imagine what could be different on both of my systems that could cause this issue.
I can always go back to running the stand-alone version - actually, it would be a good test. I'll reinstall the standalone version and see if that fixes the problem.

DocLotus · « **Reply #3 on:** August 21, 2015, 03:21:13 pm »

I would like to know how one even gets the Year, Season or Episode data with the integrated mx2xml grabber? I've never seen that data on my system & have been wondering why it is missing in action?

Have used both MS Rovi & SD.

CountryBumkin · « **Reply #4 on:** August 21, 2015, 03:58:37 pm »

If you are in Standard View, and go to Video>Files then you can add fields (if not already shown) by right clicking on the column header, and a list will popup where you add a check mark to the fields you want to see (such as Series, Season, and Episode).

For TV Shows you should have a Name, Description, Series Name, Season and Episode. The Season is generally 1, 2, 3... etc.

DocLotus · « **Reply #5 on:** August 21, 2015, 04:18:43 pm »

Quote

If you are in Standard View, and go to Video>Files then you can add fields (if not already shown) by right clicking on the column header, and a list will popup where you add a check mark to the fields you want to see (such as Series, Season, and Episode).

For TV Shows you should have a Name, Description, Series Name, Season and Episode. The Season is generally 1, 2, 3... etc.

That sounds great. however I'm confused (as usual) by "Video>Files". I'm in Standard view but can't locate "Video>Files". Are we talking about Options/Video? I see no "Files" listed.

Or... are we talking about the left menu "Video/File Type/Genre Name"?

DocLotus · « **Reply #6 on:** August 21, 2015, 04:41:23 pm »

I'm wondering if we are talking about two entirely different things here?

I want things like... Date, Season, Episode etc in the TV Description which would be on the left menu Television window & would be in the Description when I use the "D" key.

It would be MOST helpful if we could modify the Television TV headers as we can with audio simply by right-clicking on the header & modifying/adding.

CountryBumkin · « **Reply #7 on:** August 21, 2015, 05:01:48 pm »

You can add those descriptions/fields to the "TV View" by modifying the "Info Panel" at Tools>Options>Theater View>Customize file info panel.

What I'm trying to determine is "are the correct Season numbers being obtained from the epg download?" Because I can't get the correct metadata (carnac) without the correct series, season, and episode number - and currently I'm not getting the correct season number.

It may be that the recordings I made over the last two days just didn't have the Season number in the epg download - or it could be something on my setup (but 'm seeing this problem on two separate computers) - so I'm asking if others are having problems getting the season data (season 1, season 2, etc, Not Season 2011, Season 2015, etc.).

RoderickGI · « **Reply #8 on:** August 21, 2015, 06:59:17 pm »

DocLotus, You are off topic in CountryBumkin's thread, but quickly.

Get out of the Television item in the left menu. It isn't the only place to view TV shows, or the best for some things.
In the left menu, click on Video, then Shows. The display is sorted by Series, so you will see all your recorded and downloaded TV shows. Anything with a "Media Sub Type" of "TV Show" in fact.
Double click on one of the TV Series. The display is now split into two panes, and upper and a lower. The lower pane shows Episodes within the Series you selected.
Each line is an Episode. You can immediately see Series, [episode] Name, Season, Episode, Description. If you can't see any of those, scroll to the right. You will see Year, Genre, etc.
Hover your mouse over one Episode line. A popup windows will give you a readable version of the description, a thumbnail, etc.

If any field you want to see is missing, you can right click on any column header, and tick the fields you want to see. You can also drag and drop column headers to change the sequence.
You can see all data MC has about a program by clicking on it, then opening the Tag Windows using the Action Window menu, or pressing Alt-Enter.
You can change the Tag Window display by clicking on the small menu icon in the top left of the Tag Window. The minimise and close buttons are in the top right of that window, as usual.

DocLotus · « **Reply #9 on:** August 21, 2015, 07:38:16 pm »

Quote

DocLotus, You are off topic in CountryBumkin's thread, but quickly.

Opps... sorry about that. We seem to be talking apples & oranges. I'll start another thread.

One other thing which I'll discuss in the other thread...

Quote

Get out of the Television item in the left menu. It isn't the only place to view TV shows, or the best for some things. In the left menu, click on Video, then Shows.

I have no Shows under the Video menu.

CountryBumkin · « **Reply #10 on:** August 22, 2015, 04:57:32 am »

This is my issue:

and prior to MC21 (using same data source):

RoderickGI · « **Reply #11 on:** August 22, 2015, 08:59:03 pm »

Well that certainly looks like a data mapping problem, although your images don't show the data for the same set of programs. That would have been irrefutable evidence, if EPG data from SD used in MC20 showed up differently to EPG data from SD in MC21, for the exact same programs. Or the same for Microsoft EPG data.

I have been tardy in not updating to MC21 yet. Life and other issues and all that.

I guess the question in addition to the one you asked is; Are the any users that are not using mc2xml who are on MC21 and are, or are not seeing the problem. Perhaps Astromo?

CountryBumkin · « **Reply #12 on:** August 24, 2015, 12:33:28 pm »

Bump

--- please --- I am still waiting for a response from anyone as to whether this problem is just happening to me - or is everyone using mc2xml in the US recording TV getting the same thing?

If you are in the US and record TV using MC21's built in mc2xml EPG data retrieval source - are you getting a Season Number (like a "1", "2", etc.), or only getting a Year for the Season (like "2007", "2010", etc.) for each recorded TV show?

Yaobing · « **Reply #13 on:** August 24, 2015, 05:32:11 pm »

The xml file produced by mc2xml contains episode info only in two formats, "ms_progid" which we can not interpret, and "onscreen" which apparently has a human-readable numerical number, such as "23" which we interpret as "episode 23". If there is season info, it would be in the value in "ms_progid" format.

Since we can not get any season info, we fill it with "year", if available.

Are you sure your pre-MC21 data was not obtained using zap2xml data? That was the source from which we would get episode data in "xmltv_ns" format.

RoderickGI · « **Reply #14 on:** August 24, 2015, 06:07:05 pm »

That would be consistent with my post over here then: http://yabb.jriver.com/interact/index.php?topic=99542.msg690017#msg690017

So Yaobing, did you licence mc2xml from John Douglas to include in MC, and if so, did you discuss having the xmltv_ns data added to its output?

If you did licence mc2xml, are there any restrictions on how it can be used? For example, are users allowed to run the included version independently, which would be useful in diagnosing problems with the source data, and maybe even changing collection parameters to a user's preferred settings?

You have said that the xmltv.xml file is not kept after the MC EPG loading process. Could that be changed so that it is kept, for problem solving purposes?

As an example, EPG Collector uses the following process to ensure that a copy of the most recent successful xmltv.xml file is available for all but a very short time.
1. The xmltv.xml file is not deleted at the end of the previous run, and still exists in the output folder.
2. EPG Collector runs and saves the output to a temporary file in the output folder until it completes all collection, including metadata collection from thetvdb and themoviedb.
3. EPG Collector deletes the existing xmltv.xml file and renames the new file to the configured xmltv.xml file name.

That process has helped me solve EPG problems by analysing the most recent xmltv.xml file.

Thanks for your assistance Yaobing. I hope you had a great holiday by the way. Too short I am sure!

Yaobing · « **Reply #15 on:** August 24, 2015, 09:00:04 pm »

Quote from: RoderickGI on August 24, 2015, 06:07:05 pm

So Yaobing, did you licence mc2xml from John Douglas to include in MC, and if so, did you discuss having the xmltv_ns data added to its output?

I do not know the name of the developer who provided mc2xml. He never wrote his name in our email exchanges.

My hunch is that zap2xml scrapes data from a website and thus knows how to interpret the data. mc2xml on the other hand, may just directly download data from Microsoft without interpretation, thus I am not sure whether it is even possible to add xmltv_ns format.

Quote

If you did licence mc2xml, are there any restrictions on how it can be used? For example, are users allowed to run the included version independently, which would be useful in diagnosing problems with the source data, and maybe even changing collection parameters to a user's preferred settings?

I was not involved in licensing negotiations. But I think we are not allowed to let our users run it independently.

Quote

You have said that the xmltv.xml file is not kept after the MC EPG loading process. Could that be changed so that it is kept, for problem solving purposes?

Probably not.

Quote

Thanks for your assistance Yaobing. I hope you had a great holiday by the way. Too short I am sure!

Thanks. My vacation was fulfilling but also tiring.

CountryBumkin · « **Reply #16 on:** August 25, 2015, 05:03:29 am »

Quote from: Yaobing on August 24, 2015, 05:32:11 pm

The xml file produced by mc2xml contains episode info only in two formats, "ms_progid" which we can not interpret, and "onscreen" which apparently has a human-readable numerical number, such as "23" which we interpret as "episode 23". If there is season info, it would be in the value in "ms_progid" format.

Since we can not get any season info, we fill it with "year", if available.

Are you sure your pre-MC21 data was not obtained using zap2xml data? That was the source from which we would get episode data in "xmltv_ns" format.

Thanks. Question answered. Yes, my MC20 epg data was being pulled from Zap2it which is where the Season number came from.
Since Season number is no longer available from Microsoft/Rovi, the choice now is to rearrange all my TV show collections to group by Year rather than Season, or find a third-party scraper program that can pull Season data.

I asked on the Schedules Direct forum if "xmltv_ns" could be added to the epg data, but I'm not sure that is even possible or helpful - especially if the Micrososft/Rovi data doesn't contain Season numbers to start with. I'm not sure where the xmltv_ns data comes from.

RoderickGI · « **Reply #17 on:** August 25, 2015, 05:46:12 am »

Quote from: CountryBumkin on August 25, 2015, 05:03:29 am

Since Season number is no longer available from Microsoft/Rovi,

Don't be too quick to assume that. The xmltv_ns format wasn't available in zap2xml until I asked for it. The data was available, it was just that the format wasn't output in the xmltv file.

Quote from: CountryBumkin on August 25, 2015, 05:03:29 am

I asked on the Schedules Direct forum if "xmltv_ns" could be added to the epg data, but I'm not sure that is even possible or helpful - especially if the Micrososft/Rovi data doesn't contain Season numbers to start with. I'm not sure where the xmltv_ns data comes from.

As per my post over here, I think you needed to ask the developer of mc2xml (who is John Douglas I suspect, same developer as zap2xml) to include the xmltv_ns Season and Episode format in his output file, rather than ask Schedules Direct. As long as SD have the Season and Episode data in their dataset for programs, John can format the output.

The xmltv_ns format is just another format for presenting the Season and Episode number data. Not supporting the format, yet, doesn't mean that the data isn't available from Microsoft. In fact, I would be surprised if it wasn't. Perhaps ask WMC users, if you can find any on the new system, if they are getting Season and Episode numbers in their EPG data.

I can chase this stuff up and ask John if you don't wish to. But I can't even use the Microsoft data here in Australia, nor can I really test the implementation in MC without destroying my setup. All I could do was pick a location in the USA and download the data, then check it had the data and format required.

But I'm happy to do it you wish. Just let me know.

Yaobing · « **Reply #18 on:** August 25, 2015, 11:22:00 am »

Yes, the information is in the file, but it is just not in the format that we can interpret.

<episode-num system="ms_progid">1.309729053</episode-num>
<episode-num system="onscreen">26</episode-num>

The "ms_progid" system appears to be different for different data sources. The above is from Rovi. When they had data from zap2it, the format is different. I have also seen something different from these two (perhaps for data outside of North America).

CountryBumkin · « **Reply #19 on:** August 25, 2015, 11:51:21 am »

Would it be possible to have JRiver provide a "lookup" database where, when the Series and Year is known the database would return the Season number? Some sort of "User contributed" database (like the MyMovies program does to collect cover art and other metadata from Users).

For example, if the Series is "The Big Bang Theory", Episode 5, and the Year is "2007" the database would return "Season 1".

The reason Season is needed is that Carnac won't look up the series metadata based on the current data (which is "The Big Bang Theory S2007E05". It needs to be in the format "The Big Bang Theory S01E05").

RoderickGI · « **Reply #20 on:** August 25, 2015, 08:38:12 pm »

Quote from: CountryBumkin on August 25, 2015, 11:51:21 am

Would it be possible to have JRiver provide a "lookup" database where, when the Series and Year is known the database would return the Season number? Some sort of "User contributed" database (like the MyMovies program does to collect cover art and other metadata from Users).

That database already exists. The data is in thetvdb. Specifically, all series I have ever looked up on thetvdb have had an "Originally Aired" date, which includes the year.
See example at http://www.thetvdb.com/?tab=season&seriesid=247808&seasonid=463443&lid=7

The trouble is, MC works in this sequence (Yaobing, correct me if I am wrong please);

EPG is imported -> Program is scheduled to be recorded based on rules -> At the beginning of a recording, the program file is named and created, using Season and Episode numbers from EPG if available -> Recording finishes and files are finalised -> Auto Import starts in MC -> CARNAC is run and parses information from the file name (Which is unnecessary if Season & Episode data is already known from the EPG data, but is necessary if a video file is added to a MC monitored folder from an external source) -> Get TV & movie data is run and gets TV data from thetvdb, and updates tags.

So to create the correct file name at recording time, either thetvdb has to be looked up prior to a recording starting, perhaps when a rule is used to select a program for recording, or at the actual time of a recording.

Looking up thetvdb when a rule is used to select a program for recording is bad, because that happens very often in MC. Rules are regularly examined to see if a recording needs to be scheduled.
Looking up thetvdb at the actual time of a recording would be bad, as it could result in problems with recordings waiting for a response. That could be worked around by using a temporary name until a response is received, but then multiple files would need to be renamed at some time, before CARNAC runs. Messy. Especially if there is no internet connection at that time, for whatever reason.

No, Yaobing has now confirmed that the data is in the EPG source. It just isn't in the xmltv_ns format. Zap2xml also used to only provide "dd_progid" and "onscreen" format data, but the developer, John Douglas, was able to either interpret the data, or he had access directly to the Season and Episode data, and was able to quickly produce the xmltv_ns format, and include it in the zap2xml output. So the developer of mc2xml, who is probably also John Douglas, could probably do the same thing.

There is a slight difference, in that it looks like Microsoft have invented their own format in "ms_progid", because naturally they couldn't keep using the "dd_progid" format from zap2it. That could make the issue more diffiuclt.

Bottom line, do I ask the developer if he can add the xmltv_ns Season and Episode format, or do one of you two ask him?

CountryBumkin · « **Reply #21 on:** August 26, 2015, 05:15:20 am »

Quote from: RoderickGI on August 25, 2015, 08:38:12 pm

Bottom line, do I ask the developer if he can add the xmltv_ns Season and Episode format, or do one of you two ask him?

Are you asking me or Yaobing?

I would think that JRiver should ask since they made a donation, they have more goodwill with the developer.

But if you have a connection with him it wouldn't hurt to ask twice (and if you PM me the contact info/name, I can ask as well).

Thanks for following up.

Yaobing · « **Reply #22 on:** August 26, 2015, 07:59:19 am »

I will ask.

Yaobing · « **Reply #23 on:** August 26, 2015, 10:35:51 am »

The answer came back, and it is not good.

I contacted mc2xml developer, who confirmed that he also wrote zap2xml. He said that ms_progid is just an ID that they use distinguish episodes of programs and it does not really contain season and episode numbers. So there is no way of adding xmltv_ns in the output.

Users wanting season/episode numbers will have to use zap2xml with -D option (one http request per program). This is explained on zap2xml webpage.

RoderickGI · « **Reply #24 on:** August 26, 2015, 08:15:24 pm »

Thank you for doing that Yaobing.

I believe that it is very important that the majority of MC TV functionality users can get Series, Season, and Episode numbers in their EPG data, and that that EPG needs to be obtained, at least for the American market and as many markets as possible outside the America, by functionality seen to be built into MC, such as with the integration of mc2xml. Without this, series recording can not be reliable. Without that, you will not win over the many WMC users that come looking for a new solution, or the many users that are still using independent point solutions, because they are perceived as better.

Converting those people to new MC users will drive sales, which will drive TV development, which is what I want out of all this.

So a solution needs to be found.

For example, EPG Collector can look up Season and Episode numbers from thetvdb based solely on Series and Episode names. It is open source software, so it would be easy to see how it does that. If a post processing step of EPG data could be done, after each collection, and a local cached database of current TV programs was used to minimise lookups to thetvdb, just as EPGC does, then the EPG data could be significantly improved before being imported into MC.

A smart MC user could even use mc2xml to get good quality data from Microsoft, then use EPG Collector to add Season and Episode numbers to that data, and then import it into MC via an XMLTV file. But as we have seen recently in the forum, there are a lot of users that won't even download and install a separate program to get EPG data, let alone set up two applications that work together to improve the quality of the EPG data.

Combine EPG data that includes Season and Episode with improved functionality around TV recording rules, and JRiver MC will be an obvious choice as a replacement for WMC users, and will pull in many of the point solution users as well. Except for all those with encrypted cable channels, of course!

So, what can be done Yaobing?

Yaobing · « **Reply #25 on:** August 26, 2015, 09:17:24 pm »

All I can say is I will investigate our options. Thanks for the tips.

rammingspeed · « **Reply #26 on:** August 26, 2015, 10:14:39 pm »

There is also another open source java program called XMLTVDB that parses the XML file, looks up the season and episode information from TVDB, and then re-writes the XML file. Someone on this forum mentioned it and I have been using it with a good degree of success. I use the paid version of mc2xml and schedule it to download the guide XML file. XMLTVDB is set up to monitor the folder for new XML files and immediately goes to work to update the XML. The updated file is scheduled to be imported by MC at a later time. Messy but I get it to work most of the time.

The program can be found here -- https://code.google.com/p/xmltvdb/

It doesn't appear to be in active development, but it was originally written to work with ~~TVHeadEnd~~ ForTheRecord. There appears to have been a process by which it would only lookup the series / episode numbers for shows hat were scheduled to be recorded. I assume this was to lessen the load on the TVDB site.

This process got me thinking that if MC scheduled this type of lookup to TVDB ONLY for programs being scheduled to record, the data could then be used to name the file and add the season / episode numbers to the tags. My guess is that this would meet the requirements for a majority of your customers, but maybe I just say that because it meets mine.

RoderickGI · « **Reply #27 on:** August 27, 2015, 12:07:15 am »

Quote from: Yaobing on August 26, 2015, 09:17:24 pm

All I can say is I will investigate our options. Thanks for the tips.

I understand, and appreciate all the good work you do.

CountryBumkin · « **Reply #28 on:** August 27, 2015, 05:11:31 am »

Quote from: RoderickGI on August 26, 2015, 08:15:24 pm

EPG Collector can look up Season and Episode numbers from thetvdb based solely on Series and Episode names. It is open source software, so it would be easy to see how it does that. If a post processing step of EPG data could be done, after each collection, and a local cached database of current TV programs was used to minimise lookups to thetvdb, just as EPGC does, then the EPG data could be significantly improved before being imported into MC.

I assume it needs to know the "Year" too (which we have). Otherwise how would EPG Collector know what Season something like "The Big Bang Theory" "Episode 2" is? I have an Episode "2" is every Season from Season 1 through 8. I'll look into EPG Collector.

I'm looking into another program I read about on the Green Button (called the "BigScreen EPG" http://bigscreenglobal.com/bgproduct.aspx?PID=BSEV1) that is reported to provide an alternate/better source for WMC epg data. Not sure yet if this is the answer (may only be useful for WMC users), but if I find anything that works I'll post back.

Also according to this document https://msdn.microsoft.com/en-us/library/windows/desktop/dd776338.aspx - Season data is in the Rovi data going to WMC users.

RoderickGI · « **Reply #29 on:** August 27, 2015, 06:06:14 am »

Quote from: CountryBumkin on August 27, 2015, 05:11:31 am

I assume it needs to know the "Year" too (which we have).

Nope just the Series and Episode names.

For example, Series = "The Big Bang Theory", Episode = "The Codpiece Topology" Lookup result, Season 2, Episode 2.
What that means is that the source EPG data must have both the <Title> and <Sub-title> fields filled in correctly. When both are filled MC puts the <Title> value into the [Series] field, and the <Sub-title> value into the [Name] field.
As the [Name] field must be filled in MC, if the <Sub-title> field is not present in the EPG data, the <Title> value is put into the [Name] field. Also, the <Title> field must always have a value in XMLTV data, so the [Name] field is always filled.

As a result of the above, if the source EPG data doesn't have a value in the <Sub-title> field for a series, then the program isn't recognised as a series, and it is just identified as a TV program or movie, and given the series name as the [Name] in MC. That caused some discussion and angst in the thread recently.

I wrote up a quick process flow for Astromo not so long back, but my search skills are failing me. Basically;
Run mc2xml to get the base EPG data.
Run EPG Collector and import the mc2xml output into the processing stream, as additional information. EPGC can import multiple external XMLTV files. (I'm not sure how difficult this would be to set up in EPGC, as it will probably need to know something about the channels, frequencies, etc.)
EPGC looks up Season and Episode numbers for all the EPG it has collected.
EPGC outputs XMLTV file for import into MC.

BigScreen EPG has been around for a very long time. It is an Australian product, developed in my home town of Melbourne. Last time I looked into it there hadn't been much activity for some time, but it was still available. I think it is a one man product. It was supported out of our Australian MCE forum (now defunct) at one stage, but I think that changed. Original just an on screen EPG collator, it was then integrated into WMC. It never had its own EPG data, because that was not possible for copyright reasons back when it was developed. Whenever I've looked at it I couldn't see the value, as it seemed to just be adding another layer of software to the EPG chain. It is supposed to clean up EPG data though, so may be useful.
But by all means check out if it adds value. My information may be out of date.

That Microsoft document is from July 2009. I doubt it is talking about Rovi data. I can't see any reference to the source.

CountryBumkin · « **Reply #30 on:** August 27, 2015, 08:23:08 am »

Quote from: RoderickGI on August 27, 2015, 06:06:14 am

Nope just the Series and Episode names.

Ahh, "name" yes. Thank you for "bolding".

Quote

BigScreen EPG has been around for a very long time. It is an Australian product, developed in my home town of Melbourne. Last time I looked into it there hadn't been much activity for some time, but it was still available. I think it is a one man product. It was supported out of our Australian MCE forum (now defunct) at one stage, but I think that changed. Original just an on screen EPG collator, it was then integrated into WMC. It never had its own EPG data, because that was not possible for copyright reasons back when it was developed. Whenever I've looked at it I couldn't see the value, as it seemed to just be adding another layer of software to the EPG chain. It is supposed to clean up EPG data though, so may be useful.
But by all means check out if it adds value. My information may be out of date.

That Microsoft document is from July 2009. I doubt it is talking about Rovi data. I can't see any reference to the source.

It is from this post that I thought it might have some potential : http://www.thegreenbutton.tv/forums/viewtopic.php?f=5&t=9052

Not sure if it will work/help. Just playing around.
I suppose I could just manually enter the Season number for each group of recordings.

But if EPG Collector gets me the Season number I'm done.

RoderickGI · « **Reply #31 on:** August 27, 2015, 06:06:13 pm »

The two things I got out of a quick scan of the thread were;
WMC users are having trouble with poor quality EPG data, and trying to use alternate sources. (Which means that MC will have trouble with the data as well.)
Schedules Direct are allowing WMC users to use their data, presumably because WMC is now Abandonware.

I think the people using Big Screen EPG were using it to load WMC with data from their new data source. It is tightly integrated with WMC, and probably works better or is easier than using Microsoft's LoadMXF program.

willrbaar · « **Reply #32 on:** August 30, 2015, 03:51:51 pm »

Has anyone from JRiver talked to Rovi about a license?, or Zap2it? I saw on Rovi's site that they license API's. See: http://developer.rovicorp.com/docs . I don't know what it would cost. It may not be that expensive if they get user data back, or the Rovi logo is used, or who knows what else. Just thought I would ask.

CountryBumkin · « **Reply #33 on:** August 30, 2015, 05:49:34 pm »

JRiver has access to Rovi data now in MC21. You just need to use the "mc2xml" program (selecting "Microsoft" as the source) to access the data.

willrbaar · « **Reply #34 on:** August 31, 2015, 08:20:58 am »

I thought you wanted season number, vice year. Getting the data directly from the source, Rovi, vice an intermediary like mc2xml, may give the JRiver developers some flexibility and a little more control over the data: and who knows, maybe a little bit more data that may be helpful. Just asking if anyone had explored a direct relationship with Rovi, or if it was even feasible?

CountryBumkin · « **Reply #35 on:** August 31, 2015, 11:53:01 am »

Quote from: willrbaar on August 31, 2015, 08:20:58 am

I thought you wanted season number, vice year.

Yes, I would like to get the Season Number from somewhere - but it does not appear that that Season number is in the Rovi data.

I think WMC users are getting the same data as we (MC users) are. Does anyone know if Windows Media Center users are getting Season Numbers? I looked at the Rovi data coming from Microsoft (via the msc2xml program) and there is no Season Number in it - only the Year (and a progID code).

Now I'm trying to use the program XMLTVDB to pull additional data - so far I have not got it working (but I just started working with it). I'm not sure if this program still works, as it is now a couple of years old and development stopped in 2012.

rammingspeed · « **Reply #36 on:** August 31, 2015, 03:32:06 pm »

Quote from: CountryBumkin on August 31, 2015, 11:53:01 am

Now I'm trying to use the program XMLTVDB to pull additional data - so far I have not got it working (but I just started working with it). I'm not sure if this program still works, as it is now a couple of years old and development stopped in 2012.

I got XMLTVDB working and here are a list of things to look out for.
1.) Use Java 7 (6 won't work, didn't try 8 )
2.) Make sure you are using the latest paid version of XMLTVDB. There was a previous release that worked with Rovi, but didn't handle ampersands correctly in the XML.
3.) review the "XMLTVDB.properties" file and change settings correctly. There is documentation in the file
4.) If possible set up EXCLUDED_CATEGORIES as it will reduce the amount of lookups and time to complete.
5.) Consider setting "HTTP_REQUEST_RETRIES=" to a smaller number
6.) If you need to iterate testing, change the timestamp on your XML file by re-saving it. XMLTVDB will pick that up as a new file.

Note that it takes a long time to complete, so be patient once it starts. You'll see lots of errors in the window. As long as it keeps running, everything is working.

CountryBumkin · « **Reply #37 on:** August 31, 2015, 03:46:34 pm »

Quote from: rammingspeed on August 31, 2015, 03:32:06 pm

I got XMLTVDB working and here are a list of things to look out for.
1.) Use Java 7 (6 won't work, didn't try 8 )
2.) Make sure you are using the latest paid version of XMLTVDB. There was a previous release that worked with Rovi, but didn't handle ampersands correctly in the XML.
3.) review the "XMLTVDB.properties" file and change settings correctly. There is documentation in the file
4.) If possible set up EXCLUDED_CATEGORIES as it will reduce the amount of lookups and time to complete.
5.) Consider setting "HTTP_REQUEST_RETRIES=" to a smaller number
6.) If you need to iterate testing, change the timestamp on your XML file by re-saving it. XMLTVDB will pick that up as a new file.

Note that it takes a long time to complete, so be patient once it starts. You'll see lots of errors in the window. As long as it keeps running, everything is working.

I'm working on this now - I think this may be the savior - if it works.

Currently I am not getting any data from XMLTVDB (I need to check all of your suggestions) but it does appear to be correctly noticing the (updating) xmltv.xml file. In the Debug log the last lines say something like "LISTENER: Skipping Event ENTRY_MODIFY. File xmltv.xml". (same message file mc2xml.dat).

I suggest you start a new Thread on setting up and using XMLTVDB (because it's buried in this thread and will probably get missed by many.

CountryBumkin · « **Reply #38 on:** August 31, 2015, 05:02:55 pm »

Quote from: rammingspeed on August 31, 2015, 03:32:06 pm

2.) Make sure you are using the latest paid version of XMLTVDB. There was a previous release that worked with Rovi, but didn't handle ampersands correctly in the XML.

I wasn't aware there was a paid version for XMLTVDB. I need to go back to that website and look.

rammingspeed · « **Reply #39 on:** August 31, 2015, 08:03:15 pm »

Quote from: CountryBumkin on August 31, 2015, 05:02:55 pm

I wasn't aware there was a paid version for XMLTVDB. I need to go back to that website and look.

Sorry head isn't in it today. I meant the paid version of mc2xml.

CountryBumkin · « **Reply #40 on:** September 01, 2015, 05:12:08 am »

Quote from: rammingspeed on August 31, 2015, 08:03:15 pm

Sorry head isn't in it today. I meant the paid version of mc2xml.

I have not been using the "paid" version (ver 1.4) of mc2xml because I have a subscription to Schedules Direct so I am getting good EPG data (just no Season numbers) with mc2xml ver 1.3.

Since mc2xml is creating the XMLTV.xml file and it is updating, and XMLTVDB sees that it is updating/changing, why am I not getting the XMLTVDB file ("xmltv.enhanced.xml") created. It looks like something else is wrong. Perhaps I should post my XMLTVDB Debug Log to see if anyone spots anything wrong in the setup.

The setup in XMLTVDB seems easy enough; set the path to java with arguments provided; set the path to where the xmltv.xml is located (the folder to be monitored); then set the path to the folder where the xmltv.enhanced.xml file is to be outputted.

imeric · « **Reply #41 on:** January 15, 2016, 10:41:39 pm »

Just realized I was missing Season numbers in the guide replaced with the Year in some cases ...Please tell me there is a way around this.

The only ones showing correctly are the ones coming from Sage recordings...

From what I can recall yes zap2xml has season and episode with the -d option but I don't think the description is as complete as Rovi...

It's an old post hopefully there is a Simple workaround

CountryBumkin · « **Reply #42 on:** January 16, 2016, 05:38:52 am »

Quote from: imeric on January 15, 2016, 10:41:39 pm

Just realized I was missing Season numbers in the guide replaced with the Year in some cases ...Please tell me there is a way around this.

The only ones showing correctly are the ones coming from Sage recordings...

From what I can recall yes zap2xml has season and episode with the -d option but I don't think the description is as complete as Rovi...

It's an old post hopefully there is a Simple workaround

Are you in USA? If yes,
The most reliable solution I have found is to run "mc2xml" (version 1.4 or 1.5) as stand-alone (not in MC) then use "XMLTVDB program" to enhance the xmltv.xml data file that mc2xml creates. That creates a new file "xmltv.enhanced.xml" that you import into MC.
OR, what I do now, pay for a subscription to Schedules Direct then run the stand-alone version mc2xml (ver 1.5) and select "Schedules Direct JSON" as you epg data source, then import the xmltv.xml file into MC.
This has been 100% successful in getting me the correct Season and Episode numbers.

imeric · « **Reply #43 on:** January 17, 2016, 07:31:44 am »

Canada.

Isn't it what Carnac is all about? Couldn't it use what Rovi is providing in mc2xml to figure out Season and Episode number?

Oh man back to playing with zap2xml and mc2xml as a separate entity to try to figure this out and compare it with what Sage is giving me...

ANybody esle out there with workarounds?
thx

CountryBumkin · « **Reply #44 on:** January 17, 2016, 12:39:47 pm »

Carnac is the step "after" the TV Show is recorded and imported. If there is a Series Name, Season Number and Episode Number Carnac will look up the meta data info for you. If you don't have the Season Number and Episode Number Carnac won't work.

imeric · « **Reply #45 on:** January 17, 2016, 03:25:09 pm »

Quote from: CountryBumkin on August 27, 2015, 05:11:31 am

Also according to this document https://msdn.microsoft.com/en-us/library/windows/desktop/dd776338.aspx - Season data is in the Rovi data going to WMC users.

Yaobbing if the info is available from Rovi to WMC any way to get it in MC as well?
thx

imeric · « **Reply #46 on:** January 17, 2016, 03:29:50 pm »

I'll check with WMC see what happens and post back.
thx

RoderickGI · « **Reply #47 on:** January 17, 2016, 05:21:04 pm »

Woo. Slow down. A bit of misinformation here.

First, CARNAC does not look up metadata. It happens before metadata is looked up, and uses "filename, length, file type, and other details" to "fill empty database fields". Okay, database fields are metadata, but CARNAC isn't looking them up, just reading them from the file name. It happens when a file is first introduced to MC, during import, and is most useful when importing heavily structured video file names such as; "Marvel's.Agents.Of.S.H.I.E.L.D.3x02.Purpose.In.The.Machine.HDTV.x264-KILLERS.[tvu.org.ru].mp4".

CARNAC is much less useful for recorded TV programs, as the tag information comes from the EPG data, rather than the file name. In fact, MC will use information that it has from the EPG data to decide what to name the file. For example, MC will use the Season and Episode numbers in the file name, if it knows them at the time the file is created.
i.e. "Marvel's Agents of S.H.I.E.L.D - S03E01 - Laws of Nature.jtv"

It is the "Get Movie & TV Info" function that gets additional metadata for the program from TheTVDB, and that happens when the program is imported into MC, even for recorded TV. In fact, I don't know exactly when it happens for recorded TV. It could happen when the file is created, which is when it is "imported" into MC, or it could happen when the recording is finished, but most likely it happen sometime after being imported and before the recording finishes, when MC kicks off the TV maintenance function.

If we use the correct terminology, we might gain a better understanding of what is happening.

Next, since MC21.0.28 MC has been able to look up TheTVDB using just the Series and Episode names, just like EPG Collector has been doing for some time. Try it and see. For a recording that has no TVDB ID or Season or Episode number, but just has Series and Episode Name, run the "Get Movie & TV Info" function, it will return any match it finds (as long as TheTVDB actually knows about the program). It is still a bit crude and can be tricked, but it works. So having Season and Episode numbers in the EPG data is a little less important than it used to be.

Quote

21.0.28 (12/14/2015)
5. Changed: TheTVDB lookup can look up tracks that don't have a season or episode number and will now look up just by the episode name.

See the bolded blue paragraph in this post, and read the following posts.

Now I can only do limited testing of this new functionality, because most of my EPG data now includes Season and Episode numbers supplied by EPG Collector. So testing it would be great, and if it isn't working properly, let JRiver know.

Quote from: CountryBumkin on August 27, 2015, 05:11:31 am

Also according to this document https://msdn.microsoft.com/en-us/library/windows/desktop/dd776338.aspx - Season data is in the Rovi data going to WMC users.

Quote from: RoderickGI on August 27, 2015, 06:06:14 am

That Microsoft document is from July 2009. I doubt it is talking about Rovi data. I can't see any reference to the source.

That document refers to when Microsoft used Gracenote data, not Rovi data. Microsoft changed from using Gracenote to Rovi data in June/July 2015. The only test as to whether WMC users get Season and Episode numbers in Rovi data now is to run WMC and see, which you are doing. From what I have read, lots of WMC users aren't happy with Rovi data. Please let us know what you find.

Note that the additional Season and Episode number data is available from PerData, but as far as I know it requires additional functionality in MC to get it, which hasn't been implemented yet.

Yaobing · « **Reply #48 on:** January 17, 2016, 06:28:19 pm »

It may be true that Rovi provides season and episode numbers, but mc2xml does not get them for North America users. So we are currently stuck with this limitation. CountryBumkin provided a pretty good workaround, but it will only work if you use XMLTV option in MC, instead of the integrated mc2xml option (i.e. running mc2xml or any other tools, then use "XMLTVDB program" to enhance the xmltv.xml data file before importing the xml file into MC).

imeric · « **Reply #49 on:** January 17, 2016, 09:52:14 pm »

I pretty much spent the whole evening trying to set this up with no success as per this post: https://yabb.jriver.com/interact/index.php?topic=99845.msg712194#msg712194

I get this error and no enhanced files get created...

Code: [Select]

01/17/2016 10:44:37 PM ERROR   Error processing xml file: org.jdom.input.JDOMParseException: Error on line 304 of document file:/M:/XMLTVDB%20v1.0.1/xmltvs/xmltv.xml: The reference to entity "w" must end with the ';' delimiter.
                                   org.jdom.input.JDOMParseException: Error on line 304 of document file:/M:/XMLTVDB%20v1.0.1/xmltvs/xmltv.xml: The reference to entity "w" must end with the ';' delimiter.
                                   org.jdom.input.SAXBuilder.build(SAXBuilder.java:530)
                                   org.jdom.input.SAXBuilder.build(SAXBuilder.java:905)
                                   org.jdom.input.SAXBuilder.build(SAXBuilder.java:884)
                                   xmltvdb.xmltvdb.processXMLTVFile(xmltvdb.java:301)
                                   xmltvdb.xmltvdb.access$100(xmltvdb.java:67)
                                   xmltvdb.xmltvdb$1.onEvent(xmltvdb.java:178)
                                   btv.folderwatcher.FolderWatcher.run(FolderWatchers.java:79)
                                   java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                                   java.util.concurrent.FutureTask.run(Unknown Source)
                                   java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                                   java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                                   java.lang.Thread.run(Unknown Source)

Any help from anyone on this would be greatly appreciated!

Thx for your answer Yaobbing.

INTERACT FORUM

Author Topic: Is everyone getting the "Year" where Season "number" should be? (Read 28163 times)