Topic: NEW: PercData EPG data source (Read 31185 times)

CountryBumkin · « **on:** October 12, 2015, 05:40:15 pm »

PercData.com is working well too. I'm testing it now using the free 7 day trial. The setup and installation was very smooth.

I expect that the data will be the same as I get now with my Schedules Direct subscription (since it is the same data source). But I hope it is better and provides Season and Episode Numbers.

It will take a couple of days to test and compare data - but it's nice to have the "free" (mc2xml) choice and this new "subscription based" data source. Thanks

glynor · « **Reply #1 on:** October 12, 2015, 05:44:55 pm »

Quote from: CountryBumkin on October 12, 2015, 05:40:15 pm

PercData.com is working well too. I'm testing it now using the free 7 day trial. The setup and installation was very smooth.

I've been off the grid for a while here, but this looks very, very promising! Sweet!

RoderickGI · « **Reply #2 on:** October 12, 2015, 05:56:17 pm »

So this little item jumps out at me:

21.0.14 (10/12/2015)
4. NEW: PercData.com EPG data service is integrated in MC.

PercData.com seems to only support US and Canada at the moment. However, the data is sourced from Gracenote, which now also operates in Australia, having bought the company that handles EPG collation in Australia, HWW.

So who do we start asking about support for Australian EPG data via PercData.com? JRiver, PercData.com, or Gracenote?

glynor · « **Reply #3 on:** October 12, 2015, 06:00:28 pm »

I merged a couple of posts from the Build thread over to this one, so it can eventually be moved over to the public board or whatever.

Hendrik · « **Reply #4 on:** October 12, 2015, 06:03:05 pm »

Quote from: RoderickGI on October 12, 2015, 05:56:17 pm

So who do we start asking about support for Australian EPG data via PercData.com? JRiver, PercData.com, or Gracenote?

The first thing that needs to happen is PercData starting to offer Australian EPG data. Once that happens, MC can use it. So best would be to ask them, I suppose.

On another note, why are websites of EPG companies always straight out of the last decade?

RoderickGI · « **Reply #5 on:** October 12, 2015, 06:28:05 pm »

Quote from: Hendrik on October 12, 2015, 06:03:05 pm

On another note, why are websites of EPG companies always straight out of the last decade?

Because they are almost always created by retired old IT men who have little to do and want to earn a little money on the side, or have a hobby to fill their time.

They think they are creating up to date and highly useable web sites.

Plus the whole broadcast TV industry is still living in the 1970's, based on their stubborn adherence to their revenue model.

Thanks for merging this Glynor. I was hoping someone would, once I saw CountryBumkin's post. I did create the separate thread so that it could be moved later.

Yaobing · « **Reply #6 on:** October 12, 2015, 07:07:10 pm »

I just realized, there is a bug in my PercData code. The time is not converted to local time. So all programming times are wrong.

Yaobing · « **Reply #7 on:** October 12, 2015, 07:36:39 pm »

PercData is a commercial version of Schedules Direct. It allows its data to be used in approved commercial products, while its twin, Schedules Direct, only allows non-commercial products.

RoderickGI · « **Reply #8 on:** October 12, 2015, 09:19:28 pm »

Quote from: Yaobing on October 12, 2015, 07:36:39 pm

PercData is a commercial version of ScheduleDirect. It allows its data to be used in approved commercial products, while its twin, ScheduleDirect, only allows non-commercial products.

I am very interested to hear about the quality of the EPG data provided. If it is better than the OTA data I use now, and PercData are willing and able to provide Australian EPG data, since they own the business now, I would probably convert over.

If the data still has Series and Episode numbering, and other data shortcomings then I may not be as interested.

astromo · « **Reply #9 on:** October 12, 2015, 10:04:43 pm »

Quote from: Hendrik on October 12, 2015, 06:03:05 pm

The first thing that needs to happen is PercData starting to offer Australian EPG data. Once that happens, MC can use it. So best would be to ask them, I suppose.

On another note, why are websites of EPG companies always straight out of the last decade?

Fair point. Question asked:

Quote

I see that gracenote (whose data you work with) has acquired HWW and therefore has a presence in Australia for EPG capability:
http://www.gracenote.com/gracenote-acquires-hww-expands-global-reach-into-australia/

In a beta build of JRiver MC21 (I'm on the beta team), the capability has been included to link into percdata but your EPG data is limited to the US/Canada.

Any plans to extend your capability to include Australia?

Thanks

If I get a response, I'll share what comes back.

astromo · « **Reply #10 on:** October 13, 2015, 01:10:34 am »

Here's the response, just in, hot off the internet:

Quote

Robert Eden <admin@percdata.com>
16:00 to me
On 10/12/2015 10:02 PM, Iain wrote:
Quote
I see that gracenote (whose data you work with) has acquired HWW and
therefore has a presence in Australia for EPG capability:
http://www.gracenote.com/gracenote-acquires-hww-expands-global-reach-into-australia/

In a beta build of JRiver MC21 (I'm on the beta team), the capability
has been included to link into percdata but your EPG data is limited
to the US/Canada.

Any plans to extend your capability to include Australia?

No plans at the moment.

From what I understand about the Australian data, it's in a very different format and no where close to the quality we're used to.

If there were a lot of interest, we'd certainly approach Gracenote and see if it was financially viable.

Robert

astromo · « **Reply #11 on:** October 13, 2015, 01:47:39 am »

My further reply:

Quote

Robert

Thanks for the quick reply. Please bear in mind that I'm an end user (in my case of JRiver Media Centre as a personal video recorder using a digital TV tuner).

If you're not across this detail, the state of Australian EPG provision is in flux at the moment. Are you aware of the financial position of the current provider (and I believe there's only one) that offers EPG services to individual users for a fee?
https://www.icetv.com.au/
https://www.icetv.com.au/administrator_update.html
http://forum.icetv.com.au/iceforum/general-discussions/1/elephant-in-the-room/4644/
http://forum.icetv.com.au/iceforum/general/44/icetv-in-voluntary-administarion/4638/

IceTV changed the Australian TV landscape by challenging the established broadcasters successfully:
http://www.smh.com.au/news/home-theatre/nine-loses-electronic-program-guide-case/2007/08/09/1186530493565.html
Apparently their business model is along the lines of using staff to curate their published EPG:

Quote
IceTV has always maintained its EPG did not infringe Nine copyright because it used publicly-available information and wrote its own program descriptions.

I suspect that gracenote's data would largely avoid a labour intensive overhead and thereby offer improved profit margin.

The above may not suit your business plans but I can point to this as a potential user base, depending on how the administrators handle matters going forward. With the ongoing demise of Windows Media Centre as a result of Windows 10, proponents like JRiver are looking to fill the vacuum. Presumably, this is part of the reason that JRiver are beta-testing a link to percdata.

I leave it to you to assess market potential in Australia but I would point out that EPG data quality has a value if it's delivery is timely, accurate, reliable and has quality meta data (such that it aligns with public meta-data databases such as https://www.themoviedb.org/ and http://thetvdb.com/).

Thanks

If percdata can reach out as far as Australia, it may inspire them to explore other markets.

Who knows? Asking the question is always the first step.

RoderickGI · « **Reply #12 on:** October 13, 2015, 04:17:46 am »

I'll add my voice to the discussion with them tomorrow.

CountryBumkin · « **Reply #13 on:** October 13, 2015, 11:11:03 am »

Quote from: Yaobing on October 12, 2015, 07:07:10 pm

I just realized, there is a bug in my PercData code. The time is not converted to local time. So all programming times are wrong.

Any chance of getting this fixed and releasing a new beta version before my PercData trial runs out - in 6 days?

However, I checked the two recordings made last night with PercData and I see the "programming time error" now, but at least the EPG data is coming in correctly for the Shows I scheduled so I can still test/compare the data.

Yaobing · « **Reply #14 on:** October 13, 2015, 11:56:41 am »

The bug I saw was that MC was using UTC. So everything was shifted a few hours (for example PBS News Hour at 6:00 pm was place at 11:00 pm).

The bug is fixed. I do not know when we will make another build though.

RoderickGI · « **Reply #15 on:** October 13, 2015, 03:28:36 pm »

Can you tell what the quality of the data is like though CountryBumkin?

Does it have season and episode numbers? Categories? Actor/Director/etc.?

RoderickGI · « **Reply #16 on:** October 13, 2015, 04:13:22 pm »

My voice has been added.

Quote

Australian EPG data
G'Day folk.

I'm a user of JRiver Media Center, and I'm very pleased to see that you have added JRiver's software to your list of approved devices.

I'm in Australia, and as you may know we have long had difficulty in getting any decent and reliable EPG data to use with our devices and software. You may also be aware that it looks like our one commercial provider of EPG data to end users, IceTV, is probably about to close shop, and is currently under Administration to see if it can be saved.

Since Gracenote bought HWW, the only EPG data consolidator in Australia, last year and as your use Gracenote as you source, I am adding my voice to the request that you provide Australian EPG data via your service.

If IceTV does fail completely there will be a lot of users looking for an alternate source of EPG data. The only alternatives here are the Over The Air EIT and MHEG-5 data, which isn't very good or reliable, and a complex Linux based solution. Your service could possibly become the one reliable solution for Australia.

I understand that you have received a similar request recently, and I'm sure you will receive more once word gets around. Is there anything we can do to make this happen, perhaps by talking to Gracenote Australia? Of course we will be promoting you as a potential solution once the current Beta implementation goes public.

Thanks for your interest.
Rod

Yaobing · « **Reply #17 on:** October 13, 2015, 06:04:26 pm »

Quote from: RoderickGI on October 13, 2015, 03:28:36 pm

Can you tell what the quality of the data is like though CountryBumkin?

Does it have season and episode numbers? Categories? Actor/Director/etc.?

No season and episode numbers. There is only one value that is possibly related, syndicatedEpisodeNumber, but I can not make sense of it, so I did not map it to anything. The spec says "Optional. Distributor designated number corresponding to an episode of a specific show."

Categories, yes. They are called "program genres", and I am treating them the same as categories in XMLTV.

Actor/director/guest star/writer/etc. Yes. In MC, actors and guest stars are put in Actors field. Director and writer are in their own fields. Producer and executive producer are in Producer field. We have no "Host" tag. So it is dumped in "Credits".

jmone · « **Reply #18 on:** October 13, 2015, 07:13:03 pm »

Is it worth someone across the details about lobbing Perc for Oz (eg not me) posting in the Whirlpool ICE TV thread? It already has almost 500 posts of users wanting an alternative.

RoderickGI · « **Reply #19 on:** October 13, 2015, 11:23:07 pm »

Quote from: jmone on October 13, 2015, 07:13:03 pm

Is it worth someone across the details about lobbing Perc for Oz (eg not me) posting in the Whirlpool ICE TV thread? It already has almost 500 posts of users wanting an alternative.

I am still reading through the extensive IceTV threads on Whirlpool.

I may start a PercData thread, but it would be one of many, and may get lost. Also, as PercData doesn't have a solution now, and even if it did, it would just be EPG data and not the extended IceTV functionality. I'm not sure there would be a lot of interest. Maybe when the new recording rules functionality is delivered, we could sell the idea of MC plus PercData being a WMC replacement, and possibly other PVRs. But a lot of people want to keep there existing hardware.

Plus, the EPG data that Gracenote would provide to PercData would probably be similar to the FTA EPG data, since they both have the same source. That isn't very good data.
Plus, PercData will only provide EPG data to approved devices, probably due to its agreement with Gracenote. Not many Australian users are going to be using one of those devices.
Plus, it seems unlikely that PercData could provide EPG data in an open XMLTV format, which is what many users would need. Their agreement with Gracenote is unlikely to allow it.

Basically, the whole IceTV situation is a mess, but I don't think PercData providing a locked up EPG is a complete answer.

RoderickGI · « **Reply #20 on:** October 15, 2015, 05:56:14 am »

Some more information about the EPG situation in Australia.

I have been doing some more research and came across an iOS App developed by Gracenote for Australia. The App is OzTV.
EDIT: It actually wasn't created by Gracenote, but by a couple of Australian guys (OzTV Apps Pty Ltd & Apps Perhaps), who seem to have sold it to eBroadcast. It is using data from Gracenote though.

Looking at the images on the above page, I noticed that the App included Repeat Flags, Season and Episode numbers, Major Category, Year, Country of Origin, Artwork, a good Description and so on.

So I downloaded the App, which is free, when I had an iOS 8 device available tonight, and compared my OTA broadcast EPG data in MC to the data in the App. Indeed, for programs that had no Season or Episode numbers, or Repeat Flags, or any of the other data, it was all there in the App. So the OTA EPG, which is sourced from Gracenote, is not being provided with the valuable metadata, but rather just the basic program data.

So I have to ask why they would do that, now that the company is not owned by the broadcasters? The only real reason I can think of, since they are providing the data for free in the App, with minimal advertising, is that they don't want the data to be available to everybody in a copyable form, which it is in the OTA data stream, via XMLTV files.

So I'm wondering, is PercData being given the same limited EPG data deliberately as well, for the same reasons? If so, why, if only approved devices can use it, and end users can't see the data? Maybe PercData could obtain and provide better data, including the extra metadata, if Gracenote could be convinced that their data would be protected.

Then if PercData can't get the extra metadata, then maybe JRiver can, since you guys can guarantee the security of their data, through obfuscation, encryption, or whatever means you use with PercData now.

BTW, I had a response back from Robert Eden of PercData, who is also the President and Treasurer of Schedules Direct, regarding providing Australian EPG, which was;

"Noted... I'm not sure how much development work would be needed to supported it, and how much time I have to set it up... that's assuming I can even get the data from Gracenote for a reasonable price."

It doesn't sound like PercData is his primary focus.

Finally, Jim and Yaobing, download a copy of the OzTV App and check it out. Set your location as Melbourne and you will see the same data I did. Even if none of the above is of interest, the App is a very good implementation of an EPG Guide, at least on an iPad (iOS 8 required). I really liked some of the capabilities, which could be included in MC, although use of a remote or keyboard and mouse would need to be considered.

CountryBumkin · « **Reply #21 on:** October 15, 2015, 07:56:06 am »

I recorded two days worth of TV shows using PercData.

As you can see in the screenshot, there are Season and Episode Number metadata being entered/imported in MC, however it (appears that it) is the wrong Season and Episode Numbers.

If I use the "Get Movie and TV Info" tool for these Shows, the titles and descriptions based on Season and Episode from TVDb do not match the Name and Description shown.

It is odd that the "Two and a Half Men" series recorded back-to-back have different Season and Episode formats.

Since these Shows were recorded using version 21.0.14 the actual recordings are whatever was showing at the UTC time, so I can't confirm if the show's Name and Description actually match what would have been recorded with the correct time.

I have more recordings scheduled for tonight and Friday, now using version 21.0.15. it would be nice if the xmltv.xml file did not have to get deleted in the import process so one could examine it for accuracy.

imugli · « **Reply #22 on:** October 15, 2015, 06:02:27 pm »

Quote from: Yaobing on October 13, 2015, 06:04:26 pm

No season and episode numbers. There is only one value that is possibly related, syndicatedEpisodeNumber, but I can not make sense of it, so I did not map it to anything. The spec says "Optional. Distributor designated number corresponding to an episode of a specific show."

Categories, yes. They are called "program genres", and I am treating them the same as categories in XMLTV.

Actor/director/guest star/writer/etc. Yes. In MC, actors and guest stars are put in Actors field. Director and writer are in their own fields. Producer and executive producer are in Producer field. We have no "Host" tag. So it is dumped in "Credits".

I found this https://github.com/SchedulesDirect/JSON-Service/wiki/Home-Schedules-Direct-API-20130311 which may be of some use. Seems (for schedulesdirect data at least) the season and episode # data can be found by calling the metadata field.

And I've also sent an email to PercData expressing an interest in signing up if they can make Australian data happen.

Yaobing · « **Reply #23 on:** October 15, 2015, 10:16:07 pm »

Quote from: CountryBumkin on October 15, 2015, 07:56:06 am

it would be nice if the xmltv.xml file did not have to get deleted in the import process so one could examine it for accuracy.

I made some change today that will leave xmltv file from mc2xml undeleted until MC restarts. As for PercData, the format is not the same. There is an xml file that you can examine, it is just not in the xmltv format. The file is cached in MC's Settings folder (if you can find the Temp folder, you can find Settings folder

Yaobing · « **Reply #24 on:** October 16, 2015, 09:26:29 am »

Quote from: CountryBumkin on October 15, 2015, 07:56:06 am

I recorded two days worth of TV shows using PercData.

As you can see in the screenshot, there are Season and Episode Number metadata being entered/imported in MC, however it (appears that it) is the wrong Season and Episode Numbers.

Season and episode numbers are not imported as part of EPG download. Did you set MC up to get the info elsewhere? A program ID contains series ID and episode ID, and I am not sure how I can use them.

glynor · « **Reply #25 on:** October 16, 2015, 10:41:29 am »

Quote from: Yaobing on October 16, 2015, 09:26:29 am

Season and episode numbers are not imported as part of EPG download. Did you set MC up to get the info elsewhere? A program ID contains series ID and episode ID, and I am not sure how I can use them.

How you use them is documented in the GitHub article linked above, at least with SchedulesDirect. But, since they're the same outfit, I'd think you could do the same with PercData.

You have to download each one individually, though, per ProgramID. It looks like they tie it directly to tvdb episode data though, which is perfect.

CountryBumkin · « **Reply #26 on:** October 16, 2015, 11:14:10 am »

Quote from: Yaobing on October 16, 2015, 09:26:29 am

Season and episode numbers are not imported as part of EPG download. Did you set MC up to get the info elsewhere? A program ID contains series ID and episode ID, and I am not sure how I can use them.

I'm only using PercData for testing right now.
I don't know where the Season and Episode numbers are coming from. The Shows I recorded last night (Oct 15th with MC version 21.015) also have Season and Episode numbers (and some of it it is correct).

Possibly there is some "cross contamination" happening.
I am also downloading data using "Zap2xml" and "Mc2xml ver1.3 with my Schedules Direct subscription" which I then run through "XMLTVDB" program to get Season and Episode numbers".

However, I only have MC setup to use PercData which is coming from the "JRiver folder" - whereas the other EPG data I'm getting is in a different folder and nothing in MC points to that data/folder.

RoderickGI · « **Reply #27 on:** October 16, 2015, 06:21:39 pm »

Quote from: glynor on October 16, 2015, 10:41:29 am

It looks like they tie it directly to tvdb episode data though, which is perfect.

It is all there, isn't it.

Quote

[{"dataSource":"thetvdb","episode":"5","seriesID":"73940","season":"4"}],"programID":"EP000000060020","syndicatedEpisodeNumber":"405","alternateSyndicatedEpisodeNumber":"27","sourceType":"Syndicated","colorCode":"Color","md5":"b+dPVPEDsECNil0kXrCzaA","episodeTitle150":"The Sausages in the Trousers"}

Although it looks like the latest API requires an extra step to get the metadata.

Quote

NOTE: As of 2013-02-21, the metadata field will be removed in the next API; clients will need to run the "get"->"metadata" function to download metadata.

It also looks like the latest/current JSON API is this one: https://github.com/SchedulesDirect/JSON-Service/wiki/API-20141201

imugli · « **Reply #28 on:** October 16, 2015, 07:05:10 pm »

Quote from: RoderickGI on October 16, 2015, 06:21:39 pm

It is all there, isn't it.
Although it looks like the latest API requires an extra step to get the metadata.
It also looks like the latest/current JSON API is this one: https://github.com/SchedulesDirect/JSON-Service/wiki/API-20141201

Re Extra step, that looks correct. But I think what YB said in his post is that the PercData is provided in a different format than the SD API provides it in, so the SD API may not apply.

RoderickGI · « **Reply #29 on:** October 16, 2015, 07:37:04 pm »

SD provide data two ways; via a JSON API and via an XML API. I suspect that the XMLTV.org executable still uses the XML API, and it produces XMLTV files.

I have to assume that PercData uses the same APIs, based on what I have read, and that it is really just a legal entity for licencing the data, and to control who can use the data, through "approved devices". So the XML file format may be different, but the source data is the same. There seems to be a lot of flexibility in what data is collected from SD through the JSON API at least, and probably through the XML API.

Yaobing · « **Reply #30 on:** October 16, 2015, 07:42:53 pm »

PercData and SD should have the same data. I use a SOAP request method that downloads an xml file (not in XMLTV format, that was what I meant). I will have to figure out whether I can use separate JSON requests for additional metadata after downloading the xml.

glynor · « **Reply #31 on:** October 16, 2015, 07:59:22 pm »

Quote from: Yaobing on October 16, 2015, 07:42:53 pm

I will have to figure out whether I can use separate JSON requests for additional metadata after downloading the xml.

That's exactly what you have to do. You need to submit a list of ProgramIDs and then it'll have you download a ZIP file containing entries for that ProgramID. However, you're supposed to cache the results, so you don't re-download details for the same ProgramIDs over and over (since the same episode will likely appear in a schedule multiple times, for multiple airings and whatnot).

Quote

Or, the same program is played multiple times in a 2 week period over multiple stationIDs. It will still be the same program, with the same MD5 hash. Your application would only need to download it once as part of a batch request you send to the server for programIDs that you require. If it's different, then there may be new guest stars, or a late-breaking program update has occurred, so your client can include the programID in the batch request of new / updated programIDs.

Quote

Determine which programID's need to be downloaded. During an initial download, the client will have an empty cache of programID's / MD5 hashes, so the client will need to download all relevant programID's.

NOTE: the server has been successfully tested with over 30,000 programID requests, but the response may overload the client. Therefore there is a hard-coded limit on the server; you may only request 5000 programIDs, schedules, or schedule MD5's per request. (But you may issue multiple requests if you need more than 5000 pieces of information from the server.) There is a 10-minute timeout on the server; if the response to your request can not be sent to you within 10 minutes, it will terminate at the 10 minute point. So, do not request 5000 programIDs if you are on a 33.6Kbps dial-up.

Quote

Parse the schedule, determine if the MD5 of the program for a particular timeslot has changed. If the programID for a timeslot is the same, but the MD5 has changed, this means that some sort of metadata for that program has been updated.
Request the "delta" programID's as determined through the MD5 values

Requesting only stationID's and programID's that the user is interested in will minimize the time and data required in each download. Downloading only schedules and programs that are different than what you have already downloaded will minimize the download time and processing your client must perform.

And, then... The meaty part:
https://github.com/SchedulesDirect/JSON-Service/wiki/API-20141201#download-program-information

Quote

Download program information

POST https://json.schedulesdirect.org/20141201/programs

Token: Required

The body of the request will have the programIDs you are requesting.

Your client must send an Accept-Encoding that has "deflate,gzip" in it, even though the response will be gzip'ed. This is due to an implementation bug in 20140530 which will be fixed in 20141201.

NOTE: No more than 5000 programIDs in a single request.

The response contains Season and Episode numbers. It contains a bunch of other stuff too, but honestly... You could just disregard all of that and pull it from TVDB directly then. Once you have this, that is:

Code: [Select]

    "metadata": [
        {
            "Gracenote": {
                "season": 5,
                "episode": 18
            }
        }
    ]

glynor · « **Reply #32 on:** October 16, 2015, 08:02:42 pm »

It also looks like, for the record, they have very nice metadata for sporting events:

Code: [Select]

{
    "programID": "EP005544683862",
    "titles": [
        {
            "title120": "MLB Baseball"
        }
    ],
    "eventDetails": {
        "venue100": "Busch Stadium",
        "teams": [
            {
                "name": "St. Louis Cardinals",
                "isHome": true
            },
            {
                "name": "Chicago Cubs"
            }
        ],
        "gameDate": "2015-10-10"
    },
    "descriptions": {
        "description1000": [
            {
                "descriptionLanguage": "en",
                "description": "The Cubs, led by young sluggers Anthony Rizzo and Kris Bryant, visit Matt Carpenter and the Cardinals in Game 2 of the National League Division Series."
            }
        ],
        "description100": [
            {
                "descriptionLanguage": "en",
                "description": "From Busch Stadium."
            }
        ]
    },
    "genres": [
        "Playoff sports",
        "Baseball"
    ],
    "episodeTitle150": "Chicago Cubs at St. Louis Cardinals",
    "entityType": "Sports",
    "showType": "Sports event",
    "hasImageArtwork": true,
    "md5": "FSMo+rHrKUc7460HLy5E3g"
}

More:
https://github.com/SchedulesDirect/JSON-Service/wiki/API-20141201-Program-Response

RoderickGI · « **Reply #33 on:** October 16, 2015, 08:59:29 pm »

Quote from: glynor on October 16, 2015, 07:59:22 pm

It contains a bunch of other stuff too, but honestly... You could just disregard all of that and pull it from TVDB directly then.

Oooo, no. Please don't disregard the extra data. In order for recording rules to be as flexible as they can be with the new functionality you are planning, MC needs to have the extra metadata before the recording rules are evaluated.

Okay, you could just get the schedule of programs, and then get the extra data from TheTVDB, but why do it that way? MC doesn't currently take whatever EPG data is available and enhance it with lookups from TheTVDB. If doing so was so easy, why hasn't it already been done. In fact, why isn't MC looking up the Season and Episode numbers from TheTVDB right now, instead of being reliant on those Season and Episode numbers to get any metadata from TheTVDB.

No. Get the metadata from PercData with the schedule, while there is no doubt that the data belongs to the program on the schedule. That will allow recording rules to specifiy recording TV and movies by Director or Actor, or Genre, or any of the other data that is available from SD. Plus make sure you get the Series ID (well at least one of the two, if not both) as if that data is always available in the EPG, then proper Series Recording can be implemented in MC, based on the Series ID and not a dubious match on the program name.

The caching of data will be important, as I've mentioned before when discussing EPG Collector's capabilities, though the PercData/SD requirements are far greater, and the features more extensive.

Of course, none of this effects me, yet, since there is no Australian data from PercData. But understanding the requirement, and having the capability, could make a big difference for me in the future.

CountryBumkin · « **Reply #34 on:** October 19, 2015, 10:05:21 am »

Last update on testing PercData:

I have had no issues with PercData during my testing (my free test subscription expires today so this will end my review). The recordings shown below were for scheduled shows over the weekend (10/17 and 10/18).

The data is as good as I get with my "paid subscription" using Schedules Direct (SD) with mc2xml ver1.3. Since this is built-in with mc2xml ver1.4 (and SD is not available to JRiver users), this is a great EPG data source.

As I noted earlier, I am getting Season and Episode Numbers for most of the recordings. Oddly, thay are mostly accurate. However, Yaobing indicated that this data was/should not be coming from PrecData - so this is a mystery. Also, notice in the screenshot that I got two recordings of the same episode of "Two and a Half Men", but the Episode numbers are different. Perhaps it would be best not to fill in the Season and Episode numbers at all if the data is going to be bad.

Last odd thing (not really related to PercData) is that for some reason the "media sub type" is being entered as "Educational". I don't know why this is happing since I have my Import Rules set to add media sub type = TV Show for all recordings to the D:\Video... folder where these recordings are saved.

JimH · « **Reply #35 on:** October 19, 2015, 10:27:34 am »

Thanks for the report.

CountryBumkin · « **Reply #36 on:** October 20, 2015, 07:47:26 am »

Quote from: Yaobing on October 15, 2015, 10:16:07 pm

I made some change today that will leave xmltv file from mc2xml undeleted until MC restarts. As for PercData, the format is not the same. There is an xml file that you can examine, it is just not in the xmltv format. The file is cached in MC's Settings folder (if you can find the Temp folder, you can find Settings folder

Yaobing, I'm not seeing the xml file. I am looking at the "C:\Users\username\AppData\Roaming
J River\media center 21\Temp\Settings" folder, but I only see a ".dat" file and some ".ini" files, no "xml" file. Can you check your folder and confirm that the xml file is there?

Also, you stated that the xml file would remain until MC restarts. Since I hardly ever restart MC, will the older xml file just be overridden by a new xml file every day when MC downloads the new EPG data?

Thanks

Yaobing · « **Reply #37 on:** October 20, 2015, 07:50:14 am »

Quote from: CountryBumkin on October 20, 2015, 07:47:26 am

Yaobing, I'm not seeing the xml file. I am looking at the "C:\Users\username\AppData\Roaming
J River\media center 21\Temp\Settings" folder, but I only see a ".dat" file and some ".ini" files, no "xml" file. Can you check your folder and confirm that the xml file is there?

Also, you stated that the xml file would remain until MC restarts. Since I hardly ever restart MC, will the older xml file just be overridden by a new xml file every day when MC downloads the new EPG data?

Thanks

Sorry about that. The Settings folder I was referring to is parallel with Temp folder, not as a sub-folder. "C:\Users\username\AppData\Roaming
J River\media center 21\Settings\PercData\"

The PercData cache is not in Temp file, but the mc2xml data file is. The former will be cleaned out automatically. The latter is cleaned out upon restart of MC. We previously delete the file immediately. Now we leave it in the Temp folder. The files do not override each other because their names are different. So, yeah it is a problem in your case.

RoderickGI · « **Reply #38 on:** October 20, 2015, 06:04:14 pm »

Not such a bad thing to accumulate XMLTV files if you are chasing EPG errors.

Maybe an enhancement for the future would be being able to specify how many XMLTV files are retained before deletion. That may be going too far though.

But for the average user who may rarely restart MC, filling the hard disk, which is possibly a small SSD, with XMLTV files is probably not a good thing. Maybe if the files were in a user specified location this wouldn't be such an issue, as users would be made aware of the location and could find the files easily.

Yaobing · « **Reply #39 on:** October 20, 2015, 06:38:08 pm »

Yeah, I will need to add a clean up routine.

glynor · « **Reply #40 on:** November 15, 2015, 12:26:18 pm »

Bump. Where do we stand on this?

Yaobing · « **Reply #41 on:** November 15, 2015, 04:03:40 pm »

Quote from: glynor on November 15, 2015, 12:26:18 pm

Bump. Where do we stand on this?

On what?

The clean up of xml files? It was done in build 18. Other things? Not done yet.

glynor · « **Reply #42 on:** November 15, 2015, 07:05:22 pm »

Quote from: Yaobing on November 15, 2015, 04:03:40 pm

On what?

Mostly Season/Episode numbers. I really want to switch to MC for my DVR needs. The setup finally works pretty reliably for me, but I really don't want to give up automatic metadata inside MC from TVDB.

This works quite well now with my setup with SageTV, though. I don't actually use Sage for anything but running in the background recording shows, and it is annoying every time I have to touch it, but my shows come into MC fully tagged without intervention.

That's my standard. I think, frankly, that's most people's standard.

muzicman0 · « **Reply #43 on:** November 16, 2015, 09:08:25 am »

If you are OK using third party java apps, you could use XMLTVDB...I used it for a few months with no issues at all...I just ran it on the downloaded XML file, which added season and episode info, and then imported. The only reason I don't still do it, is because my XML provider (I don't use the built in Rovi) adds the info now for me.

I just created some tasks in the Winodws scheduler that would run the various things at certain times, and for months, it went without a hitch. I am happy to provide the tasks in xml format if anyone wants them.

BUT, I don't believe percdata has the season and episode info in it...could be wrong though. It's been a while since I saw any comments on it.

glynor · « **Reply #44 on:** November 16, 2015, 09:32:00 am »

Quote from: muzicman0 on November 16, 2015, 09:08:25 am

BUT, I don't believe percdata has the season and episode info in it...could be wrong though. It's been a while since I saw any comments on it.

It does, as is discussed at length above. You have to use a separate query to resolve the EpisodeIDs into TVDB-style Season/Episode numbers.

muzicman0 · « **Reply #45 on:** November 16, 2015, 09:37:19 am »

Right...poor wording on my part, I guess I meant that it doesn't have it in a way that MC currently understands...or at least that was my understanding...I'm not really in the know at all, I just really wanted season and episode numbers like you, so I resorted to using a 3rd party app to get it.

CountryBumkin · « **Reply #46 on:** November 16, 2015, 09:48:33 am »

If your willing to pay for a subscription to PercData, you might as well just get the subscription for Schedules Direct. Then you can use the JRiver built-in mc2xml program (using ver 1.5) and select the new "SD JSON" feature ("-J") and you will get all the correct Season and Episode numbers. http://mc2xml.awardspace.info/#SDJSON

CORRECTION: You can't use the MC built-in mc2xml because MC uses version 1.4.
To use the "Schedules Direct JSON, you need to use mc2xml version 1.5 (then you import the EPG data into MC by selecting use "XMLTV" data.

muzicman0 · « **Reply #47 on:** November 16, 2015, 10:32:43 am »

Yep...that's what I am doing now.

glynor · « **Reply #48 on:** November 16, 2015, 10:38:18 am »

Quote from: CountryBumkin on November 16, 2015, 09:48:33 am

If your willing to pay for a subscription to PercData, you might as well just get the subscription for Schedules Direct. Then you can use the JRiver built-in mc2xml program (using ver 1.5) and select the new "SD JSON" feature ("-J") and you will get all the correct Season and Episode numbers. http://mc2xml.awardspace.info/#SDJSON

That's great info, thanks.

However, that is, of course, violating the license for SD. If it works with SD, then it would work equally well with PercData (which has the exact same API and data). So, MC needs to be fixed to be able to use this natively for PercData, IMHO.

codpiece · « **Reply #49 on:** June 28, 2016, 04:38:47 pm »

Quote from: glynor on November 16, 2015, 10:38:18 am

That's great info, thanks.

However, that is, of course, violating the license for SD. If it works with SD, then it would work equally well with PercData (which has the exact same API and data). So, MC needs to be fixed to be able to use this natively for PercData, IMHO.

Bump.

Could we update the built-in mc2xml program version, please?

INTERACT FORUM

Author Topic: NEW: PercData EPG data source (Read 31185 times)