INTERACT FORUM

More => Old Versions => JRiver Media Center 23 for Windows => Topic started by: Headcool on October 02, 2017, 12:49:30 pm

Title: TheTVDB API change
Post by: Headcool on October 02, 2017, 12:49:30 pm: I noticed yesterday, that the scraping of episode thumbnails did not work anymore. I investigated a little bit, and it seems that thetvdb.com planned to discontinued (https://forums.thetvdb.com/viewtopic.php?f=3&t=36557) their APIv1 with the 1. October 2017.
So I guess the new API is not implemented in MC.

I hope this issue can be fixed soon. Note that this may be the ideal opportunity to switch to themoviedb.org (which is already used for movies) instead of thetvdb.com.
Title: Re: TheTVDB API change
Post by: Matt on October 02, 2017, 12:50:37 pm: It was done for MC22 in May.

22.0.107 (5/16/2017)

Changed: Updated the TheTVDB integration to the v2 TheTVDB API.
Title: Re: TheTVDB API change
Post by: Headcool on October 02, 2017, 01:03:52 pm: It is implemented, which is a good message.

I use 23.0.62. Since thumbnail scraping does not work, it is likely that there is a bug in MC or a problem with the tvdb.com servers.

Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.
Title: Re: TheTVDB API change
Post by: Ashfall on October 02, 2017, 01:38:02 pm: Quote from: Headcool on October 02, 2017, 01:03:52 pm
It is implemented, which is a good message.

I use 23.0.62. Since thumbnail scraping does not work, it is likely that there is a bug in MC or a problem with the tvdb.com servers.

Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.

It's working for me. I'm on 23.0.61.
Title: Re: TheTVDB API change
Post by: tls62dk on October 02, 2017, 04:02:37 pm: It is working for me on 23.0.66. The tvdb site did not respond for some time yesterday, but is back up again today.
Title: Re: TheTVDB API change
Post by: blgentry on October 02, 2017, 04:16:45 pm: I just tried this myself. I'm on MC23.0.65 for Mac.

The episode images show when you pull up an individual episode with "Get Move and TV info". You can see it in the preview window. But when you press "Use this data", it will not save the episode cover art.

Kinda strange.

Brian.
Title: Re: TheTVDB API change
Post by: Hendrik on October 02, 2017, 06:05:29 pm: Quote from: Headcool on October 02, 2017, 01:03:52 pm
Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.

The v2 API uses HTTPS, so wireshark can't actually decode that traffic, so how would you figure out what its actually telling you? :)
In any case MC would follow a redirect and request it from the new location - and episode, season and series images actually work just fine here - just now, anyway.
Title: Re: TheTVDB API change
Post by: Headcool on October 03, 2017, 10:29:53 am: Well some parts of the communication seem to be in http.
After starting scraping one epsiode, a total of 2 HTTP requests are sent and 2 are received.
One pair goes to to wikimedia, the other one goes to thetvdb.com to receive the image.
(https://img3.picload.org/image/dglirocw/wireshark.png)
The other parts are indeed encrypted

Edit:
I've used the logging function of MC. I've cleared the log directly before initiating the scraping.

Code: [Select]
0009969: 5576: Reader: CWinINetReader::Open: Start 0009969: 5576: Reader: CWinINetReader::Open: Opening http://thetvdb.com/banners/episodes/121361/6185708.jpg 0009969: 8664: Reader: CWinINetReader::Thread: Start 0009969: 8664: Reader: CWinINetReader::Thread: url: http://thetvdb.com/banners/episodes/121361/6185708.jpg 0009969: 8664: Reader: CWinINetReader::Connect: Start 0009969: 8664: Reader: CWinINetReader::Connect: Finish (0 ms) 0009969: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Start 0009969: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpOpenRequest successful 0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpSendRequest successful 0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Retry needed, trying again... 0010016: 8664: Reader: CWinINetReader::Connect: Start 0010016: 8664: Reader: CWinINetReader::Connect: Finish (0 ms) 0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpOpenRequest successful 0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpSendRequest error: 0 0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Failed (header status code: -1) 0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Finish (109 ms) 0010078: 8664: Reader: CWinINetReader::Thread: Finish (109 ms) 0010078: 5576: Reader: CWinINetReader::Open: ConnectToURL failed, m_bConnectError=true, URL=https://www.thetvdb.com/banners/episodes/121361/6185708.jpg 0010078: 5576: Reader: CWinINetReader::Close: Start 0010078: 5576: Reader: CWinINetReader::Close: Finish (0 ms) 0010078: 5576: Reader: CWinINetReader::Open: Finish (109 ms)
I've attached the full log. Requesting the episode information works flawlessly. Requesting the cover art image through Firefox works.
Title: Re: TheTVDB API change
Post by: Headcool on October 04, 2017, 03:23:18 pm: I've debugged the binary a little bit. So the HttpSendRequestW function returns 12158(ERROR_INTERNET_UNABLE_TO_CACHE_FILE). The code directly after the call checks for ERROR_INTERNET_INVALID_CA and for ERROR_INTERNET_SEC_CERT_CN_INVALID via GetLastError().
There is a check for ERROR_INTERNET_UNABLE_TO_CACHE_FILE further down, directly after "HttpSendRequest error: %d" is written to the log file. At this point the value is always zero, so I assume somewhere between these points a functions is called which resets the error code that can be retrieved with GetLastError().
Title: Re: TheTVDB API change
Post by: Hendrik on October 04, 2017, 04:36:05 pm: I've updated the URL to remove the need for the redirect, and fixed the error check there. Nevertheless, it has reliably returned results here with old versions as well.
Title: Re: TheTVDB API change
Post by: blgentry on October 04, 2017, 04:43:02 pm: ^ Wait a second. I'm not sure that the problem is being understood. The problem isn't MC retrieving the data. The problem is MC writing the data. When I go "Get move and TV Info" on a tv show episode, I see the art displayed in the dialog box. But it will not save to database/disk.

To test this for yourself, find a TV show or two and do Cover Art > Remove Cover art . Then do Get Movie and TV Info. You should see art come up in the dialog, but will not save to the library.

Brian.
Title: Re: TheTVDB API change
Post by: Hendrik on October 04, 2017, 04:49:39 pm: You are the only one talking about that problem. And that also works just perfectly for me. Note that this operation is not supported on library server clients.
Title: Re: TheTVDB API change
Post by: Headcool on October 04, 2017, 04:53:13 pm: @blgentry
Well, my problem is different. The image doesn't even appear in the dialog.

@Hendrik
I should add that I use the 64-bit version.
Title: Re: TheTVDB API change
Post by: JimH on October 04, 2017, 05:42:34 pm: Please check your firewall.
Title: Re: TheTVDB API change
Post by: blgentry on October 04, 2017, 05:43:07 pm: Hmmm. I guess I misunderstood. I'll have to try this with freshly imported files and see if it still happens. I'm not using a server or client; standalone MC machine.

Brian.
Title: Re: TheTVDB API change
Post by: Headcool on October 04, 2017, 07:00:07 pm: @Hendrik
I'm using the Windows firewall. Turning off the firewall does not make any difference. There are 2 rules for MC 23 for TCP and UDP in there that seem to be fine. I've reset all the rules to default and MC added them again as I expected. However no difference.
I activated the log which states that all the connections to thetvdb.com were allowed.
Title: Re: TheTVDB API change
Post by: blgentry on October 05, 2017, 06:04:07 pm: The "problem" I observed on my system is not happening any more. I think I had something wonky going on with my external disk drives. Not sure, but I *am* sure that MC is working correctly now with TV Episode art and nothing about MC has changed on my end.

Sorry for the misleading report.

Brian.
Title: Re: TheTVDB API change
Post by: Headcool on October 10, 2017, 09:04:57 am: I can confirm that the problem is now fixed in 23.0.70.

Good work.