INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: TheTVDB API change  (Read 4108 times)

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
TheTVDB API change
« on: October 02, 2017, 12:49:30 pm »
I noticed yesterday, that the scraping of episode thumbnails did not work anymore. I investigated a little bit, and it seems that thetvdb.com planned to discontinued their APIv1 with the 1. October 2017.
So I guess the new API is not implemented in MC.

I hope this issue can be fixed soon. Note that this may be the ideal opportunity to switch to themoviedb.org (which is already used for movies) instead of thetvdb.com.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42444
  • Shoes gone again!
Re: TheTVDB API change
« Reply #1 on: October 02, 2017, 12:50:37 pm »
It was done for MC22 in May.

22.0.107 (5/16/2017)

Changed: Updated the TheTVDB integration to the v2 TheTVDB API.
Logged
Matt Ashland, JRiver Media Center

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #2 on: October 02, 2017, 01:03:52 pm »
It is implemented, which is a good message.

I use 23.0.62. Since thumbnail scraping does not work, it is likely that there is a bug in MC or a problem with the tvdb.com servers.

Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.
Logged

Ashfall

  • World Citizen
  • ***
  • Posts: 187
Re: TheTVDB API change
« Reply #3 on: October 02, 2017, 01:38:02 pm »
Quote from: Headcool on October 02, 2017, 01:03:52 pm
It is implemented, which is a good message.

I use 23.0.62. Since thumbnail scraping does not work, it is likely that there is a bug in MC or a problem with the tvdb.com servers.

Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.

It's working for me. I'm on 23.0.61.
Logged

tls62dk

  • Galactic Citizen
  • ****
  • Posts: 406
Re: TheTVDB API change
« Reply #4 on: October 02, 2017, 04:02:37 pm »
It is working for me on 23.0.66. The tvdb site did not respond for some time yesterday, but is back up again today.
Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: TheTVDB API change
« Reply #5 on: October 02, 2017, 04:16:45 pm »
I just tried this myself.  I'm on MC23.0.65 for Mac.

The episode images show when you pull up an individual episode with "Get Move and TV info".  You can see it in the preview window.  But when you press "Use this data", it will not save the episode cover art.

Kinda strange.

Brian.
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10970
Re: TheTVDB API change
« Reply #6 on: October 02, 2017, 06:05:29 pm »
Quote from: Headcool on October 02, 2017, 01:03:52 pm
Edit: Using 23.0.66 does not fix the problem. According to Wireshark, the server return status code 301 - Moved permanently.

The v2 API uses HTTPS, so wireshark can't actually decode that traffic, so how would you figure out what its actually telling you? :)
In any case MC would follow a redirect and request it from the new location - and episode, season and series images actually work just fine here - just now, anyway.
Logged
~ nevcairiel
~ Author of LAV Filters

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #7 on: October 03, 2017, 10:29:53 am »
Well some parts of the communication seem to be in http.
After starting scraping one epsiode, a total of 2 HTTP requests are sent and 2 are received.
One pair goes to to wikimedia, the other one goes to thetvdb.com to receive the image.

The other parts are indeed encrypted

Edit:
I've used the logging function of MC. I've cleared the log directly before initiating the scraping.

Code: [Select]
0009969: 5576: Reader: CWinINetReader::Open: Start
0009969: 5576: Reader: CWinINetReader::Open: Opening http://thetvdb.com/banners/episodes/121361/6185708.jpg
0009969: 8664: Reader: CWinINetReader::Thread: Start
0009969: 8664: Reader: CWinINetReader::Thread: url: http://thetvdb.com/banners/episodes/121361/6185708.jpg
0009969: 8664: Reader: CWinINetReader::Connect: Start
0009969: 8664: Reader: CWinINetReader::Connect: Finish (0 ms)
0009969: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Start
0009969: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpOpenRequest successful
0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpSendRequest successful
0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Retry needed, trying again...
0010016: 8664: Reader: CWinINetReader::Connect: Start
0010016: 8664: Reader: CWinINetReader::Connect: Finish (0 ms)
0010016: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpOpenRequest successful
0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: HttpSendRequest error: 0
0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Failed (header status code: -1)
0010078: 8664: Reader: CWinINetReader::DownloadFromHTTPURL: Finish (109 ms)
0010078: 8664: Reader: CWinINetReader::Thread: Finish (109 ms)
0010078: 5576: Reader: CWinINetReader::Open: ConnectToURL failed, m_bConnectError=true, URL=https://www.thetvdb.com/banners/episodes/121361/6185708.jpg
0010078: 5576: Reader: CWinINetReader::Close: Start
0010078: 5576: Reader: CWinINetReader::Close: Finish (0 ms)
0010078: 5576: Reader: CWinINetReader::Open: Finish (109 ms)
I've attached the full log. Requesting the episode information works flawlessly. Requesting the cover art image through Firefox works.
Logged

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #8 on: October 04, 2017, 03:23:18 pm »
I've debugged the binary a little bit. So the HttpSendRequestW function returns 12158(ERROR_INTERNET_UNABLE_TO_CACHE_FILE). The code directly after the call checks for ERROR_INTERNET_INVALID_CA and for ERROR_INTERNET_SEC_CERT_CN_INVALID via GetLastError().
There is a check for ERROR_INTERNET_UNABLE_TO_CACHE_FILE further down, directly after "HttpSendRequest error: %d" is written to the log file. At this point the value is always zero, so I assume somewhere between these points a functions is called which resets the error code that can be retrieved with GetLastError().
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10970
Re: TheTVDB API change
« Reply #9 on: October 04, 2017, 04:36:05 pm »
I've updated the URL to remove the need for the redirect, and fixed the error check there. Nevertheless, it has reliably returned results here with old versions as well.
Logged
~ nevcairiel
~ Author of LAV Filters

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: TheTVDB API change
« Reply #10 on: October 04, 2017, 04:43:02 pm »
^ Wait a second.  I'm not sure that the problem is being understood.  The problem isn't MC retrieving the data.  The problem is MC writing the data.  When I go "Get move and TV Info" on a tv show episode, I see the art displayed in the dialog box.  But it will not save to database/disk. 

To test this for yourself, find a TV show or two and do Cover Art > Remove Cover art .  Then do Get Movie and TV Info.  You should see art come up in the dialog, but will not save to the library.

Brian.
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10970
Re: TheTVDB API change
« Reply #11 on: October 04, 2017, 04:49:39 pm »
You are the only one talking about that problem. And that also works just perfectly for me. Note that this operation is not supported on library server clients.
Logged
~ nevcairiel
~ Author of LAV Filters

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #12 on: October 04, 2017, 04:53:13 pm »
@blgentry
Well, my problem is different. The image doesn't even appear in the dialog.

@Hendrik
I should add that I use the 64-bit version.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72546
  • Where did I put my teeth?
Re: TheTVDB API change
« Reply #13 on: October 04, 2017, 05:42:34 pm »
Please check your firewall.
Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: TheTVDB API change
« Reply #14 on: October 04, 2017, 05:43:07 pm »
Hmmm.  I guess I misunderstood.  I'll have to try this with freshly imported files and see if it still happens.  I'm not using a server or client; standalone MC machine.

Brian.
Logged

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #15 on: October 04, 2017, 07:00:07 pm »
@Hendrik
I'm using the Windows firewall. Turning off the firewall does not make any difference. There are 2 rules for MC 23 for TCP and UDP in there that seem to be fine. I've reset all the rules to default and MC added them again as I expected. However no difference.
I activated the log which states that all the connections to thetvdb.com were allowed.
Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: TheTVDB API change
« Reply #16 on: October 05, 2017, 06:04:07 pm »
The "problem" I observed on my system is not happening any more.  I think I had something wonky going on with my external disk drives.  Not sure, but I *am* sure that MC is working correctly now with TV Episode art and nothing about MC has changed on my end.

Sorry for the misleading report.

Brian.
Logged

Headcool

  • Junior Woodchuck
  • **
  • Posts: 70
Re: TheTVDB API change
« Reply #17 on: October 10, 2017, 09:04:57 am »
I can confirm that the problem is now fixed in 23.0.70.

Good work.
Logged
Pages: [1]   Go Up
« previous next »