INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: scraping - is this as good as it (currently) gets?  (Read 1600 times)

yannis

  • World Citizen
  • ***
  • Posts: 229
scraping - is this as good as it (currently) gets?
« on: January 17, 2013, 04:26:13 am »

I've tried to read about scraping but the info is sparse and sometimes conflicting. So here are a few questions and thoughts.

Is there really no way to add more search engines? I've tried some films and it works ok with the big ones, but it's hopeless with anything just a little obscure, eg The Ransom (1963) by Kurosawa. All the eye candy of Theater View for almost nothing.

Is there a way to incrementally update from different sources? Can't we select one source to keep specific fields?

Is there no way to import info from other sources? I have maintained my library in another program and I can export an XML or XLS list from there. Can MC take advantage of that?
Logged

yannis

  • World Citizen
  • ***
  • Posts: 229
Re: scraping - is this as good as it (currently) gets?
« Reply #1 on: January 22, 2013, 02:01:24 am »

bump?
Logged

InflatableMouse

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3978
Re: scraping - is this as good as it (currently) gets?
« Reply #2 on: January 22, 2013, 02:16:58 am »

I've tried to read about scraping but the info is sparse and sometimes conflicting. So here are a few questions and thoughts.

If you can point to the sources of conflicting info someone could correct it. Right now its a statement that no one can do anything about.

Is there really no way to add more search engines? I've tried some films and it works ok with the big ones, but it's hopeless with anything just a little obscure, eg The Ransom (1963) by Kurosawa. All the eye candy of Theater View for almost nothing.

No. When you do it manually on a single item, you can play with the query. In this case, use the alternate title, "High and Low". It's found with all its metadata and cover.

Especially with foreign titles, search IMDB or use Google to find alternative titles and use them to query.

Is there a way to incrementally update from different sources? Can't we select one source to keep specific fields?

Yes. At least for covers or metadata. Select the one for which you want either a cover or the metadata, and uncheck the tickbox. Select the other one and uncheck the other box. Voila :).

Is there no way to import info from other sources? I have maintained my library in another program and I can export an XML or XLS list from there. Can MC take advantage of that?

Not that I'm aware of, no.

If you have a good source that supports scraping, propose it and you may find the developers willing to incorporate it into the program.
Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: scraping - is this as good as it (currently) gets?
« Reply #3 on: January 22, 2013, 02:18:18 am »

I would say that the answer is no, no and probably no. The scraping engine i very basic, and only searches for some very limited fields from the moviedb and thetvdb. That's it. I really hope that this will get more flexible in time. At least to get the data that are available on those sources.
Logged
- I may not always believe what I'm saying

struct

  • Galactic Citizen
  • ****
  • Posts: 380
Re: scraping - is this as good as it (currently) gets?
« Reply #4 on: January 22, 2013, 03:37:37 am »

Is there no way to import info from other sources? I have maintained my library in another program and I can export an XML or XLS list from there. Can MC take advantage of that?

JRiver has its own sidecar file for storing information for video (see an example in one of your directories with a tagged movie).  You may be able to make a template in your current program to export in this format.  You could also write a translation script to convert from your current xml format to jriver's, it doesn't look too complex. 

JRiver can still (I think) read a MyMovies xml file if there is one.  Maybe you can export to it? 

What are you using currently?

We are all still hoping that they can come back to improve the scraping and flexibility.  There were a ton of great ideas presented last year, but nothing has come of it yet.  I hope it requires just a little more patience.

Craig
Logged

yannis

  • World Citizen
  • ***
  • Posts: 229
Re: scraping - is this as good as it (currently) gets?
« Reply #5 on: January 22, 2013, 05:31:09 am »

Thank you all for your replies.

If you can point to the sources of conflicting info someone could correct it. Right now its a statement that no one can do anything about.
Right now there's only the posts, and the "conflict" comes from having to find and read them all, because the older ones have no updated info. I would point to the wiki but nothing pops up when searching there for "scrape" or "scraping".


No. When you do it manually on a single item, you can play with the query. In this case, use the alternate title, "High and Low". It's found with all its metadata and cover.
Especially with foreign titles, search IMDB or use Google to find alternative titles and use them to query.
Just like I did - but I guess you realize this is not an option if you have a large number of files. To reiterate, it's hopeless when one has non standard tastes...

Yes. At least for covers or metadata. Select the one for which you want either a cover or the metadata, and uncheck the tickbox. Select the other one and uncheck the other box. Voila :).
I was talking about selecting what metadata to keep. Right now there's only a blanket Replace ALL "option".


JRiver has its own sidecar file for storing information for video (see an example in one of your directories with a tagged movie).  You may be able to make a template in your current program to export in this format.  You could also write a translation script to convert from your current xml format to jriver's, it doesn't look too complex. 
JRiver can still (I think) read a MyMovies xml file if there is one.  Maybe you can export to it? 
What are you using currently?
And that's what I did in the few days awaiting a response. I eventually found a way to export nfo files which are read properly by MC, even with custom fields used. But I'd love it if I could replace some fields with other/newer data. And generally, it would just be much nicer if we could have this functionality inside MC, as it is a much better program in other respects.

We are all still hoping that they can come back to improve the scraping and flexibility.  There were a ton of great ideas presented last year, but nothing has come of it yet.  I hope it requires just a little more patience.
My thoughts exactly.
Logged

InflatableMouse

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3978
Re: scraping - is this as good as it (currently) gets?
« Reply #6 on: January 22, 2013, 05:44:12 am »

Just like I did - but I guess you realize this is not an option if you have a large number of files. To reiterate, it's hopeless when one has non standard tastes...

I feel your pain. I guess what could work is if MC is able to find a list of original and alternative titles and try to match on them as well.

I was talking about selecting what metadata to keep. Right now there's only a blanket Replace ALL "option".

I know ;). I was being sarcastic. Its metadata or covers. Sorry.
Logged
Pages: [1]   Go Up