Topic: Request: Meta Data scraping additions (Read 5868 times)

MrHaugen · « **on:** March 07, 2012, 11:23:27 am »

We've had the pleasure of having a meta data scraping system in MC for the last 50-70 builds or so in MC. Which is an enormous improvement compared to MC16 and previous versions. However, some people think that the current system is not adequate. Some of us still have to relay on tagging from other applications, or manually adding data within MC, because some meta data is left out on MC's part.

This is what we have today, as far as I've seen. I believe all of this data is currently retrieved through themoviedb.org and thetvdb.com respectively
Movies: Actors, Budget, Description, Director, Keywords, MPAA Rating, Genres, Year/Date, Producer
TV Shows: Access Rating, Actors, Critic Rating (Site Episode Rating?), Description, Description, Director, Screenwriter

There are some important data here that I know many users would appreciate. The data is weighted with the most important once first (from my perspective).
Movies: Rating (themoviedb.org), Critic Rating (from rottentomatoes), Languages (themoviedb.org), Trailer URL (themoviedb.org), Writer (themoviedb.org)
TV Shows: First Aired (thetvdb.com), Genre (thetvdb.com), Network (thetvdb.com), TMDB Show Rating (thetvdb.com)

I think the 2-3 first tags here is really important for quite some users. Can you please look at this when you have the time?

lise · « **Reply #1 on:** March 07, 2012, 04:46:44 pm »

The only fields I care about that I am not getting automatically from TMDB are language, country and screenwriter. I care most for the first two, as I have views dedicated to foreign films. But... I can get these from Wikipedia.

So, whenever MC does it's automatic check with TMDB, I select the new films, get the info from Wiki then repeat with TMDB again (they have a better synopsis). That does the trick.

What would be fantastic is if I could somehow tell MC to get the language, country and screenwriting stuff from Wikipedia and the rest from TMDB automatically. Ohhh, that would be sweet.

CountryBumkin · « **Reply #2 on:** March 07, 2012, 05:12:35 pm »

+1 for "First Aired (for TV shows).

MrHaugen · « **Reply #3 on:** March 12, 2012, 07:50:36 am »

It's hard to believe that there's only three users that are missing such meta data.

xynoe44 · « **Reply #4 on:** March 12, 2012, 08:56:04 am »

+1 for scraping the new data.

Something else that could be useful is:

- independent movie/tvshow scraping systems (I don't know It works actually) with custom settings
- a way to enable/disable datasources (TMDB, TvDB, Rotten, etc...) for info/covers/fanarts -> similar to artists images
- a way to choose/change between available covers/fanarts from enabled datasources
- a way to choose what info to get from which enabled datasources (Name, Year from TMDB, Critic Rating from Rotten...)

These would made nice additions to the new metadata scraping system

bunglemebaby · « **Reply #5 on:** March 12, 2012, 05:21:42 pm »

Quote

These would made nice additions to the new metadata scraping system

Agreed, but I would also call these things part of v2 of the metadata lookup system. (If such a thing ever exists...)

The items mentioned in the original post are much more basic and mostly relate to pieces of metadata that can be downloaded for Movies but not for TV Shows (and vice-versa). It's a little odd to me that the lookup system has not received any attention at all lately given that these pieces are still missing.

MrHaugen · « **Reply #6 on:** March 13, 2012, 03:39:06 am »

There's been lots of requests for a more customizable version of the meta scraper, but that is not really the point of this thread. It's a complicated subject, and the more we ask for the harder it will be to get it I'm afraid

I'm just looking for some easy and quick additions to todays system, so it will work better for most of us. I believe that adding a few tags to the already implemented system should be very simple.

struct · « **Reply #7 on:** March 13, 2012, 04:24:59 am »

+1

Are you proposing that the auto import or selectmultiple and update gets data from both rt and imdb. if so this would be good as we currently can only get multiple file import to get data from tmdb ( as far as i can figure).

I would like the ttvdb or tvdb id saved somewhere.

i would very much like a tick box on the multiple select import to update empty fields only. this is surely a really simple addition and lets us top up missing info without risk of ruining good data.

justsomeguy · « **Reply #8 on:** March 13, 2012, 05:28:10 am »

+1

I switched to MC for my htpc after the metadata scraper was implemented. I'm happy for the most part. I was disappointed that several obvious pieces of data were missing though.

Absolutely need:
TV:
First air date
Genre
Movies:
Ratings is this biggest thing missing.

These are just my opinion but you can never have too much metadata so why not just add all data that is available?

lboregard · « **Reply #9 on:** March 13, 2012, 06:27:24 am »

+1
ratings and perhaps mpaa rating ... i like the idea someone posted above about adding language, country .. so that you can have a foreign view

MrHaugen · « **Reply #10 on:** March 21, 2012, 02:46:00 am »

*bump*

Would it be possible to at least add a few of the big missing pieces here?
Rating, Genre and Aired date should be a minimum for all video.

MrHaugen · « **Reply #11 on:** April 02, 2012, 04:29:18 am »

*bump again*

MrHaugen · « **Reply #12 on:** April 03, 2012, 05:25:42 am »

Last attempt before easter. Adding this 5 data fields to already existing MC library fields would make my day so much better. This should be very easy to add I presume. Please look into it some day. It's so frustrating to have a half working automatic scraping system, when you still have to relay on other software to do the job that should be done in MC.

Movie (data only from TMDB.com)
Rating = "Critic Rating" field (to easily use numbers higher than 5. At least I think this is the best fitting field)
Languages = "Language" field
Writer = "Screen Writer" field

TV Shows (data only from thetvdb.com)
First Aired = "Date" field
Genre = "Genre" field

darichman · « **Reply #13 on:** April 03, 2012, 07:34:04 am »

I'd really like to see these too

c1c9k72 · « **Reply #14 on:** April 03, 2012, 05:56:02 pm »

Agreed. Especially with the television fields. +1.

MasterC · « **Reply #15 on:** April 04, 2012, 02:43:49 am »

Agreed on all points....

struct · « **Reply #16 on:** April 04, 2012, 08:36:30 am »

Agreed on this and multiple points related to getting metadata into the best metadAta manager there is.

MrHaugen · « **Reply #17 on:** May 07, 2012, 08:03:13 am »

Recently we got the IMDB ID tag added to the scraper, but we're still missing some commonly used fields here. I would at least think that many would like to use "Rating" for movies as well as the "First Aired" date for episodes. It would also be pretty convenient to be able to sort by genre for TV Shows if you have many TV Shows, as well as Language tags for Movies to be sure you don't need any subtitles. Pretty please take the time to look into adding this few remaining bits of the meta data system, so we can finally get rid of third party apps/plug-ins and scripts to do do this job

Hendrik · « **Reply #18 on:** May 07, 2012, 08:42:17 am »

First Aired is already being imported for TV Shows, iirc.

MrHaugen · « **Reply #19 on:** May 07, 2012, 09:14:59 am »

Yepp! You are correct. I missed that one. The list is shrinking, and that is good to know.

TV Shows
Access Rating
Actors
Critic Rating
Description
Director
Writer
tvdb ID
IMDB ID
TV Show art
Episode Thumbnails
First Aired
Genre
Network
Season Art
Banner Art
Backdrop Art (only automatic today)

Movies
Actors
Budget
Description
Director
Keywords
MPAA Rating
Genres
Year/Date
Movie Covers
Producer
tmdb ID
Ratings
Language
Writer
Backdrops (only automatic today)

CountryBumkin · « **Reply #20 on:** May 07, 2012, 10:45:46 am »

Maybe with Movies some would find it useful to know if a movie is one part of a sequel (or part of a trilogy and whatever the term is for four, etc.)

c1c9k72 · « **Reply #21 on:** May 07, 2012, 12:26:22 pm »

Quote from: nevcairiel on May 07, 2012, 08:42:17 am

First Aired is already being imported for TV Shows, iirc.

What field is First Aired being stored in?

glynor · « **Reply #22 on:** May 07, 2012, 12:29:39 pm »

Quote from: c1c9k72 on May 07, 2012, 12:26:22 pm

What field is First Aired being stored in?

I think just vanilla [Date].

c1c9k72 · « **Reply #23 on:** May 07, 2012, 12:32:34 pm »

Quote from: glynor on May 07, 2012, 12:29:39 pm

I think just vanilla [Date].

I'm still using 17.0.125, and I've got shows with empty [Date] fields that I know have TVDB First Aired data. Is it something that's been introduced recently?

MrHaugen · « **Reply #24 on:** May 07, 2012, 12:53:28 pm »

It's not extremely many builds since it was added, so yes. An update might be needed.

c1c9k72 · « **Reply #25 on:** May 07, 2012, 12:59:53 pm »

Quote from: MrHaugen on May 07, 2012, 12:53:28 pm

It's not extremely many builds since it was added, so yes. An update might be needed.

Just updated, and it's working now. Thanks.

MrHaugen · « **Reply #26 on:** May 29, 2012, 08:19:24 am »

We're still missing some meta data and artwork, for example missing rating on movies.

The background change still takes a lot of time sometimes, because of the time it takes to download them. Please consider doing this once for a show or movie for those that want it. I think it would help the data traffic from tvdb and themoviedb as well (this is perhaps some of the reasons why episodes are missing screen shots from tvdb.com?).

This is the updated list of what I believe is missing.

Quote from: MrHaugen on May 07, 2012, 09:14:59 am

TV Shows
Genre
Network
Season Art
Banner Art
Backdrops

Movies
Ratings
Language
Writer
Backdrops

INTERACT FORUM

Author Topic: Request: Meta Data scraping additions (Read 5868 times)