Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[glynor]

As discussed in this thread, MC's Automatic Medatada Lookup system is currently mapping TMDb's Plot Keywords "field" to MC's [Keywords] field.  This is useless because those Plot Keywords on TMDb's site really seem to be designed to be "search hints" for TMDb's Search Engine.

So, if you use MC's built-in lookup system currently, your [Keywords] field for movies very quickly becomes filled with all sorts of awesome things like "on the run", "cemetery", "bravery", and "snake" for Harry Potter and the Deathly Hallows, Part 1 (as an example).  The Plot Keywords are junk, and most users (who commented in the above-referenced thread) agree.

Proposed Solution:  Map TMDb's Genres field to [Keywords] instead.

TMDb has nice information in the Genres field.  You can't map this directly to MC's [Genre] field though because the TMDb version is a list-type field, and MC's [Genre] field is a string-type field.

There are two upsides to switching this:

1. [Keywords] is used in the default Video views in both Theater View and Standard View.  Therefore, it will be very painless to existing users to make use of this new automatically-obtained data without having to reset their views or build a new view from scratch.

2. TMDb's Plot Keywords are bad, and they are already filling our [Keywords] field up with junk.  Any user who is already using the [Keywords] field to store Genre information is a LOT more likely to be happy with the contents of TMDb's Genre field rather than the Plot Keywords field (that I can't imagine anyone wants).  If they aren't, then they can move their existing data to a new custom field before they enable the Metadata lookup.

The downside is that anyone who has been using the Plot Keywords form TMDb on purpose would be unable to continue doing so.  But, is there anyone actually in this camp?  The other downside is that the name [Keywords] does not necessarily "intuitively map" to TMDb's use of "genre" to describe the contents of the field.

Alternate Solution:  Create a [Movie Genre] default field in MC (list-type) and map TMDb's Genre information to this field (and leave the existing mapping alone).

This has the upside of not messing with an existing system, and the Field Naming is more reasonable.  However, it has a potentially big downside:

You will have to create new default views that include this new [Movie Genre] field for it to be useful (it will need to be turned on in the Video views' columns by default, and it will need to be used as a category in Standard, Theater, and Media Network views.  This isn't a big deal for new users, but most existing users will never notice that it is there.  Existing users will have to reset all of their views back to the default, or manually modify their relevant views, to even KNOW that MC is filling this new tag with information and be able to use it.

Either of these two solutions would work for me, and would, I believe, work for the other people commenting in the above-referenced thread.  So, which of these two options you choose is your call.

In any case, the current behavior of mapping TMDb's Plot Keywords field to [Keywords] is useless.

[MrHaugen]

I'll try to say what I did in the other thread with as few words possible.

I'm VERY interested in getting Genres data from TMDB. I would use it a lot for Movies, and I think many others would too! But there several right ways of doing this, and a few bad ways imo.

- Mapping Genres to Keywords is bad. People using Keywords today are screwed. They will not get Genres. This can be people using Keywords from TMBD, though I doubt there are many who actually use this crap. More likely there is users who manually or automatically uses Keywords from other services. Genres in Keywords is not logical. It will confuse and irritate. We have the ability to add what ever fields in MC. Use that. Do not map things to wrong fields.

- Create a "Genres" field (list type?), and fill it with Genre info from TMDB. This would be a more logical way. It would not screw things up for the users. It would not confuse them. The downsides as Glynor points out is the new potential views. Several solutions should be possible here.

1. Create views, but only activate new views if the users manually reset the views. And on new installations of course.
2. Do not create new views. Let users add them if they want the Genres views
3. Create the views and force the users to reset views (this is not a great option imo)

[glynor]

They made a decision in the other thread.  I'm going to lock this now.

I can't.

Why can I lock my own threads sometimes and not other times?  It confuses me.

[JustinChase]

Well, since you can't lock it, I'll chime in

As for mapping the Genres list to Keywords; yeah techinically wrong, and potentially bad.  However, anyone that uses Keywords will have the field populated already, so as long as that's not overwritten, no foul, no problem.

As for blank Keyword fields on purpose, having these populated with Genres might be bad/frustrating, but probably won't affect too many, and will not be not too hard to "fix", considering the Genres will likely be totally out of place with the rest of the keywords, so pretty easy to find/delete.  Perhaps they would need to enter a 'junk' keywork/placeholder to keep them from being populated??

However, the benefit of this is that people that do want multiple Genres can get them this way.  I don't think the wish was "please put Genres in Keywords" but more of "current Keywords are useless, but Genres might be useful, so let's grab those instead."

Putting them in a third/new field would be fine, but has the disadvantage of not being included in any views anywhere, so would likely be lost to most users, so pretty much defeats the purpose/benefit of grabbing them.

Anyway, as Glynor said, a decision/change was just  made, but like with much of this scraping progress lately, changes can lead to new "issues" so this may not have been the FINAL word on all this, which is the only reason I bothered to write all this

[glynor]

However, the benefit of this is that people that do want multiple Genres can get them this way.  I don't think the wish was "please put Genres in Keywords" but more of "current Keywords are useless, but Genres might be useful, so let's grab those instead."

Exactly.

Putting them in a third/new field would be fine, but has the disadvantage of not being included in any views anywhere, so would likely be lost to most users, so pretty much defeats the purpose/benefit of grabbing them.

Exactly.

As for mapping the Genres list to Keywords; yeah techinically wrong, and potentially bad.  However, anyone that uses Keywords will have the field populated already, so as long as that's not overwritten, no foul, no problem.

Exactly.

As for blank Keyword fields on purpose, having these populated with Genres might be bad/frustrating, but probably won't affect too many, and will not be not too hard to "fix", considering the Genres will likely be totally out of place with the rest of the keywords, so pretty easy to find/delete.  Perhaps they would need to enter a 'junk' keywork/placeholder to keep them from being populated??

That is a darn good point, and is why I commented with all of my "exactlies."

If you are, like Bill in the other thread, the type of user who never wants [Keywords] to be filled with stuff looked up (from TMDb or anywhere), there is a very simple solution:  Simply add a Tag On Import rule to your Auto Import folders that manually sets [Keywords]=Undefined (or "blank", or "new", or something like that).  Then, problem solved.  Since the [Keyword] field for all newly imported files will then "contain" something, MC won't overwrite it with metadata it gets from TMDb.

I don't know how you solve the other problems any other way.  I didn't even think about Tag On Import rules being able to solve the problem, so that was great thinking Justin.

[MrHaugen]

As for mapping the Genres list to Keywords; yeah techinically wrong, and potentially bad.  However, anyone that uses Keywords will have the field populated already, so as long as that's not overwritten, no foul, no problem.

Wrong. Those that does use Keywords will not get Genres. That is bad for some people. There is nothing that says you can't use Genre if you're using Keywords. A person should be able to choose both.

As for blank Keyword fields on purpose, having these populated with Genres might be bad/frustrating, but probably won't affect too many, and will not be not too hard to "fix", considering the Genres will likely be totally out of place with the rest of the keywords, so pretty easy to find/delete.  Perhaps they would need to enter a 'junk' keywork/placeholder to keep them from being populated??

This is things that is populated on import. Services that updates this fields at a later time might have problem with it. Perhaps the XML info would not be picked up? I doubt it, but non the less. It's a potential problem. What about users that fill this stuff in manually? They will not know which fields has not been processed. They have no Unassigned "tag". Sure, you can create a Import rule to clear it, but why all this trouble?

However, the benefit of this is that people that do want multiple Genres can get them this way.  I don't think the wish was "please put Genres in Keywords" but more of "current Keywords are useless, but Genres might be useful, so let's grab those instead."

I agree. It's a quick fix. And I would be satisfied for some time. But it's a dirty and quick fix that WILL cause problems. Just as potentially resetting views. But resetting views will not haunt us for years to come.

Putting them in a third/new field would be fine, but has the disadvantage of not being included in any views anywhere, so would likely be lost to most users, so pretty much defeats the purpose/benefit of grabbing them.

They will be included in new views. So new users would see it. The people that is interested in it would probably manage to create a view, and we could have the ability to manually reset the views if people want it. I do not see many downsides. I see lots more negative by mixing two completely different data sets just to get thing up and running faster.

Anyway, as Glynor said, a decision/change was just  made, but like with much of this scraping progress lately, changes can lead to new "issues" so this may not have been the FINAL word on all this, which is the only reason I bothered to write all this

True. Let's hope a better and permanent solution is on the drawing board.

One other thing. I do not know how this View setups is coded. But would it not be possible to just ADD one single view to the Video category, rather than having to reset it all. In this changes we're talking about here, there are not changes to the roller items. Just a view name extra. We've not changing the menu rollers or anything. Just a thought.

If this is not possible today, I would strongly advice to look into this possibility for future additions to default views. Because it will probably happen again.

[glynor]

Wrong. Those that does use Keywords will not get Genres. That is bad for some people. There is nothing that says you can't use Genre if you're using Keywords. A person should be able to choose both.

I really don't get what you're saying here.

Changing it to use [Keywords] would NOT prevent you from using anything on your own.  The ONLY thing it would prevent would be if you WANT to use TMDb's Plot Keywords AND TMDb's Genres, and not have to manually fill a custom field.  But you can't do that now, so we're discussing a complete hypothetical!

I'm not suggesting that you HAVE TO use TMDb's [Keywords], just that they pre-fill if otherwise blank.  That seems like it would be ideal for 98% of the users out there.  If you don't want them to pre-fill at all under any circumstances, just set a Tag On Import rule to blank it out.

If you DO want to use TMDb's Genres, but not their Keywords, then you ALSO don't have a problem.  You will have to add a custom field and move YOUR keywords to the custom field (because now TMDb's genres would be filling it up on import), but that's no big deal.  There's like 10 of you.  I'd rather those ten people have to change their views than EVERYONE ELSE.

They will be included in new views. So new users would see it. The people that is interested in it would probably manage to create a view, and we could have the ability to manually reset the views if people want it. I do not see many downsides.

I really think you're glossing over this.

I think that is the single biggest problem with any tagging change in MC.  It has enormous implications.  It seems silly to me to do this to protect the small number of users who want to use the Plot Keywords tags from TMDb.  Is there seriously anyone out there who fits this criteria?

[glynor]

PS.  I'm really not trying to be a jerk here, but it seems like a useful feature is being neutered because of weird ideological reasons and nothing practical.

Can you explain why you say you can't use both?  What do you mean by both, specifically?

Are you unwilling to change your views to not use the existing [Keywords] field?  Is that what you mean?  I'm just really confused...

[MrHaugen]

I really don't get what you're saying here.

When Genres is mapped to Keywords, you effectively leaves out the use of Keywords or Plot Keywords. At least in it's logical linked library field. You'll need to create a custom field to get it, and you'll have to find other means to get the data.  Because if MC fills Genres to Keywords and I would be surprised if they create another field for this so people can use keywords in the automatic or manual Scraping.

Sure, the data retrieved today are not good. But some day this might be fixed by using another source. And at that time, we'll have an awkward situation.

I really think you're glossing over this.

I think that is the single biggest problem with any tagging change in MC.  It has enormous implications.  It seems silly to me to do this to protect the small number of users who want to use the Plot Keywords tags from TMDb.  Is there seriously anyone out there who fits this criteria?

There might not be lots of people using the keyword field, but remember that this is the first time we have had scraping possibilities in MC. This can change fast. I for one would love to have all the meta data I can. So I can decide to use it later.

I think it's a bit nuts to make a illogical meta data and library field mapping change that prevents other user scenarios, just because you want something working. Quick and dirty. The problems with views is a problem. I see that. But it should really be ways to minimize this. We've had to live without tags for LOTS of years. I think it will be no harm if some users does not pick up this changes and does not get the views. They will survive. And one day they will probably start with new news. Those that are really interested in genres will probably pick up this changes, and reset the views or create a custom view to adapt.

Summary: No real problems. Some unknowing users. No problems with illogical tags or scraping crash later on.


Anyone else are allowed to throw in their thoughts here!
Glynor and Justin, you have your way of seeing this "logic". I have mine. Nothing will change my view of this, and it's tiresome to repeat your self I've said what I want to say about this subject.

However... I'm all for the second alternative, and I'm confident it can be implemented to minimize view problems if done right.

[glynor]

When Genres is mapped to Keywords, you effectively leaves out the use of Keywords or Plot Keywords. At least in it's logical linked library field. You'll need to create a custom field to get it, and you'll have to find other means to get the data.  Because if MC fills Genres to Keywords and I would be surprised if they create another field for this so people can use keywords in the automatic or manual Scraping.

This is the part I don't get.

Can you give me a real-world example.

I really just don't get it.  Who in their right mind wants the Plot Keywords from TMDb?  If you DON'T want them, how would mapping something maybe-useful-to-some-people data there instead hurt you?

[glynor]

I should add... I have absolutely no interest in spending a bunch of time worrying about future things that may or may not ever happen.  I understand you don't want to BREAK things for the future, but this seems like it is all discussing hypotheticals without any real-world examples.

If your main problem is that the field is called [Keywords] and not [Genres], but you're not actually using the existing [Keywords] field (which, again, is already all junked up with crap) then I don't get what difference that makes.

[MrHaugen]

Ok. My last few sentences. I'm off for bed now.

Not many would want the keywords today. But the day we can use other sources, people might. There are also other tools and services that use this library field. Like Couchpotato (probably) and Media Center Master. I'm testing this apps to fill in blanks as we speak.

If you get your way, and Genres is being mapped to this field, I'll have to use a custom field because my logical choice was taken by another field which I ALSO require.

If I'm only using MC and the scraping tool today or with future enhancements, I'll be stuck with Genres in Keywords field. If I choose to actually use todays keywords from TMDB I have to create a new field. Which is pretty idiotic, when we know that this was available before, but is now used to something totally different. In one year, JRiver might have given us the possibility to select source for each of this fields (custom things like IMDB access). Then I would definitely find a better source for Keywords and start to look at the possibilities to use this.

So, there you have it. 3 perfectly reasonably ways people would want to use Keywords field.

PS. I think this is actually the first time in plenty of years that Glynor and I have ha so different opinions and heated debate. Have to mark this day with a few beers tomorrow

Good night

[glynor]

I also just wanted to say, I don't feel like it is "heated".  I absolutely respect you, MrHaugen.

I'm just confused.

I'll comment more later when I have more time.  Have a nice sleep!

[JustinChase]

Not many would want the keywords today. But the day we can use other sources, people might. There are also other tools and services that use this library field. Like Couchpotato (probably) and Media Center Master. I'm testing this apps to fill in blanks as we speak.

If you get your way, and Genres is being mapped to this field, I'll have to use a custom field because my logical choice was taken by another field which I ALSO require.

I can see both sides of this, and the only "solution" I see is the ability to custom map {metadata source}-[Field] to {MC}-[Field], which was discussed in another thread, so I will not bring it up here again.

I will only add that if I had to pick a side, I'd opt for the solution good for today vs. the solution that might avoid the potential problem that might be created at some point in the future, if more changes happen.  i.e. put Genre list in Keywords; but I certainly see the reasons for not wanting that.

[Matt]

I'm not loving the genres to keywords idea.

I'd probably rather work on making genre and artist list fields, instead of hacking around the not-a-list issue by using keywords.

[JimH]

... the only "solution" I see is the ability to custom map ...

This won't happen anytime soon (if ever).  We're trying to keep it simple.  Let's see if it will work.

[JustinChase]

Let's see if it will work.

It's definitely getting better 

[bil1010105]

This won't happen anytime soon (if ever).  We're trying to keep it simple.  Let's see if it will work.

Good luck with keeping it simple.  

To me, it seems inherently complex & wide-ranging. To make it a useful feature in the long term, it needs more configuration IMO. Even if that configuration is in the form of an API, which people can plug in to, and need never see a user interface. I just don't know what it's going to overwrite or populate without my knowledge. I don't trust it. Sorry.  

P.S. Sorry about the 'Bones' comment in the other thread.  

[glynor]

I'm not loving the genres to keywords idea.

I'd probably rather work on making genre and artist list fields, instead of hacking around the not-a-list issue by using keywords.

I came to basically the same conclusion on my way home tonight.

To me, the term "keywords" and the term "genres" are functionally synonyms.  They don't mean quite the same thing in English, but they're close enough that you can figure out from context what they are referring to, so you can treat them as equivalent in most circumstances.  But it seems like that terminology is what is causing a lot of the "issue".

MrHaugen (and probably others too) just don't want something that they feel should be called a "genre" to be called a "keyword".

Keywords are something specific to MrHaugen.  Something specific that transcends his use of MC.
The term "keyword" means literally nothing to me outside of the context of MC.  It does not bother me (and Justin) at all that the field [Keywords] may be filled with one type of data for Movies, and a different type of data for Photos or Home Videos.  In my views, I'd be able to tell what they are used for by context.  English is a "squishy" language.  Our vocabulary isn't very precise.  (It annoys the Germans I work with at the office too.)

Neither opinion is really wrong.  Just different interpretations.

I've been thinking about the "Artists" and "Genres" problem for a long time... The solution isn't easy.

Perhaps what we need is a way to "link" fields to one another so that they pre-fill.  For example, if you could have a list-type [Artists] field, that automatically pre-filled with the contents of [Artist] (if blank), then you would be able to use [Artists] in your views, but not have to worry about the fact that for most newly imported files it would be empty.  That's my big problem with using systems like that.  I've tried before with my own custom field, but it was way too much of a pain to manually fill the [Artists] tag for every single file I imported, when 95% of the time, what MC picks up on its own is good enough.

Likewise, you could actually do what Rick suggested before, and never directly tag any of the "stock" fields in MC (used in the views) with metadata from "the cloud".  All metadata lookup would map to fields specifically made for that metadata.  So, in this case, TMDb's Genres would map to a [Cloud Genres], or maybe even [TMDb Genres] field.  Then, in the Manage Library Field dialog for [Genre] you'd have a Link To: [Cloud Genres] setting.  If someone would rather link this to [Keywords] or whatever, they could just change the link.  You'd have a "custom mapping" solution right there, but no one would need to go looking for it unless it was doing something they didn't expect.

I don't know that this is perfect... Is the link persistent?  So, for example, using the [Artist]/[Artists] example, what happens if a file DOES have multiple artists added, but then the user changes the [Artist] tag.  What happens to the [Artists] field?  Does it stay untouched, or does it alter that "first value" and replace it with the new linked value?  The latter would probably be ideal, but it might be very difficult to implement.

I realize that none of this is happening now, but I'm just thinking out loud....

[MrHaugen]

I also just wanted to say, I don't feel like it is "heated".  I absolutely respect you, MrHaugen.

I respect you as well Glynor! This is not a problem at all. I just said it was heated because of the speed new replays popped in. Did not manage to write my response before one or two new posts popped up sometimes

I'd probably rather work on making genre and artist list fields, instead of hacking around the not-a-list issue by using keywords.

This would be a more logical choice imo. What would happen to our Genre today then? They would be preserved, right? And what about those that ONLY want to use single genres for Music? They'll have to just be careful not to choose more than one tag for each file?

Likewise, you could actually do what Rick suggested before, and never directly tag any of the "stock" fields in MC (used in the views) with metadata from "the cloud".  All metadata lookup would map to fields specifically made for that metadata.  So, in this case, TMDb's Genres would map to a [Cloud Genres], or maybe even [TMDb Genres] field.  Then, in the Manage Library Field dialog for [Genre] you'd have a Link To: [Cloud Genres] setting.  If someone would rather link this to [Keywords] or whatever, they could just change the link.  You'd have a "custom mapping" solution right there, but no one would need to go looking for it unless it was doing something they didn't expect.

THAT is an interesting thought! this could solve lots of problems in the future. People could virtually do what ever they want. There would have to be a default setup linking it all to logical fields, but letting the users control the mapping like this would be superb. It would solve all problems like we've discussed in this thread and more.

[struct]

This won't happen anytime soon (if ever).  We're trying to keep it simple.  Let's see if it will work.

This is a really disappointing statement.  This was my only hope for MC17.  Simple is good, but unless you do something close to the super set of what people want downloaded we are going to be restricted from getting what we want.  I don't understand the point of having the best database and video program out there if there is no way of getting the metadata.  Seems to defeat the purpose.  Giving us the superset of available metadata and letting us map it is just an interface, you can still keep it simple by having the default mapping as you see fit.

MC is much more than audio now.  Charge twice as much, one half for video, one half for audio, I don't care, a couple years of accelerated improvement in video is needed and metadata is critical.  Without this we are going to be cast into a halfway house of compromise for video and still have to use pvd or yannfoe.  Nothing else about JRiver is compromised, why this critical element?

Sorry to whinge, I know it is not helpful, I am just disappointed.

Craig

[JimH]

This is a really disappointing statement.  This was my only hope for MC17.  Simple is good, but unless you do something close to the super set of what people want downloaded we are going to be restricted from getting what we want.  I don't understand the point of having the best database and video program out there if there is no way of getting the metadata.  Seems to defeat the purpose.  Giving us the superset of available metadata and letting us map it is just an interface, you can still keep it simple by having the default mapping as you see fit.

MC is much more than audio now.  Charge twice as much, one half for video, one half for audio, I don't care, a couple years of accelerated improvement in video is needed and metadata is critical.  Without this we are going to be cast into a halfway house of compromise for video and still have to use pvd or yannfoe.  Nothing else about JRiver is compromised, why this critical element?

Sorry to whinge, I know it is not helpful, I am just disappointed.

Craig

Sorry you're disappointed, but this is a big new feature.  It does a lot that couldn't be done before.  Embrace it and use it before you start redesigning it.

[glynor]

Sorry you're disappointed, but this is a big new feature.  It does a lot that couldn't be done before.  Embrace it and use it before you start redesigning it.

I have to say, I agree.  The way it is working right now is utterly amazing on its own.  It is so simple, and it really does "just work" for what most people would want it to do, I suspect.  Most of the rest of this seems to be from "around the edges."

PS.  I'm not saying "those edges" aren't important.

[struct]

Hi Jim,

I have been giving the Get Movie/TV info a try out, and you are correct, it is a great improvement and is really a great tool.  Well done.

But I would like to put in a vote of encouragement for continuing the evolution of this tool.  I thought I would jot down the things I can't do that I would like to do.  I suspect that many of these types of partial updates will be desired by others who come with a large set of existing data but would like to get a bit more from your excellent tool (think those from XBMC, etc)..

1. Update only covers for a selection
2. Update on MPAA rating that is missing from 212 movies otherwise adequate data. I don't want to do a re-import as I will lose last played etc (won't I??), but am scared to do multiple select and update as all fields get overwritten and although it is excellent at guessing it isn't perfect.
3. Get rottentomatoes critic ratings for all files, but leave everything else intact?  At the moment we only get tvdb data on bulk import.

How might these be resolved without the super flexible solution asked for previously...

1. Simple tick box on bulk import for image only (similar to the tickbox on the individual import)
2. Simple list of fields that are about to be imported and a tick box on each (solid tick, replace/add data; grey tick, replace only if empty; no tick, leave alone).  This could also solve (1) with image a tick box in this list.
3. On bulk import, choose source and then preferably have list as per item 2.

This is a pretty simple dialog and given the relatively small number of imported fields, is a pretty manageable list. It increases the flexibility for those of us with a biggish set of good data that needs a tweak, and will give current users some flexibility to update as you add fields, and expand options.

Thanks
Craig

[MrHaugen]

2. Update on MPAA rating that is missing from 212 movies otherwise adequate data. I don't want to do a re-import as I will lose last played etc (won't I??), but am scared to do multiple select and update as all fields get overwritten and although it is excellent at guessing it isn't perfect.

Nothing should get overwritten except for Name or Title. So, if this is the only field you're missing, then it will update only that one.

Other than that, I do completely agree. Automatic scraping is not complete, and I hope more work is done here. Examples of missing data is Critics rating (rottentomatoes), TMDB rating, aired date, genre for series to mention a few.

[struct]

Just did a double check, if you do a multiple select and a get movie/tv info request, all data is overwritten.  I think this was stated as intended behaviour.  It is only the autoimport that won't overwrite existing data. 

Would be good to have a toggle to update only / overwrite upon multiple select and get movie/tv info request (but my suggestion to solve item (2) would be more flexible).

Craig