INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Tag "TheTVDB Series ID" is being cleared with new recordings  (Read 5042 times)

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186

Okay, I'm following the thought process from my post over here: https://yabb.jriver.com/interact/index.php/topic,111156.msg768747.html#msg768747 but I didn't want to muddy the waters further in that thread.

I think something has been broken in the way tagging is occurring for new recordings, which probably isn't related to CARNAC issues, but could be related to any "Get Movie & TV Info.." functionality changes.

For some reason I haven't been able to determine, my TV recordings of the series "Doctor Who" is having the tag [TheTVDB Series ID] cleared in all recordings whenever a new recording for that series is made.

Some background:
I record the series which is identified on TheTVDB as "Doctor Who (2005)" (TheTVDB ID 78804).

This series is named "Doctor Who" in my library, because that is the name used in all EPG data.

If I use any other series name, then new recordings would not be associated with the existing recordings, but would be considered a new series. For example, if I called the series "Doctor Who (2005)" to match TheTVDB name, a new recording would be given the [Series] name "Doctor Who" from the EPG data, and so would be a different series, and so would not get the tag [TheTVDB Series ID] set correctly to 78804, and therefore the "Get Movie & TV Info..." function would find the incorrect 1960's series "Doctor Who" (TheTVDB ID 76107) and the wrong episode data.

I have, in the past, searched for the series in TheTVDB using the series name "Doctor Who (2005)" which finds the correct Series and Episodes, and saved TheTVDB ID to the tag for all recordings and files from other sources in the series that the file selected belonged to. This works as the [TheTVDB Series ID] tag is a relational tag, and can only have one value for each [Series] name.

If the correct value is in the [TheTVDB Series ID] tag, and lookup I do using the "Get Movie & TV Info..." function it works correctly, giving priority to TheTVDB ID during a lookup, and finding the correct metadata for episodes in the series "Doctor Who (2005)", even though the series is called just "Doctor Who" in my library.

Setting TheTVDB tag used to work beautifully, as Glynor explained, and allowed the series named to be change to whatever you wanted, because MC used TheTVDB ID to look up TheTVDB, and not the series name.


The problem. New behaviour:
For some time now, every time a new episode of "Doctor Who" is recorded, the [TheTVDB Series ID] tag is deleted from all episodes of that series in my library.

Maybe the changes around saving of tags for recordings from some time back included the [TheTVDB Series ID] tag being saved based on EPG data, and as that was empty, it overwrote the tag for the whole series. I think those changes eventually included "save all tags, even those that are empty", which would cause this problem, if the episode had the same series name as an existing series, and MC had already associated the episode with the existing series. This could have broader consequences where any EPG sourced series name has the same name as an existing series in the library. I can understand saving the [TheTVDB Series ID] tag where there is a value in it, as there is for some EPG sources, but I don't think it should be saved when it is empty.

Because of this problem, when MC does a "Get Movie & TV Info..." lookup it it finds the wrong metadata. This isn't noticed, because from memory no tags are overwritten when MC does the lookup at the end of a TV recording, so the data from the EPG is left alone, and looks correct. But of course no Cover Art is retrieved for that series and episodes, which I would like to have working automatically at the end of a recording process. That hasn't worked as expected for some time, without Auto Import running first, and not even then when this TheTVDB ID issue is in play.

Recommendation:
No relational tags should be saved when that tag is empty in EPG data. Make that a generic rule, rather than specific to the [TheTVDB Series ID] tag, as other or future tags will be affected.

I even have one custom relational tag for a series which is doing strange things for one series at the moment. Could be related to this problem.


As a side note, the changes being asked to be made to CARNAC over here: https://yabb.jriver.com/interact/index.php/topic,111156.0.html will break the above process, in a situation where a user both imports episodes from a source of files, and records episodes OTA. The cause of the problem is that EPG data uses a different series name to TheTVDB. That is unavoidable, so in a mixed episode source environment, user intervention to change the series name will be required. In may case, if I ever imported a Doctor Who episode from a file, and it was given the [Series] name "Doctor Who (2005)", I would change that to just "Doctor Who" and all would be good.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #1 on: July 10, 2017, 12:07:41 am »

Okay, scratch all the stuff below, left for reference.

I have found that other programs using the "FTV" field are working correctly. The only difference I can see between programs that work, and the one that doesn't, is that for "ABC News" series which doesn't work the Recording Rule name "ABC News Daily" is different to the [Series] name. The EPG data uses "ABC News".

Could that be causing the problem? If so I guess this is a corner case. But still the capability to name a recording rule differently to the series name is good functionality, much appreciated when it was provided. Also, this doesn't account for trouble with the [TheTVDB Series ID] tag example with "Doctor Who", where the tag was cleared for all previous recordings of the program.

I have tried editing and saving the rule to see if that fixed the problem. No. So just now I have changed the recording rule name to match the series name, "ABC News" to see if that fixes the problem with tonight's recording.

EDIT: I also changed the rule name of one of the working rules to see if it fails.

Any other ideas, or anyone else seeing something like this?


I'm still seeing this problem. I've provided a non-[TheTVDB Series ID] tag example below and in the attached image, but I believe if this is happening for others, it will be affecting series selection on TheTVDB for certain programs.

My example is the [Series] "ABC News". I have added an Integer, Check edit, relational field to view my favourite programs in one specific view, rather than all programs. The field is called [FavouriteTV] with a Display Name of "FTV", and has only one value for each [Series], as defined in the "Manage Library Fields" dialogue. I restrict files to display in my "Shows" View in Theatre View using the simple rule "[FavouriteTV]=1".

When I tick the "FTV" Check box all programs in the series also have the "FTV" field set to "1", or checked. All programs should then appear in my "Shows" view. This used to work as I relied on it for the "ABC News" and other program recordings. However, it stopped working at some time. Now any new programs do not get the "FTV" field checked, even though the [Series] is checked, and all earlier programs have the "FTV" field checked. See the image to understand what I mean.

This example is a bit different to the [TheTVDB Series ID] tag in that programs that already have a check in the "FTV" tag do not have that tag cleared. But the relational tag isn't applied to the new programs either, and it used to, and should be.

I'm currently on MC23.0.18 and the problem is still occurring.

Any ideas?


Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Yaobing

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10926
  • Dogs of the world unite!
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #2 on: August 02, 2017, 05:07:34 pm »

For some reason I haven't been able to determine, my TV recordings of the series "Doctor Who" is having the tag [TheTVDB Series ID] cleared in all recordings whenever a new recording for that series is made.


Quote

The problem. New behaviour:
For some time now, every time a new episode of "Doctor Who" is recorded, the [TheTVDB Series ID] tag is deleted from all episodes of that series in my library.


Sorry for not looking into this sooner.  Do you still have this issue?


Quote
Recommendation:
No relational tags should be saved when that tag is empty in EPG data. Make that a generic rule, rather than specific to the [TheTVDB Series ID] tag, as other or future tags will be affected.


I am quite confused.  As far as I can tell, [TheTVDB Series ID] is not available in all EPG modes.  It is available in some XMLTV data.  If so, we do save the data in the corresponding EPG entry.  PercData, on the other hand, does not provide this, or at least the way my parsing code does, it does not. 

When a show is recorded, we copy field values from the EPG entry to the recorded file only if the field is not empty.  That therefore should not have emptied the field for existing recordings.

Could that have happened when an EPG entry is updated?  I suppose it could.  If the EPG data for a show does not have [TheTVDB Series ID], we do clear it.  This is done for all fields because it is possible that a TV station may replace a show with an entirely different show, and leaving any old data would cause confusion.

Another related question you previously asked was "when does 'Get Movie & TV Info' happen" after a show is recorded.  I told you I would look into it.  I am however still confused.  I can swear I have not done any coding myself about it, and I can not find any code (that other people did) that does it either.  So as far as I am concerned, automatic execution of "Get Movie & TV Info" does not happen.   ? ? ? ?

Even though PercData does not provide [TheTVDB Series ID], I do find some of my EPG entries carry this data.  Most of the shows that do carry this ID seem to be those that I have recorded in the past.  Maybe that explains something?  Maybe auto execution of "Get Movie & TV Info" does happen?  And such execution puts TheTVDB ID into the recorded shows?  And then they spread into even the new EPG entries with the same series names?  However, more confusingly I found a few movies (not TV series) that carry this ID too ("Money Talks" and "Shanghai Noon" both have the ID 97618, and "Tyler Perry's Madea Goes to Jail" has 72368) .
Logged
Yaobing Deng, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #3 on: August 03, 2017, 12:53:14 am »

Right, this will take a little while to work through. Yes, it is confusing. But there is some good stuff here, that can fix and improve TV in MC.

Sorry for not looking into this sooner.  Do you still have this issue?

I still have an issue. But not the Doctor Who one, because the season has finished. I still have the issue with the "ABC News" recording. But it is a strange one, and different to the Doctor Who issue. I'll deal with that in a second post.

But first...

Another related question you previously asked was "when does 'Get Movie & TV Info' happen" after a show is recorded.  I told you I would look into it.  I am however still confused.  I can swear I have not done any coding myself about it, and I can not find any code (that other people did) that does it either.  So as far as I am concerned, automatic execution of "Get Movie & TV Info" does not happen.   ? ? ? ?

This does not surprise me. But it is supposed to happen as part of the recording process, isn't it? I haven't looked up when I asked that question, or what I said around the issue then, but I have pretty much concluded that it doesn't happen until Auto Import detects the new program, and runs it then. The trouble with that is that the TV recording process creates a record in the MC Library, and so Auto Import may not immediately detect that it is a new program. Some of my recordings seem to never get a run of "Get Movie & TV Info..." against them, until I do it manually.

I have also noticed that when I record a new Series, or the first Episode of a new Season of a Series, that Cover Art isn't immediately collected for the Series and/or Season. I often have to run the manual process to get it. But I have seen that is can take quite a while to get the Cover Art when I import a whole new season, or multiple seasons, of a series into MC from other sources. So maybe the TV recording process isn't doing that either, and Auto Import takes a day or more to get around to collecting it. Not sure.

Anyway, I haven't done a controlled experiment on new TV recordings, with the "Get Movie & TV Info..." function turned off in Auto Import, or the whole function turned off to check exactly what gets down with a recording, because no-one else seems to be commenting on this issue.


I am quite confused.  As far as I can tell, [TheTVDB Series ID] is not available in all EPG modes.  It is available in some XMLTV data.  If so, we do save the data in the corresponding EPG entry.  PercData, on the other hand, does not provide this, or at least the way my parsing code does, it does not. 

I am using XMLTV data source OTA using EPG Collector, which also enhances the EPG data with metadata from TheTVDB. Theoretically my EPG could include the TheTVDB ID for programs which find a match. But I just checked, and I don't have the data in my current EPG, nor could I find evidence that it has been in the past. But I have deleted old data from when I looked into Doctor Who, for example. I think I can assume though that my EPG data will not have TheTVDB ID in it, and if it does, it will have the correct value, in which case writing it to the recording record, which will then update all other Episodes in the Series, would be fine. The problem occurs when the [TheTVDB Series ID] is empty in the EPG.


When a show is recorded, we copy field values from the EPG entry to the recorded file only if the field is not empty.  That therefore should not have emptied the field for existing recordings.

Could that have happened when an EPG entry is updated?  I suppose it could.  If the EPG data for a show does not have [TheTVDB Series ID], we do clear it.  This is done for all fields because it is possible that a TV station may replace a show with an entirely different show, and leaving any old data would cause confusion.

The text in blue should mean that the [TheTVDB Series ID] tag on the specific recording for a Series isn't emptied if the EPG data for the new Episode recording doesn't have it. But what does the code that handles Relational fields, such as the [TheTVDB Series ID] tag, do if the tag does not exist? Does it treat the record the same way as it does if the tag exists, but is empty? Because if the library record for an Episode of a [Series] is saved with an empty [TheTVDB Series ID] tag, all records for that [Series] have the [TheTVDB Series ID] tag emptied.

This could have been what I was seeing for Doctor Who. I think the problem started around the time of discussion around new programs getting data from old programs, which is why you started clearing the EPG data fields for new programs, as per the red text above, I think.

However this is happening, it breaks the "Glynor Method" of defining the [TheTVDB Series ID] tag once for a Series, as long as the EPG name and the name in MC match. i.e. EPG="Doctor Who", TheTVDB="Doctor Who (2005)", MC=Doctor Who". In that case, a lookup of TheTVDB will match on TheTVDB ID, and not on the program name.



Even though PercData does not provide [TheTVDB Series ID], I do find some of my EPG entries carry this data.  Most of the shows that do carry this ID seem to be those that I have recorded in the past.  Maybe that explains something?  Maybe auto execution of "Get Movie & TV Info" does happen?  And such execution puts TheTVDB ID into the recorded shows?  And then they spread into even the new EPG entries with the same series names?  However, more confusingly I found a few movies (not TV series) that carry this ID too ("Money Talks" and "Shanghai Noon" both have the ID 97618, and "Tyler Perry's Madea Goes to Jail" has 72368).

Are you suggesting that the EPG data gets updated with the [TheTVDB Series ID] if there are already recordings for the Series in MC? Or could this just be a coincidence?

I'm now quite sure that the "Get Movie & TV Info..." function does not happen for new Episode recordings. I just checked recordings I am still doing for three Series that are easily and accurately found on TheTVDB using the "Get Movie & TV Info..." function; "The Simpsons", "NCIS", and "NCIS: New Orleans". None of the Episodes for any of those Series have anything in the [TheTVDB Series ID] tag. The tag is empty for every Episode. As I know that the at least the NCIS programs had a value in the tag previously, it is obvious that something has cleared them. In fact, early Episodes of NCIS and The Simpsons have the correct Cover Art for their Epsiodes, so they would have had a manual run of the "Get Movie & TV Info..." function to populate those. Dates below show which Episodes have the correct Cover Art.

NCIS                           Correct Cover Art up to 2 June 2015. Video frame grabs from 2 February 2016 onwards. Latest 13 June 2017.
NCIS: New Orleans       All Episodes have video frame grabs, from 24 August 2015 to 30 July 2017.
The Simpsons              Correct Cover Art up to 23 May 2016. Video frame grabs from 17 September 2016 onwards. Latest 24 May 2017.

I just did a "Get Movie & TV Info..." lookup on just one Episode for each of those Series, and every Episode was updated with the correct TheTVDB ID, as expected.

Note that each of those Series is a favourite of mine, and so have my custom Relational tag [FavouriteTV] set to one (check mark) for all existing Episodes, and every new episode has picked up that value as it was recorded, the most recent only four days ago. So it would seem that the clearing of the [TheTVDB Series ID] tag is probably related to the EPG data, and clearing the tag by one means or another, as per the above discussion.

Note that as the EPG data would never have my Custom Relational tag, so it would not exist in the EPG data rather than be empty, it would appear that the problem is that the [TheTVDB Series ID] is being emptied, rather than being deleted from the EPG data. Or at least the tag is being saved for the new recording while empty, rather than not being saved with it at all.

I think that is enough for now. I will have another look at the issue I have with the ABC News program, and come back to describe it. I hope I haven't confused you more, and perhaps helped clarify the situation. Thanks for looking at this.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #4 on: August 03, 2017, 01:23:48 am »

One more thought.

If in the past the "Get Movie & TV Info..." function has been run for new TV recordings of Episodes of a Series that already exists in the MC Library, the following could have been happening.

  • A Series exists and all Episodes have the correct value in the [TheTVDB Series ID] tag.
  • A new Episode is recorded, and the EPG doesn't have a value for the [TheTVDB Series ID] tag. The [TheTVDB Series ID] tag is cleared (emptied, deleted, but still saved as an empty tag) for the new Episode, and hence updated for every existing Episode of the Series in MC.
  • The "Get Movie & TV Info..." function is run for the new Episode. A record is found using the Series, Season, and Episode numbers, or the Series and Episode names. The [TheTVDB Series ID] tag is saved for the new Episode, and so all Episodes in the Series have the [TheTVDB Series ID] tag updated to the new, correct value, which is the same as the previous, recently deleted value.

No-one would notice that the [TheTVDB Series ID] tag was deleted and then reinstated almost immediately if the correct Series was found on TheTVDB. Of course, this would break the "Glynor Method" of setting the [TheTVDB Series ID] tag correctly for a new Episode of Series which has a different name in MC and the EPG data from TheTVDB, as per my example above with Doctor Who. So I don't think that this is actually what is happening. I think that the changes around clearing EPG data so that new programs don't get old program EPG data has broken what used to work.

Which means, of course, that the "Get Movie & TV Info..." function was probably never being run for new TV recordings. Maybe deliberately, because if it was, it would be overwriting the program description (and other data, such as Genre, Keywords, etc.) from the EPG data, which some users may not want. Maybe even me.

In fact, I thought in the past that I had read that the "Get Movie & TV Info..." function run against TV recordings was different because it didn't overwrite existing tag values like the bulk manual (and maybe Auto Import) version of the function does. Of course, I would like the version run against a new recording to at least overwrite the video frame grab with the correct Episode Cover Art. Maybe there are some other tags that should be overwritten as well, if a record is found on TheTVDB.

I may have had or read a discussion with Hendrik on these issues of running the "Get Movie & TV Info..." function against new TV recordings at some stage in the past... maybe.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Yaobing

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10926
  • Dogs of the world unite!
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #5 on: August 03, 2017, 04:45:09 pm »

I was never sure running "Get Movie, TV Info" automatically at the end of a recording was a good idea.  Maybe as an option.

For now, I have made a change so that we will not empty "TheTVDB Series ID" in EPG if we are updating an existing entry that is exactly the same show (according to program ID), or at least is of the same series (according to series name).

This will leave a corner case: If a TV station replaces a show of "Series 1" with a show of "Series 2", and XMLTV data do not contain "TheTVDB Series ID" for  the new show, then it is cleared.  That means if you happens to have some existing recordings of "Series 2", their "TheTVDB Series ID" will be erased too.
Logged
Yaobing Deng, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #6 on: August 03, 2017, 07:01:26 pm »

Okay. I'll wait for the change to come through and check it out.

I was never sure running "Get Movie, TV Info" automatically at the end of a recording was a good idea.  Maybe as an option.

I agree that it is potentially fraught with problems. The EPG data may be better than the data on TheTVDB/Wikipedia, so you wouldn't want to overwrite the EPG sourced data on the recorded file. The lookup may find he wrong program first, overwrite the correct data, and then all information is lost until the program is identified by watching it and the metadata corrected with a manual lookup. If the program is obscure there may be no source of valid data, which would mean identifying the program and typing in the correct data. Difficult. But at the moment I don't get the correct Cover Art for Episodes until I manually run the "Get Movie & TV Info..." function. As I pad recordings quite a bit I tend to get screen grabs of Ads or completely different programs.

The solution for me would be to always have Cover Art overwritten when the "Get Movie & TV Info..." function is run at the end of a recording. Possibly there are other fields that should be overwritten, like first air date, or year. That would need some thought.

For me, I can manually run the "Get Movie & TV Info..." function whenever I want to and get the correct data. But for other users with a wife, kids, and other people using MC in the household, they just want a new recorded Episode to have the correct Cover Art and data without user intervention, I'm sure.



Just to be sure we are on the same page though, when you say;

For now, I have made a change so that we will not empty "TheTVDB Series ID" in EPG if we are updating an existing entry that is exactly the same show (according to program ID), or at least is of the same series (according to series name).

This will leave a corner case: If a TV station replaces a show of "Series 1" with a show of "Series 2", and XMLTV data do not contain "TheTVDB Series ID" for the new show, then it is cleared.  That means if you happen to have some existing recordings of "Series 2", their "TheTVDB Series ID" will be erased too.

You are talking about when updates are made for a time slot in the EPG program guide data, I believe.

In all of my discussion above I am talking about when a TV recording is created (files are created) and EPG data is copied across to the media file.

The difference is subtle, but it is still a difference.

I think the problem I saw with the three programs in my example above would be prevented if an empty Relational tag ([TheTVDB Series ID] or my [FavouriteTV] tag) is never saved to a new TV recording file, regardless of what is saved or cleared in the EPG data.

The fact that relational tag functionality works for my custom relational tag, [FavouriteTV], shows that a new TV recording record can be created without saving the tag with an empty value, which is all that is needed for the [TheTVDB Series ID] to work correctly for new recordings. I think.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Yaobing

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10926
  • Dogs of the world unite!
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #7 on: August 03, 2017, 07:54:01 pm »

Just to be sure we are on the same page though, when you say;

You are talking about when updates are made for a time slot in the EPG program guide data, I believe.

In all of my discussion above I am talking about when a TV recording is created (files are created) and EPG data is copied across to the media file.

The difference is subtle, but it is still a difference.

I think the problem I saw with the three programs in my example above would be prevented if an empty Relational tag ([TheTVDB Series ID] or my [FavouriteTV] tag) is never saved to a new TV recording file, regardless of what is saved or cleared in the EPG data.

The fact that relational tag functionality works for my custom relational tag, [FavouriteTV], shows that a new TV recording record can be created without saving the tag with an empty value, which is all that is needed for the [TheTVDB Series ID] to work correctly for new recordings. I think.

Yes I am talking about EPG "program".  These entries are in a separate database "guide", as opposed to the main database.  Empty data are not copied over to the recordings.

Your custom tag is safe because we do not mess with it, neither in EPG entries, nor in recorded programs.  When recording finishes, non-empty tags are copied from the corresponding EPG entry to the recorded file.
Logged
Yaobing Deng, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #8 on: August 03, 2017, 08:29:53 pm »

Okay. I pretty much understood that from the discussion above. So I am still at a loss as to why the [TheTVDB Series ID] is being deleted for Series I am recording.

I have some new recordings coming up from Monday next week for a couple of those series. They all have the [TheTVDB Series ID] set for all Episodes now, so I will keep an eye on them and see if and when that changes.

Thanks for putting up with my long posts and questions.  :D
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Yaobing

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10926
  • Dogs of the world unite!
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #9 on: August 03, 2017, 09:06:27 pm »

Okay. I pretty much understood that from the discussion above. So I am still at a loss as to why the [TheTVDB Series ID] is being deleted for Series I am recording.


I can confirm that clearing that tag in EPG entries does clear the corresponding tag in recorded shows.  Hopefully with my latest change this will improve.
Logged
Yaobing Deng, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #10 on: August 04, 2017, 11:07:52 pm »

I can confirm that clearing that tag in EPG entries does clear the corresponding tag in recorded shows.

That gave me something to work with, so I was going to do some tests. However, without doing anything I found that two of my three test Series had already had the [TheTVDB Series ID] tags deleted from previously recorded Episodes. The third show hadn't lost its [TheTVDB Series ID] tags. The obvious difference is that the third show hasn't appeared in the EPG since my earlier testing. None of the shows have been recorded since then. The two shows that lost their [TheTVDB Series ID] tags do appear in the "To Be Recorded" list.

So just an update of the EPG which includes the Series in question clears the [TheTVDB Series ID] tags for all recorded Episodes. I assume it only happens if the EPG feed doesn't include the [TheTVDB Series ID] tag for the scheduled program.

So a quick test on MC23.0.21:
  • I added [TheTVDB Series ID] tags back to the existing recordings of one of the Series I am testing, The Simpsons.
  • I waited for the tag changes to finish saving, refreshed the view, and checked that the [TheTVDB Series ID] tags remained.
  • I checked my custom EPG view, and saw that the [TheTVDB Series ID] tags now existed against all upcoming Episodes of The Simpsons.
  • I deleted the [TheTVDB Series ID] tag from one EPG record for The Simpsons. The tag was removed from all EPG records immediately.
  • I check the previously recorded Episodes of The Simpsons. All had lost their [TheTVDB Series ID] tags, even without refreshing the view.

So that sets the baseline functionality that was causing the problem. I know this did not occur in the past, so something changed to cause it to happen now. I'm surprised that no-one else has noticed, but I guess for most Series, the "Get Movie & TV Info..." function just puts it back again, if the Series is easily found on TheTVDB. Although, the "Get Movie & TV Info..." function isn't being run for TV recordings!

Anyway, onward.
  • I added the [TheTVDB Series ID] tags back to The Simpsons.
  • I then upgraded to MC23.0.31, which includes your fix.
  • I checked The Simpsons recordings and EPG data still had their [TheTVDB Series ID] tags.
  • I deleted the [TheTVDB Series ID] tag from one EPG record, and it was removed from all records.
  • I check the previously recorded Episodes of The Simpsons. Again, all had lost their [TheTVDB Series ID] tags, even without refreshing the view.

I can understand why the Relational Field functionality does that. But I don't think such a change should cross the boundary from EPG data to TV recording files. It means that if ever my EPG is less than perfect, and doesn't have the [TheTVDB Series ID] data in it, the [TheTVDB Series ID] tags will be cleared from all my recordings.

I think the functionality that we need is;
If the "TheTVDB Series ID" does not exist in the EPG data, or it exists but is empty, do not save the [TheTVDB Series ID] tag to a new recording file. In that case, if existing recordings for the Series do have a value in the [TheTVDB Series ID] tags, then the Relational Field functionality will (should) add the [TheTVDB Series ID] tag to the new recording.

However, now that I have upgraded my MC Server to MC23.0.31, I will see what happens tomorrow morning, after an EPG update has occurred.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #11 on: August 05, 2017, 09:03:51 pm »

I ran a manual EPG collection last night and let the automatic one run this morning. The [TheTVDB Series ID] tags weren't deleted from my three test Series in either case.

I did find a lot of older recorded Series that would have had [TheTVDB Series ID] tags in the past that no longer do. So I assume that they had appeared in the EPG while this problem existed, and the tags were deleted.



For now, I have made a change so that we will not empty "TheTVDB Series ID" in EPG if we are updating an existing entry that is exactly the same show (according to program ID), or at least is of the same series (according to series name).

This will leave a corner case: If a TV station replaces a show of "Series 1" with a show of "Series 2", and XMLTV data do not contain "TheTVDB Series ID" for  the new show, then it is cleared.  That means if you happens to have some existing recordings of "Series 2", their "TheTVDB Series ID" will be erased too.

I'm not convinced this is the right long term solution, because the corner case only has to happen once and the [TheTVDB Series ID] tags are deleted. I don't think the corner case will be that uncommon. Without [TheTVDB Series ID] tags The TVDB lookups will lose the speed and accuracy advantage the tag affords. Without the "Get Movie, TV Info..." function being run against TV recordings the tags will not be replaced without user intervention.


Coming back to this issue;
I was never sure running "Get Movie, TV Info" automatically at the end of a recording was a good idea.  Maybe as an option.
I think whether to run the "Get Movie, TV Info..." function could be decided on a Series by Series basis. If the Series data is good on TheTVDB site, then using that data via a lookup makes sense. If the data isn't good it doesn't make sense to risk a bad match. So perhaps the option to run the function should be added to the Recording Rule, similar to the tagging option already there. I would probably want the option to only collect an Episode image for some Series, or program information, or both, similar to the current options when multiple files are selected.

I do like to get Episode Cover Art, and I would like it to be automatic, which requires the "Get Movie, TV Info..." function.


Now that this fix is in I am going to see if I can get the "TheTVDB Series ID" data into my EPG stream. Then I will have a look at the issue I have with my "ABC News" recordings.

Once again, sorry for the long post.  ;)
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #12 on: August 06, 2017, 11:18:44 pm »

So far all is good.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Yaobing

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10926
  • Dogs of the world unite!
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #13 on: August 07, 2017, 07:27:16 am »

So far all is good.

Good, except the corner case of course.

In order to solve the corner case, I will have to figure out how to cut off EPG database from the main database regarding relational fields.
Logged
Yaobing Deng, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #14 on: August 07, 2017, 04:22:21 pm »

Yes. I love the ability to use the "Get Movie & TV Info..." function against an EPG record, to enhance the data MC uses to create a recording. But other than that and direct manual editing of the EPG data I don't really want updates of the EPG data from other MC areas. The data flow should really be in one direction only: EPG => MC.

Thanks very much for the time you have spent on this Yaobing. I am quite surprised no one else has spoken up, or mentioned it elsewhere. I guess I am just the most pedantic user!  :D
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #15 on: August 07, 2017, 04:36:13 pm »

By the way I have found and fixed, I think, the "ABC News" issue mentioned above. I had a recording rule that added the [series] tag to the program, as it has never had a sub-title in the EPG data, just the Title which is the series/program name. So that was always saved in the [name] tag. For some reason I don't know, the tagging stopped working. Of course, without the [series] tag my Relational tag, [FavouriteTV] stopped working.

Then as part of the testing, I completely deleted the recording rule and recreated it, and forgot to add back in the tagging part. Once I added that back in yesterday it worked again last night. Hopefully the original cause has gone away, and it will keep working. I shall watch that.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #16 on: August 11, 2017, 05:43:52 pm »

Thanks for this change Yaobing. I didn't get to install MC23.0.33, but I'm about to install MC23.0.34, so will see how this goes. Hopefully, problem permanently solved.

23.0.33 (8/8/2017)

1. Changed: Change made in TV Guide database of "TheTVDB Series ID", or any field with relational type FIELD_RELATIONAL_TYPE_SERIES, will no longer be propagated to the main database directly.


Thanks for this one as well. It was sort of bugging me. But not enough to investigate and write a post.  ;D

23.0.34 (8/11/2017)

6. Fixed: When running "Rename, Move, & Copy" tool with multiple JTV files, some files could be copied into the same time-shifting folder, causing Windows popup complaining about "destination file already exists".


PS:
I just checked my three test series, and all of them had lost their [TheTVDB Series ID] tags despite the change in MC23.0.31. Several other TV series that had appeared in the EPG had also lost their tags. I shall have to install MC23.0.34, add back the [TheTVDB Series ID] tags, and see how this new change goes.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #17 on: August 12, 2017, 08:07:32 pm »

MC upgraded to MC23.0.34 and TheTVDB tags added back to quite a few series, just now.

I shall watch and see how this goes.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Tag "TheTVDB Series ID" is being cleared with new recordings
« Reply #18 on: August 14, 2017, 09:52:37 pm »

A couple of days and EPG data loads later I still have my [TheTVDB Series ID] tags. Excellent.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner
Pages: [1]   Go Up