INTERACT FORUM

More => Old Versions => JRiver Media Center 30 for Windows => Topic started by: Hendrik on February 03, 2023, 09:10:32 am

Title: NEW: Additional video file metadata
Post by: Hendrik on February 03, 2023, 09:10:32 am
Coming in 30.0.60, Media Center will import additional metadata from video files for improved filtering and future use in Tooltips and Theater View.

Preview:
(https://i.imgur.com/5wWUB4m.jpg)

Details of all fields:
Video
Audio
- For audio, all fields (except Audio Streams) are list fields with one value per stream. These lists may contain empty values, if a stream does not convey this information.
Subtitles
- Similar to audio, these fields are list fields with one entry for each subtitle stream

How do I use this?

The new information is automatically gathered when a video file is imported. Additionally, running the "Update Library (from tags)" tool will also populate these fields.
Additionally, there is a "Analyze Video" button under Library Tools to gather this information quickly on existing files.

This data can be imported from all local video files. Functionality with internet streams or other streaming sources will be very limited.
DVD folder structures (IFO) are not currently supported, but will be added in a future update.

Future changes

We will be expanding these features in the near future. Coming up next:

- DVD folder structures (IFO)
- Bitrate values are only populated when the file has built-in metadata to provide this data, we plan to look into measuring the values in absence of metadata
Title: Re: NEW: Additional video file metadata
Post by: mwillems on February 03, 2023, 09:15:50 am
I'm really excited to have subtitle data available in the metadata.  I often watch video in environments where subtitles are necessary, but not all of my videos have subtitles and I never figured out a good way to create a view to show which files have subs and which don't.  That also complicated the process of finding 3rd party subtitles (as I couldn't readily make a list of videos that needed them, so I'd just handle it when I encountered it).  Now both those tasks should be easy to do.  Thanks so much!
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 03, 2023, 12:04:41 pm
Awesome, thank you Hendrik :)

The new fields did not show up by default in my Tag Editor - I didn't have the "All Fields" section enabled (not sure if I removed that some time ago or if it's the default). So I had to customize the editor to add the section, as I don't see an easy way to add a "Video" section like you have on your screenshot. They also don't show up on the "Fields with values" section, which may be a bug?

Would it be possible to add "Video Info" and "Audio Info" sections to the Tag panel with the relevant fields already added? It would be nice that we could add/remove those sections just by selecting them. The "Add" button could have "Add section -> Video/Audio/Subs/File/etc", each with a predefined list of fields.
Title: Re: NEW: Additional video file metadata
Post by: eve on February 04, 2023, 02:09:08 am
Subtitle metadata is a *really* nice touch.

I've had to resort to reading mediainfo for items separately to establish what the 'index' is for audio and subs to use with my own control interface  ::)
This simplifies things a little since I guess I can just read the list from JRiver itself and figure out the index that way.


Title: Re: NEW: Additional video file metadata
Post by: amandalishus on February 04, 2023, 04:33:14 pm
Y'know, I've been using MC for well over a decade now... One might think that eventually you'll run out of ways to wow me with a new feature, and yet you keep doing it regularly and reliably, year after year. In fact, I've learned that if I haven't been blown away by a new feature in the last couple of releases, I need to go back to the changelog for each build and see what I missed.

This time it's the audio streams... I'm a sucker for the commentary track on movies I like, and I've invested quite a bit of time and money adding audio streams to movies for which I've purchased RiffTrax (http://"https://www.rifftrax.com/") as well. I've added custom database fields to help me keep track, but that only works when I remember to populate them. I've had much the same problem with subtitles, since I started trying to collect them when I recently became friends with someone who is losing their hearing. He's already quite impressed with the scope of options MC offers for displaying subtitles, but I'm sure it will make him even happier once I don't need to start the movie to determine whether I have them or not.

Anyway, thanks again for your ceaseless work improving what was already the best media library and player around  :-* I may have to apply for the beta channel if Build 60 isn't out by Valentine's Day  ;D
Title: Re: NEW: Additional video file metadata
Post by: eve on February 04, 2023, 05:15:41 pm
Y'know, I've been using MC for well over a decade now... One might think that eventually you'll run out of ways to wow me with a new feature, and yet you keep doing it regularly and reliably, year after year. In fact, I've learned that if I haven't been blown away by a new feature in the last couple of releases, I need to go back to the changelog for each build and see what I missed.

This time it's the audio streams... I'm a sucker for the commentary track on movies I like, and I've invested quite a bit of time and money adding audio streams to movies for which I've purchased RiffTrax (http://"https://www.rifftrax.com/") as well. I've added custom database fields to help me keep track, but that only works when I remember to populate them. I've had much the same problem with subtitles, since I started trying to collect them when I recently became friends with someone who is losing their hearing. He's already quite impressed with the scope of options MC offers for displaying subtitles, but I'm sure it will make him even happier once I don't need to start the movie to determine whether I have them or not.

Anyway, thanks again for your ceaseless work improving what was already the best media library and player around  :-* I may have to apply for the beta channel if Build 60 isn't out by Valentine's Day  ;D

I'm really glad it happened! It does seem like users of JRiver are more likely to place a value on, and actually have, audio commentaries due to the fact that JRiver incorporates robust BDMV ripping and playback.
Considering that there's a pretty strong set of remote control options for JRiver, it was always a weird deficit that I couldn't 'read' what tracks were available for a currently playing item.

Thanks again Hendrik!
Title: Re: NEW: Additional video file metadata
Post by: eve on February 04, 2023, 06:08:10 pm
For clarity I have a question.

I use MakeMKV for ingest (well sort of) and I try to tag most relevant audio / sub tracks with meaningful info. This I find is very helpful, especially once I start muxing in other audio mixes.
A release may have the best video, but it's audio may be lacking or I may prefer say, a theatrical audio mix, not to mention licensing issues for various commentary tracks recorded over the years which prevent their inclusion in releases. I also tend to run OCR on image based subs since rendering text based subs leads to much nicer, more consistent results.

I assume that the new "Title" field for audio and sub tracks will match what I've labelled a track in MakeMKV?

Title: Re: NEW: Additional video file metadata
Post by: jmone on February 05, 2023, 02:57:56 am
Looking good!  While most of my HDR content is UHD BD rips that I'll check with later when supported, I've run it over my HDR Home Videos and compared it to what I've been pulling with SOT (Media Info).  Couple of things:
- HDR Mastering Color Space:  Seems to be blank (was Expecting "Display P3")
- A new field for "Scan Type"?: eg Interlaced vs Progressive

Title: Re: NEW: Additional video file metadata
Post by: jmone on February 05, 2023, 03:08:06 am
...and it is quick!
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 05, 2023, 03:41:58 am
I assume that the new "Title" field for audio and sub tracks will match what I've labelled a track in MakeMKV?
That is correct
Title: Re: NEW: Additional video file metadata
Post by: eve on February 05, 2023, 03:48:15 pm
That is correct

Okay this kicks butt.

Thanks!!!

I can probably rip out this terribly inefficient code that I use for sub and audio selection from a tablet.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 14, 2023, 04:06:26 pm
The next build has a new entry in Library Tools to only run the new analysis on existing files, without messing with existing tag data etc.

Further changes with more data will be coming shortly as we update some external libraries with added features.
Title: Re: NEW: Additional video file metadata
Post by: comox on February 14, 2023, 06:46:52 pm
Please consider adding to future changes the ability to update tags from library for a few key fields like [Name], [Date], and [Genre].

This would improve the viewing and organizing experience of videos exported to devices like phones.

It also would make MC unique since no apps that I'm aware of do a nice job of writing video tags.

I sometimes wonder if this omission was an quietly agreed convention between app developers since making it easy to write video tags would break video torrents.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 15, 2023, 01:30:21 am
We do not currently plan to write tags to video files, sorry. Software support for writing video tags is very limited, especially without re-writing the entire file, which would take quite some time for video files.

This is also rather off-topic for this thread, please open a new one or use an existing one that is more appropriate.
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 15, 2023, 05:38:37 am
Thanks Hendrik!

(nonsense, skip to next post)
Looks like this EAC3+Atmos variant is not yet identified, MC tags it with [Audio Codec] = "E-AC3", but empty [Audio Title]. Atmos is not mentioned anywhere:

Audio #1
ID                          : 2
Format                      : E-AC-3 JOC
Format/Info                 : Enhanced AC-3 with Joint Object Coding
Commercial name             : Dolby Digital Plus with Dolby Atmos
Codec ID                    : A_EAC3
Title                       : (empty)


On a few files the [Audio Title] gets populated with a mix of tech info and track comentary - perhaps this should be 2 different fields, [Audio Title] and [Audio Commentary] ?

[Audio Codec] : DTS-HD MA;AC-3
[Audio Title] : DTS-HD Master Audio 5.1;Commentary from 2003 by Director Ridley Scott, Writer Dan O'Bannon, Executive Producer Ronald Shusett, Editor Terry Rawlings and Actors Sigourney Weaver, Tom Skeritt, Veronica Cartwright, Harry Dean Stanton and John Hurt


On some other files the [Audio Title] is completely empty, perhaps because MC is reading an empty commentary string into the field.



@Comox: you can add a button to the MC toolbar to run something like this on the selected file:
mkvpropedit "[filename]" --set "title=[Name]"

(sorry for another off-topic Hendrik)
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 15, 2023, 05:42:35 am
Sorry, I think I misunderstood - the "DTS-HD Master Audio 5.1" and "English TrueHD Atmos 7.1" also comes from the file metadata right? I thought it was generated by the video analyzer.
In that case, is it possible to write the "Atmos" info to some other field as many files won't have that in the Title metadata? Perhaps the [Audio Codec] should include it.
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 15, 2023, 05:49:50 am
Title is parsed AFAIK as it is from the name field just like IMO it should be done. Some people have been inserting also technical attributes of stream to the name field and that's what MC is then getting as title.

I think that object audio support in this context is not yet implemented. My assumption has been that there will either be own field for object audio or it will be part of the audio codec but that is what Hendrik can confirm or deny

By default MakeMKV for example uses channel count for audio names/titles like: "Surround 7.1", "Surround 5.1", "Mono", "Stereo". I tend to remove these and add info about what is the purpose of track instead like "English Dub", "Cantonese Mix", "Audio commentary with...", "Original Stereo Mix" etc because those channel counts, bitrates, codecs and stuff can be read from the file data
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 15, 2023, 06:27:44 am
Sorry, I think I misunderstood - the "DTS-HD Master Audio 5.1" and "English TrueHD Atmos 7.1" also comes from the file metadata right? I thought it was generated by the video analyzer.
In that case, is it possible to write the "Atmos" info to some other field as many files won't have that in the Title metadata? Perhaps the [Audio Codec] should include it.

Future changes

We will be expanding these features in the near future. Coming up next:

- Detection of object-based "immersive" audio codecs like Atmos and DTS:X

 ::)

Detection of these needs an update of an external library, which is always a lengthy process for all platforms etc, so I wanted to roll some more updates into one, so it'll be coming soon.
Right now I'm thinking it'll just get added to the Audio Codec field, eg. "TrueHD Atmos", "DTS-HD MA DTS:X" (or just DTS:X? It can't really come without being DTS-HD MA), "E-AC3 Atmos" .. something like that. I'll take formatting suggestions.

The title is indeed just a plain copy from the file, and not generated, or even looked at.
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 15, 2023, 06:42:42 am
Reading comprehension issues, clearly  ;D
TrueHD Atmos / E-AC3 Atmos / DTS:X sounds good (pun intended).
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 15, 2023, 06:51:01 am
Naming sounds good to me
Title: Re: NEW: Additional video file metadata
Post by: rec head on February 15, 2023, 07:29:21 am
Thanks, this will make zone switching for Atmos even easier.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 17, 2023, 06:53:57 am
The next build will add Atmos/DTS:X detection, as well as implementing Crop/Black Bar detection. The values are not used by the renderer yet, that'll come later, but if any of you want to analyze some files and see if any produce odd/unexpected crop values, that would be useful!
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 17, 2023, 09:25:45 am
The new features are working, but the Analyzer is now taking 5-10 seconds per file, where before it was doing 2 or 3 files per second. This is likely due to the Video Crop detection; if so, is there a way to disable just that feature?

Also, please add an "Analyze Video" option to the Auto-import settings.
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 17, 2023, 01:51:40 pm
Shutter Island UHD DV
my ffmpeg crop: 3840;1632;0;264
MC video crop: 0x264x3840x1896
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 17, 2023, 02:47:19 pm
Same info, different format. MC has start and end point, you have the movie area and the start point.
I like the list format though.
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 17, 2023, 02:49:03 pm
Cropping is wrong in MC in that case or is the format different besides in different order. I though there should be removal from y (1896) also
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 17, 2023, 02:51:56 pm
1632 + 264 = 1896

You: Width, Height, StartX, StartY
MC: StartX , StartY, EndX, EndY

StartY + Height = EndY
Title: Re: NEW: Additional video file metadata
Post by: jmone on February 17, 2023, 03:05:24 pm
Nice  ;D It's looking pretty good (only tested on "Files" so far but most of my library are BD rips):
- Mastering Display Color Primaries:  Now being populated correctly
- Atmos / DTS-X etc: working!
- Video Crop: working well enough for me..... but I'm not personally that worried about this feature yet as I only have 16:9 displays.  There are some differences but these are on files with very dark backgrounds (eg a video of a Luna Eclipse etc).  The rest are either the same or very close to what I was getting with SOT.  Zybex came up with a calculated "Aspect Ratio (Crop)" for SOT and I'd guess that a reworked version of this would pave over any difference between the two methods.
- Speed:  I find it perfectly acceptable.  Sure it will take a bit to rescan the entire video library but once that is done, it will not be an issue for normal importation and tagging of items

Updated list of what I still see that would be good to have (of which I know are probably on your list)
- BD rip support
- Update the Bitstreaming Option to now also include Atmos/DTS-X etc
- Updating the "Compression" to use the new info (eg show Atmos/DTS-X extensions)
- Scan Type to show Interlaced vs Progressive etc
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 17, 2023, 06:57:54 pm
Cropping is wrong in MC in that case or is the format different besides in different order. I though there should be removal from y (1896) also

zybex is right, our format is a typical video rectangle, which is "left x top x right x bottom", the same format you will see in the JRVR OSD for example. Other tools might provide left/top and width/height, but the result is the same, just a different convention.

1896 is of course already a adjusted number, since the original is 2160. :)
Title: Re: NEW: Additional video file metadata
Post by: Manfred on February 18, 2023, 12:11:23 pm
Will the old fields disappear in the future, e.g. Sample Rate vs. Audio Sample Rate?
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 18, 2023, 02:29:10 pm
No, those are still used for audio files and will also continue to reflect the primary audio stream.
Title: Re: NEW: Additional video file metadata
Post by: Manfred on February 18, 2023, 04:38:58 pm
It would be nice to have the same unit for both: Sample Rate is in khz Audio Sample Rate in Hz.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 18, 2023, 04:40:36 pm
Both store the same number, its just a visual difference, which is not currently possible to re-create in a list field like Audio Sample Rate.
Title: Re: NEW: Additional video file metadata
Post by: lello on February 19, 2023, 05:38:49 am
Shutter Island UHD DV
my ffmpeg crop: 3840;1632;0;264
MC video crop: 0x264x3840x1896

Same info for me too.

So, if I understand correctly, at the moment the presence of the black bands is detected (excluding BDs) but no cropping is done, right?

Will it be implemented in future builds?
Title: Re: NEW: Additional video file metadata
Post by: lepa on February 19, 2023, 05:56:44 am
Numbers are right. Currently AFAIK they are not utilized yet
Title: Re: NEW: Additional video file metadata
Post by: rec head on February 20, 2023, 08:01:43 am
I'm sure there is something simple I am missing but where do I view this information? I have run Analyze Video on an MKV but there is no Video in my Tag Pane when I view it.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 20, 2023, 08:43:25 am
Right now you would have to add the fields yourself. I have also updated the default tag window template for the next build, so you could just reset it to defaults to get it. Unfortunately there is no sensible way for us to "upgrade" existing tag windows transparently - the ever looming bane of having everything customizable by the user.
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 20, 2023, 09:08:13 am
Perhaps you could add a new tag Group containing all relevant tags so that current users can easily add it?
Title: Re: NEW: Additional video file metadata
Post by: mattkhan on February 20, 2023, 09:37:27 am
Giving the user an easy life is probably

1) Compare the tag window layout to the previous default, if same then upgrade it without asking
2) provide a mechanism for merging such updates

The former seems reasonable path for the user who doesn't tweak stuff, no clue what %age that is (I would guess it is "most people"?)
Title: Re: NEW: Additional video file metadata
Post by: rec head on February 20, 2023, 12:15:34 pm
I usually do a little poking around before asking a question. I hit Alt+Enter to open the Tag Pane on the file I analyzed. None of the new info was there so I tried the box in the upper left and selected Customize. I was surprised/confused that there was no Video Field for the Movies Template.
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 20, 2023, 12:34:16 pm
If you click "Add..." on the tag panel customization, all fields shown in Hendrik's screenshot (first post) are there.
Title: Re: NEW: Additional video file metadata
Post by: jmone on February 20, 2023, 02:04:05 pm
I'd like a Tag Window Template that shows all fields with data and hides all empty fields.  I presume it could be done with a custom template?
Title: Re: NEW: Additional video file metadata
Post by: Matt on February 20, 2023, 02:13:42 pm
I'd like a Tag Window Template that shows all fields with data and hides all empty fields.  I presume it could be done with a custom template?

Look at Add > All Fields With Values in the customization dialog.
Title: Re: NEW: Additional video file metadata
Post by: rec head on February 20, 2023, 06:18:32 pm
All Fields with Values did not add everything with values. Such as the new HDR info.
Title: Re: NEW: Additional video file metadata
Post by: rec head on February 20, 2023, 08:07:01 pm
If you click "Add..." on the tag panel customization, all fields shown in Hendrik's screenshot (first post) are there.

I'm using MC 30.0.65 and I can't find Audio Codec.
Title: Re: NEW: Additional video file metadata
Post by: HTPC Videophile on February 20, 2023, 11:07:16 pm


Updated list of what I still see that would be good to have (of which I know are probably on your list)
- BD rip support
- Update the Bitstreaming Option to now also include Atmos/DTS-X etc
- Updating the "Compression" to use the new info (eg show Atmos/DTS-X extensions)
- Scan Type to show Interlaced vs Progressive etc

++1 :) to all above with highest and immediate priority to - Updating the "Compression" to use the new info (eg show Atmos/DTS-X extensions) and include SDR/HDR info(Dolby Vision/HDR10/HDR10+ etc). And as an additional request : In the compression field , we use tags h264 and hevc for the appropriate encoding. Shoudn't we stick to one nomenclature  ?Either use h264 and h265 or use avc and hevc for the respective encodings instead of using a mixture of both for the sake of nomenclature consistency ? H.264 MPEG-4/AVC and H.265 HEVC instead of just  h264 would be better as it would be in line with rest of the vc1, mpeg2 tags. If h264 is an ffprobe output  limitation MC code can easily replace it with H264 MPEG-4/AVC  for uniformity while filling the compression field.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 21, 2023, 01:26:15 am
Shoudn't we stick to one nomenclature  ?Either use h264 and h265 or use avc and hevc for the respective encodings instead of using a mixture of both for the sake of nomenclature consistency

We will stick with whats more common in the industry, and thats the terms used already. The h26x names are relatively uncommon, with the exception of h264, which is far more popular then using AVC. There is no sense in using something for conformity if that's not what people know and use.
Title: Re: NEW: Additional video file metadata
Post by: HTPC Videophile on February 21, 2023, 02:39:36 am
We will stick with whats more common in the industry, and thats the terms used already. The h26x names are relatively uncommon, with the exception for h264 which is far more popular then using AVC. There is no sense in using something for its conformity if thats not what people know and use.

I won't further argue or press for it but FYI KODI uses H.264  and H.265 

Title: Re: NEW: Additional video file metadata
Post by: Matt on February 21, 2023, 07:38:05 am
All Fields with Values did not add everything with values. Such as the new HDR info.

That's because the fields are read-only.  I'll add a way to show read-only fields as well to a coming build.
Title: Re: NEW: Additional video file metadata
Post by: jmone on February 21, 2023, 01:54:59 pm
Thanks - It good to be able to see all the tags on an item.
Title: Re: NEW: Additional video file metadata
Post by: jmone on February 22, 2023, 02:10:43 am
Thanks (V30.0.66):
- Matt, new Tag to View all (including Read Only) is working well
- Hendrik, Scan Type also works well.
Title: Re: NEW: Additional video file metadata
Post by: jmone on February 23, 2023, 01:56:47 am
Couple of other areas in MC that needs an update to show the new Audio Info (eg ATMOS) is:
- Audio Path
- Stream Selection
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 28, 2023, 04:02:55 pm
The latest build now splits black bar detection out from the other information gathering, and also gives you an option to run it during import akin to audio analysis. Its still running as a separate process and is possibly prone to a timeout (even after increasing the allotted time), but maybe I should just remove the timeout there and make sure the canceling works.
Title: Re: NEW: Additional video file metadata
Post by: zybex on February 28, 2023, 04:09:14 pm
*decreased ?

Thanks Hendrik, good options :)
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on February 28, 2023, 04:12:42 pm
The timeout was increased, so it has more time to complete. :)
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 01, 2023, 05:36:46 am
The next build will also support Blu-ray directory structures (BDMV) but DVD structure rips (IFO) will take a bit longer.  We don't have as much support code ready for it.

I'm also working on bringing "immersive" audio codec detection to LAV, which is required to make it show up in the Audio Path, and help with more dynamic bitstreaming options.  That should be coming soon.
Title: Re: NEW: Additional video file metadata
Post by: lello on March 01, 2023, 12:15:25 pm
The timeout was increased, so it has more time to complete. :)

Audio analysis takes much more time :D
Title: Re: NEW: Additional video file metadata
Post by: eve on March 01, 2023, 04:04:16 pm
The next build will also support Blu-ray directory structures (BDMV) but DVD structure rips (IFO) will take a bit longer.  We don't have as much support code ready for it.

I'm also working on bringing "immersive" audio codec detection to LAV, which is required to make it show up in the Audio Path, and help with more dynamic bitstreaming options.  That should be coming soon.

Dude, you're killin it. Thanks! Glad to see "Immersive" codec detection coming to LAV!
Title: Re: NEW: Additional video file metadata
Post by: Ashfall on March 02, 2023, 08:08:26 am
This is really great!  I previously used a custom tag to manually fill video scan and HDR type for imported videos.  Really appreciate this.

I've noticed a few videos don't get the video scan field populated.  The only thing they seem to have in common is that they are 30 seconds.  Longer or shorter videos work fine.

Update:  after updating to 30.0.73 this is working.
Title: Re: NEW: Additional video file metadata
Post by: eve on March 05, 2023, 08:34:43 am
Is there any chance to have this populate for items that aren't in the JRiver library when a file gets added to the play queue?
It already reads the corresponding .nfo as it should to pull its metadata, and a thumbnail from a frame grab seems to get generated by JRiver.

Thanks :)
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 05, 2023, 07:29:03 pm
More testing, and it's looking good overall

The scan does not work on MPLS (created by Particles). 
Title: Re: NEW: Additional video file metadata
Post by: elprice7345 on March 08, 2023, 05:04:23 pm
Great new feature!

I was playing around with some of the video data fields and noticed an anomaly.

MC shows the frame rate as 120 for one of my videos, but MediaInfo detects it as 59.94.

Is this a bug or an issue with my setup?
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 08, 2023, 10:00:42 pm
Testing on 30.0.75 : Ran both Analyse Video and Video Black Bar Analysis over all my Videos (6,779)

Analyse Video:  Looking really solid,
- (currently) Not supported for JTV, BDMV3D, IFO
- Works now with MPLS Particles
- Only other ones that did not get any info were 2 x UHD BDMV Movie rips, and (for some reason) I have 25 x BDMV (not MPLS) particles that also did not work (but I'll re-create these as MPLS).  No idea why the 2 x BDMV did not get any info (BD UHD of "Fury" and "The Patriot")

Black Bar Analysis:  This worked OK - and we all know this will never be "perfect", but some #'s and thoughts:
- 12.5% did not get any Video Crop values
- A significant set of others are just wrong
- When running Black Bar Analysis, it seems to overwrite the Aspect Ratio BUT only if it had a value before, it did overwrite those that had "0" in it after doing the prior Analyse Video

"Aspect Ratio" Field:  It seems this field can be updated with various values pending what is run last:
- Update Library from Tags = uses "Dimension" Field
- Black Bar Analysis = use "Video Crop" Field
- I've also got some 500 videos where the "Aspect Ratio" was set to "0.00" at some point and it seems to be "stuck" on this value and neither running Update Library from Tags or Black Bar Analysis will change it.

I'm not sure what other think???, but my suggestion would be:
- Use the "Dimensions" field to calculate "Aspect Ratio" as was traditionally done
- Use the "Video Crop" field to populate a new "Aspect Ratio (Crop)" field
- Make the new "Aspect Ratio (Crop)" field user editable so the dodgy ones can be manually fixed (note: I see the "Video Crop" & "Aspect Ratio" fields are user editable which is good)

Note: Zybex came up with a calculated "Aspect Ratio (Crop)" for SOT so I'd not want to cut across his bow
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 01:28:39 am
I also have a few files that report Aspect Ratio of 0.0, I'll be looking at them to see whats up with that. This seems to be the value the analysis calculates, but i'm not sure why yet.

I do not plan to have a separate field at this time. I don't see the value in knowing the distinct aspect ratio with black bars or without, as we really only care about the active image aspect ratio - its also not used in playback and is purely informational. (feel free to present an argument why it would be useful!)
The logic should generally try to avoid resetting it when you have a value obtained from the black bar analysis.

As for cropping being wrong -
All my files analysed fine with .75 now, none were skipped any longer.

If you have any files that were skipped, or analysed to obviously wrong values, please do share.
Title: Re: NEW: Additional video file metadata
Post by: lepa on March 09, 2023, 02:02:12 am
I have some SD 4:3 (dimension 720x576) television recordings inside matroska which gets detected like
Video Crop 0x0x720x576
Aspect Ratio 1.00

Also some with dimension 1800 x 1080
Video Crop 0x0x1800x1080
Aspect Ratio 0.00
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 02:14:17 am
The 0.0 case should be fixed in the next build.

If it actually has a value, are you sure its not actually playing as a square video as well? Something in the file would have it compute that.
Title: Re: NEW: Additional video file metadata
Post by: lepa on March 09, 2023, 02:26:48 am
'Video.DisplayAspectRatio_String' from Mediainfo gives me 4:3 and file also plays as 1.33
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 02:32:46 am
I'm certainly not going to die on a hill about aspect ratios (as you say they are (currently) informational)....and I'm a 16:9 display only user.... but my argument would be:
- One is about the video frame
- The other is about what part of the video frame the content uses.

It may make a difference for those with the screens that have masking?  No Idea....

It also just feels wrong, to have one field that can have two different values pending what process was used when.

I've got all sorts of odd looking aspect ratios (but I'll send you a link to an XL dump of my videos with the new fields) that will show the ones skipped, the 0.00 and the ones that are just... odd (you can ignore the test pattern ones).  TBH I'll personally just stick to the calculated ones over black bar detection with my setup, but thanks for looking.

Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 02:49:23 am
I can't really do anything with a list of metadata. A few select samples would be helpful. (Also, Aspect Ratio seems to not be in that?). Don't need 0.00 samples, thats just a bug and will be fixed.

In general, its not making up data for aspect ratio, so if its coming up with a value that is not 0.00, then that is based on data from the file. In the simplest case just w / h, but many files also have an aspect ratio beyond that for anamorphic rendering. So if you have a file with the dimensions of 720x576, which has a natural aspect of 1.25 (w / h), then something is transforming it to 1.00, that value is not an accident.

Sometimes there can be conflicting info in a file. The video stream says one thing, the container another, and which value to give priority is a long debate. I'll double check the preference matches what playback uses.

A sample would of course let me make sure.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 02:52:57 am
Fair enough (I send the fixed XL file with aspect ratio this time ;) )  I'll redo once the 0.00 thing is fixed and report back with specific examples but two I mentioned earlier that don't get any values are the UHD BD version of "Fury" and "The Patriot"
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 02:59:08 am
For weeding out weird Black Bar detection things, I've setup this smartlist for myself:

Code: [Select]
[Media Type]=[Video] -[Video Crop]=[] -[Video Crop]=[0x" -[Video Crop]="x0x" -[Video Crop]=[2x" -[Video Crop]="x2x"
Basically what it does is find files which don't have one of the borders at zero (or at 2 pixels) - which would indicate typical cropping behavior - eg. just one side having letterboxing/pillarboxing.
If both sides have significant cropping, either the video is weird (encoded bars on both sides, it happens), or something is up.

No guarantees that other files which seem to have reasonable values won't be wrong in other means, but these are the most obvious cases.

Personally, my actual video library is pretty high quality files only. But I have a ton of sample files I'll have to throw into my test library as well over the next few days.
I do see a few outliers in my own library that I'll be looking at as well.

Once I have nailed down the issues with my files, if any issues for you remain, a sample would indeed be helpful.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 04:02:11 am
Nice - that smartlist did show a bunch of questionable vids (home vids, test files etc), but also had 100+ BDMV/MPLS rips.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 04:06:25 am
eg: Terminator 2: Judgement Day UHD BD rip - Video Crop is 408x1000x3758x1312
Title: Re: NEW: Additional video file metadata
Post by: lello on March 09, 2023, 05:05:57 am
For what it's worth, today I wanted to analyze my library and out of 219 files the Video Crop field is populated only in 8 cases.
Title: Re: NEW: Additional video file metadata
Post by: zybex on March 09, 2023, 05:13:40 am
Subtitle Language is not parsing unicode - that's Norwegian Bokmål
(https://i.imgur.com/upcf7pI.jpg)
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 06:40:44 am
I've fixed some issues with black bar detection, made the detection window a bit longer, and my samples all seem to be improving.
We already sample 7 different spots within the first 30 minutes of a video, and reading around 5-10 seconds at those spots to get a good idea of the video. Other then parsing the full video I'm not sure where there is a diminishing returns in expanding on how many spots we check to give the biggest coverage.

Subtitle Language is not parsing unicode - that's Norwegian Bokmål

Actually the language was encoded wrong in our list from language code to language name, but fixed regardless!
Title: Re: NEW: Additional video file metadata
Post by: zybex on March 09, 2023, 07:09:20 am
Maybe spread out those 7 samples to the entire length of the movie? interval=length/8 or something.

Kursk (2018) (https://www.imdb.com/title/tt4951982/) has the first 17 minutes at 1.66 and then opens to 2.39 until the last 2 or 3 minutes, which change back to 1.66. MC sets the Video Crop to 0x0x1920x804, which seems correct.
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 08:57:54 am
I changed it to spread out a bit more over the video depending on duration. And yes, the crop should cover the "biggest" possible active image. So if it changes mid-video, it would ensure both (or more) options will be fully shown inside the crop rect.
Title: Re: NEW: Additional video file metadata
Post by: lepa on March 09, 2023, 04:40:06 pm
With latest beta I haven't found any issues anymore with crop detection. Good work!
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 05:00:46 pm
I'm do in order a full:
- Update Library from tags, then
- Analyse Video
- Black Bar detection

It will take hours!
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 09, 2023, 05:06:13 pm
Update Library from Tags includes the same processing Analyse Video does, no need to do it separately. Analyse Video just does less, eg. not overwriting actual tags in your DB etc.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 05:11:26 pm
Thanks - makes sense and just saved me some hours (up to 881 of 6778)!

One other thing I found is I had a live URL in the Video section (for a Camera I think - now deleted the entry).  When Analyse Video hits that it stop and you can't cancel it, you have to close down MC and then kill the worker from Task Manager (or reboot).
Title: Re: NEW: Additional video file metadata
Post by: OverTheAir on March 09, 2023, 05:13:10 pm
... but two I mentioned earlier that don't get any values are the UHD BD version of "Fury" and "The Patriot"
Probably irrelevant but both those discs had AACS 2.1 and IIRC very few other discs have ever used it. Can't see why that would make a difference vs behavior with AACS 2.0 protected discs, regardless of whether protection has already been removed at time of rip or if its done dynamically on the rip at time of playback, but thought I'd mention it.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 05:15:30 pm
Yeah interesting point.  Both have been decrypted but I wonder.  Lets see what happens after the latest set of updates I'm doing.
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 09, 2023, 06:59:21 pm
Testing on 30.0.76 : Update Tags over all my Videos (6,779)

Looking really solid and I'm personally very satisfied with the results.
- (currently) Not supported for JTV, IFO, ISO
- Works now with BDMV3D and MPLS / BDMV Particles
- Only failure was on the 2 x AACS 2.1 BD UHD rips ("Fury" and "The Patriot") - Weird, I might try to re-rip these.
- The 0.00 Aspect Ratio is fixed (well It's gone from 500+ to 4 and these are all WMV and are not what you would call great quality)
- Aspect Ratios look good at this stage

Video Bitrate: About 2/3rds don't get a Video Bitrate.  Having a look at these in MediaInfo, I see that many just don't have a listing for Bitrate at all, but lots have the following format if that is of interest:
Code: [Select]
Bit rate mode : Variable
Bit rate : 19.7 Mb/s
Maximum bit rate : 35.0 Mb/s

Off to start Black Bar detection
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 10, 2023, 12:52:41 am
Black Bar detection is still grinding on but getting there.

Re-ripped Fury and The Patriot with makeMKV and these analyse just fine (must be some DeUHD thing back in the day on 2.1 discs). 
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 10, 2023, 02:27:41 am
So after 8 odd hours Detect Black Bars finished.  Looking really really solid!
- (currently) Not supported for JTV, IFO, ISO, bdmv3d
- Minimum Length of Video seems to be 45sec (no probs with that)
- Used Hendrik's Smartlist to look at potentially odd crops but these were only 2% of my files and they were mostly actually correct with some weird matte (combo of letter and pillar box) or older analogue transfers without crisp edges. 

No Video Crop Calculated.  Not many of these, but include:
- Taken (BD FHD)
- Sherlock Holmes (BD FHD)
- Pink Floyd Wish You Were Here (BD FHD)
- Board Walk Empire (BD FHD) S4 D1,2,4  S5 D3

Exceptions:
- Grand Budapest Hotel (BD FHD):  Reported as 272x22x1644x1058 : One of those movies that likes to switch between mattes

Notes: These are examples of some poor mastering that is shown up by this new feature
- Patriot Games, Start Trek 5 & 7, Rambo 2, and Shaun of the Dead (all BD FHD) that have a bit of unnecessary black bar edging on one side

The only thing I've found is if I clear the "Video Crop", the Aspect Ratio is not recalculated from Dimensions.  I have to run "analyze video" again.  Not a biggie but worth mentioning.

Well done!
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on March 10, 2023, 03:09:13 am
No Video Crop Calculated.  Not many of these, but include:
- Taken (BD FHD)
- Sherlock Holmes (BD FHD)
- Pink Floyd Wish You Were Here (BD FHD)
- Board Walk Empire (BD FHD) S4 D1,2,4  S5 D3

What file types are those? BDMV?

With BDMV it only picks one m2ts file to look at, I wonder how solid that is on fancy discs with lots of branching, or other shenanigans. But I can't really justify having a full BD demuxer there at this point. Maybe once I've made one for Linux, it can be used here..
Title: Re: NEW: Additional video file metadata
Post by: jmone on March 10, 2023, 03:29:16 am
All are BDMV (rips from original BD):
- Taken : One big M2TS, does not look special in any way
- Sherlock Holmes 2 : Yup, it is a seamless branding disk with several M2TS.  Largest M2TS is a 30min segment from the middle of the movie and should produce the correct crop
- Pink Floyd Wish You Were Here : Forget it, it's one of those Audio BD with a few Video segments, but the main segment is Audio only with Black Video
- Board Walk Empire : It's the typical TV BD with one large M2TS per eps.  Can't see anything obviously different between the Discs that work fine in this series and the ones that don't

It's really not worth tracking down.  At best it is a few discs out of bucket loads.  Just reporting in.
Title: Re: NEW: Additional video file metadata
Post by: lello on July 19, 2023, 06:10:59 am
With bdmv the audio title field is not populated: is this normal?
Title: Re: NEW: Additional video file metadata
Post by: Hendrik on July 19, 2023, 06:38:19 am
Blu-ray discs hold no title information for their streams. You get language and codec in other fields.