INTERACT FORUM

More => Old Versions => JRiver Media Center 31 for Windows => Topic started by: Manni on July 23, 2023, 06:31:37 am

Title: BD Folders import in library: installments missed (films)
Post by: Manni on July 23, 2023, 06:31:37 am
When scanning a share of 267 UHD Blu-ray folders, only 222 titles are detected during an auto scan (single share included). Adding the missing titles manually (dropping the folder in the library)works fine. Only the “video” box is checked for that share, and the titles are all movies.
Example: I have the first three John Wick movies (each in a separate folder) in that share, but only the first one is added to the library during the auto import.
The same thing happens with another share with more than 600 titles. Many are missed/skipped.
All these UHD BD folders are 1:1 full rips from my original discs.
Title: Re: Some Movie Titles Not Imported
Post by: JimH on July 23, 2023, 06:46:25 am
How long did you wait?  That much data might take some time, especially if the drives aren't local.
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 23, 2023, 04:52:06 pm
I'll move this to a new thread.  How long did you wait?  That much data might take some time, especially if the drives aren't local.

Hi Jim ,

I waited until the auto-import completed (I.e. “STOP” isn’t available in « importing media » window in the lower left corner, and “import summary” is displayed with the number of files imported). I tried both with black bar detection (very long, an hour or more) and without (long, about ten minutes). Even when the process is completed, some titles are still missing.

I tried to generate JRiver metadata for the share using MyMovies collection manager, then I did a new import with JRiver, it detected the same number of title (plus a couple added today), so 225 instead of 267. Of the three John Wick movies, only the first one was imported. Running another import doesn’t add any titles.
Title: Re: Some Movie Titles Not Imported
Post by: JimH on July 23, 2023, 05:02:49 pm
When you set up auto import, you need to set both the folders to watch and the filetypes you want imported.    Edit the folder and double check filetypes.
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 23, 2023, 05:27:36 pm
When you set up auto import, you need to set both the folders to watch and the filetypes you want imported.    Edit the folder and double check filetypes.

Thanks, as reported initially I disabled audio and pictures, and only enable videos. All the video file types are enabled, so I couldn’t add any if I wanted to.

Also, I only have 4K UHD BD folders in that share, so it doesn’t make sense that it would detect some and not others. If it can import 225 bd folders, which types should I enable in the video files so that it imports the others? There are no difference between John Wick 1 and 2 or 3. All three are 4K UHD BD folders, yet only the first one is imported.

The issue is not with series, I checked others titles in the same series (for example Amazing Spider-Man and Amazing Spider-Man 2) and they are imported.
Title: Re: Some Movie Titles Not Imported
Post by: blgentry on July 24, 2023, 09:55:20 am
How deeply nested are these folders?  Could any of the paths to the files be longer than 252 characters?  That might explain it.

Are there any odd characters in the directory names of some of these rips that won't auto import? 

How about the compression type?  Are all of the offenders using a different video or audio compression than the rest?

All of these are just guesses.  Here's hoping one of them helps.

Brian.
Title: Re: Some Movie Titles Not Imported
Post by: mattkhan on July 24, 2023, 11:54:06 am
Does it import if you pick the folder manually? If not, enable logging and try again the post the log here
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 25, 2023, 02:23:33 pm
Does it import if you pick the folder manually? If not, enable logging and try again the post the log here

It always work when I import the folder manually, so not a path issue.

There is no difference between the titles that are important and those that are not.

They are all BD / UHD BD folders.

They are all at the same level (\\SHARE\TITLE)

For example, the title in the folder "John Wick" with a standard BDMV structure is imported fine, but "John Wick 2" and "John Wick 3 - Parabellum" are not.

All same BD folder strucure, ripped 1:1 from my original disks.

If I drop either of the non detectede folders, they are imported fine.

I think it's always the same titles (and the same numbers) that are not detected.

Not just series, I already mentioned the "Spider-Man" examples, the two disks are imported fine.
Title: Re: Some Movie Titles Not Imported
Post by: mattkhan on July 25, 2023, 02:32:52 pm
it's probably a coincidence but, for me, john wick 3 gets imported repeatedly (I think I had nearly 10 copies at one point)

I suggest the following

* reset logs
* run auto import when there are at least a few items not imported
* grab the log and post it here
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 25, 2023, 03:56:06 pm
it's probably a coincidence but, for me, john wick 3 gets imported repeatedly (I think I had nearly 10 copies at one point)

I suggest the following

* reset logs
* run auto import when there are at least a few items not imported
* grab the log and post it here

Here you go, I did the following with the latest build 38 released today:

- Deleted all the titles in the library (in all video sections)
- Reset the log
- Imported a share with 246 4K UHD BD Folders (only one level, except maybe a couple of titles with more than one discs, but that's not the case for the missing ones). I only check the "video" type, all the subtypes are checked, I uncheck "ignore previously deleted titles" to make sure all titles are imported, and I ask to create thumbnails etc.
- The results are the same, 222 titles imported, 226 updated, though 238 titles are reported when you actually look at the list.
- When looking at the files, they all use the index.bluray virtual file name, which is good re our other discussion
- Only John Wick is imported, 2 and 3 are still missing (I didn't try to find what the other 20 or so missing titles are)
- For some reason though, the two missing John Wick titles are reported in the "updated" titles
- One thing different in build 38, it now correctly processes the JRiver metadata files created by MyMovies (I did this to see if it would help detecting some of the missing titles, it doesn't). The build version didn't use the additional metadata and kept the native JRiver Media Center one.
- I attach the log and a few screenshots of the process in case it helps to figure out what happens.

I guess that someone would need to check import of large shares of BD Folders to try to reproduce this. Only about 10% of the titles are missing, so it would be easy to miss this on smaller imports. I have another share of about 600 titles, but there are more titles with more than one disc per title, so I end up with more than 600 imported titles, so not as easy to debug.
Here you go, I did the following with the latest build 38 released today:

- Deleted all the titles in the library (in all video sections)
- Reset the log
- Imported a share with 246 4K UHD BD Folders (only one level, except maybe a couple of titles with more than one discs, but that's not the case for the missing ones). I only check the "video" type, all the subtypes are checked, I uncheck "ignore previously deleted titles" to make sure all titles are imported, and I ask to create thumbnails etc.
- The results are the same, 222 titles imported, 226 updated, though 238 titles are reported when you actually look at the list.
- When looking at the files, they all use the index.bluray virtual file name, which is good re our other discussion
- Only John Wick is imported, 2 and 3 are still missing (I didn't try to find what the other 20 or so missing titles are)
- For some reason though, the two missing John Wick titles are reported in the "updated" titles
- One thing different in build 38, it now correctly processes the JRiver metadata files created by MyMovies (I did this to see if it would help detecting some of the missing titles, it doesn't). The build version didn't use the additional metadata and kept the native JRiver Media Center one.
- I attach the log and a few screenshots of the process in case it helps to figure out what happens.

I guess that someone would need to check import of large shares of BD Folders to try to reproduce this. Only about 10% of the titles are missing, so it would be easy to miss this on smaller imports. I have another share of about 600 titles, but there are more titles with more than one disc per title, so I end up with more than 600 imported titles, so not as easy to debug.
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 26, 2023, 07:29:56 pm
Bump. Does any of the above help identify why JRiver misses 20 titles on import (UHD BD Folders)?
The John Wick 2 and 3 are listed in the log, then for some reason they are not present at the end of the import.
Thanks.
Title: Re: Some Movie Titles Not Imported
Post by: jmone on July 26, 2023, 08:11:44 pm
Looking at the log, all three John Wick's seem to have been added to (or were already part of) the library (they both get a Database Key) as follows:
- John Wick = 350
- John Wick 2 = 464
- John Wick 3 = 1


To see all items in the Database, create a SmartList "~d=a" (without comments).  This will show everything.  You may also want to add a Library Field to expose the Database Key (see attached pic). 

Find the files and have a look at what they have been tagged as.  Check fields such as ("FileDBLocation" in case they have been tagged as "Removed" from a prior deletion) and "Media Type" and "Media Sub Type" to make sure they are Video and "Movie" respectively. 
Title: Re: Some Movie Titles Not Imported
Post by: jmone on July 26, 2023, 08:41:07 pm
It could be as simple as having "Ignore files previously removed from library" checked?
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 26, 2023, 08:56:06 pm
It could be as simple as having "Ignore files previously removed from library" checked?
Thanks.
No, as indicated I unchecked this option before all imports.
Why would the library not show all titles by default?
This doesn't make sense.
I'll redo an import (there is nothing in the library) and I'll follow your instructions to see if I can display more titles, but I have no idea why JRiver doesn't display all the titles.
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 26, 2023, 09:14:00 pm
So I reimported the share, 222 titles imported, 222 titles shown in the movies list (I tag them as movies during import, check videos all subtypes, uncheck audio and pictures). Same in the files list. Nothing in audio, home videos or shows (as expected, as I tag them all as movies).
I then created a custom smartlist with the "~d=a" argument, it displays exactly the same 222 titles.

There is a bug in this import with bd folders, it would be nice if someone from support could look at it.
Title: Re: Some Movie Titles Not Imported
Post by: blgentry on July 26, 2023, 09:34:12 pm
Look in your Video > Files view.  Do a search for one of your missing movies, like Wick 2.  If MC imported it, presumably it is actually there.

Every now and then, I find that MC mistakenly imports a video as Media Type "Data".  When this happens it will not show up in any normal views because the Media Type is incorrect.  The log shows that John Wick 2 and 3 were both assigned media type = video.  So they really should show up in one of the views under Video.  Files is a good one to use to try to find them.

You might also examine the definition of the view that you are using.  In particular look at the "set rules for file display" and see if there are any criteria there that would limit what is shown.

Brian.
Title: Re: Some Movie Titles Not Imported
Post by: jmone on July 26, 2023, 10:11:26 pm
Just to be sure that the "All Smartlist" looks like this (the sort by doesn't matter but the rest does)?
Title: Re: Some Movie Titles Not Imported
Post by: mattkhan on July 27, 2023, 03:33:05 am
the logs do suggest a view issue rather than an import issue

in addition to the other suggestions, have you looked in the folder to see if a sidecar file was produced/stored? I expect it would be index_bdmv_JRSidecar.xml

it contains the values of assorted tags which may shed some light on why it's not visible

i think that assumes that Options > General > Importing & Tagging is set to write sidecar files for video (when internal tagging is not supported), fairly sure that's the default setup though (at least I don't remember ever changing it)
Title: Re: Some Movie Titles Not Imported
Post by: Manni on July 27, 2023, 06:26:18 am
Hi all,

Thanks for your help and suggestions, but I don't think many people use bd folders hence no one has actually checked for what I believe is clearly a bug.

If any of you is able to import successfully 200+ BD folders, please let me know. JRiver misses about 10% of the titles (20 out of 246 in this example), so you won't likely notice any issues if you don't import at least 20-30 titles.

Re your questions:

There are only 222 titles in any view (movies, files, etc). I had already checked that and reported it.
There are no other file types added, because I tag the titles as movies during import. Anything that's added is in the video section. Nothing in audio, data or whatever. Also already checked and reported.
I only check "video" (and all subtypes) when importing. I specifically uncheck audio and pictures. So unless there is another bug, there is no reason for any of the imported title to end up anywhere but in the videos section.
The sidecar files were not produced, even though the options are left to default, so should be produced for video files. I had generated some with MyMovies collection management to see if it would help to detect the missing titles, it didn't. I have deleted the mymovies generated sidecar files and will try importing again, but it should produce the same results as initially as I wasn't generating JRiver metadata before trying to debug this. I can conform, howver, that if you check the "update external imformation" box during import, the Mymovies generated metadata in sidecar files is imported and replaces the JRiver scrapped info. I don't really care about this, as I don't use the JRiver library and it's very slow, so I've unchecked it in my last attempt.

Please note that I've NEVER used the library in many years of using JRiver as a player because my collection is managed by MyMovies and I use CMC as a front end. I only try to get it to work now because it's needed for the new black bar detection in JRVR, that unfortunaly doesn't work in real-time like madVR but saves the metadata in the library. So if your title isn't imported correctly, you don't get black bar detection for that title.

For this reason, I have NEVER made any change to any library related settings/options, changed any default views or whatever. The only changes I made are those suggested in this thread. Unfortunately nothing view-related helps because it's an import BUG with BD folders.

The custom view displays exactly the same thing as the default files or movies view, or the "Image and video -- all" or "All files (empty search) presets.

Thanks again for all your help and suggestions, I hope that someome from the team could answer and try to reproduce the bug so that it can be fixed.

Here is what the "All titles" custom rule I've added looks like:
Title: Re: Some Movie Titles Not Imported
Post by: jespermart on July 27, 2023, 06:30:05 am
So I reimported the share, 222 titles imported, 222 titles shown in the movies list (I tag them as movies during import, check videos all subtypes, uncheck audio and pictures). Same in the files list. Nothing in audio, home videos or shows (as expected, as I tag them all as movies).
I then created a custom smartlist with the "~d=a" argument, it displays exactly the same 222 titles.

There is a bug in this import with bd folders, it would be nice if someone from support could look at it.

I have the same problem. I have all my 4k 1:1 rips on my Synology 1815+ in a 4k library and all rips in their own folder named according to IMDB/TMDB databases. Out of my 351 movies, MC 31.0.38 only findes 310.
Title: Re: BUG with BD Folders import in library: 10% of the titles are missed
Post by: jmone on July 27, 2023, 06:38:56 am
FWIW - I have over 1,300 BD / UHD movies all as full folder rips.  I do tend to use either MC to rip them (either with AnyDVD in the Background or the newer makeMKV integration).  If I rip them externally, I manually import them at that time (I don't use AutoImport).  I've never had MC not import a BDMV Folder.
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: JimH on July 27, 2023, 06:38:59 am
Manni,
Are you also using a NAS?
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: Manni on July 27, 2023, 06:46:42 am
Manni,
Are you also using a NAS?

Yes, Unraid, but I tried importing from a mapped network drive with a drive letter as part of my debugging, it doesn't make any difference.

I've just tried to import a share full of bluray BD folders to see if the issue is just with 4K UHD Blurays or with blurays as well (it shouldn't make any difference as the BD folder structure is mostly the same for both) and only 201 titles are imported out of 209.

So the issue is with 4K UHD Bluray and Bluray BD Folders, whether the share is local (mapped disk drive) or network.
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: JimH on July 27, 2023, 07:00:28 am
What's different about the ones that are missing?
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: Manni on July 27, 2023, 07:02:08 am
I think I have found a cause for issue, even if it could be a coincidence :)

As there are only 8 titles missing in the Bluray share, I looked at all of them and there is a common element: the only titles missed are those with a "-" in the folder name. That probably trips the import routine.

@Jespermart, please could you take a quick look and confirm that your missed titles during import have a "-" or possibly another special character in the name?

I avoid special characters in folder names, but usually "-" is pretty safe.

No issue with ",".

So in the bluray share:

"Austin Powers - International Man of Mystery" will NOT be imported, but
"Austin Powers, International Man of Mystery"
or
"Getaway, The"
will be imported.

I checked in the 4K UHD Bluray share, and John Wick 3 has a "-" in the folder name:
John Wick 3 - Parabellum
However John Wick 2 doesn't, which goes against the idea above.
Also there are more than 20 titles with a "-" in the title, and some of them are imported.
For example, as already reported:
"Amazing Spider-Man, The" is imported.
"Batman v Superman - Dawn of Justice" is also imported.
That's why I couldn't find a common denominator with the 4K UHD Bluray share.

Anyway, hopefully this will help the devs to resolve?
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: Manni on July 27, 2023, 07:03:33 am
What's different about the ones that are missing?

Just replied to this as you were posting.
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: Manni on July 27, 2023, 07:20:07 am
Did more sleuthing re the 4K UHD Bluray shares, and here are some of the missed titles:

John Wick 2
John Wick 3 - Parabellum
Ghostbusters (2016)

So it looks like it's often the first title in a series that is imported.

However,

Batman Begin, The Dark Knight and The Dark Knight rises are all imported under The Dark Knight Trilogy (three separate titles).
Same dor Divergent, Allegiant and Insurgent, imported under "The Divergent Series"

So not a clear rule.

I haven't checked, but the titles with a " - " in the title in the bluray share might have been part of a series too.

So maybe look at number 2 and up in franchises rather than " - " in the title?
Title: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 08:19:55 am
OK, I looked at the bluray share import and the "-" thing was a coincidence. The number of folder names with a "-" matched the number of missing titles, but the behaviour is the same as in the 4K share. The first titles I checked matched, but the others were simply later in the list because they started with "The", which JRiver sorts in the T letter, while I add ",The" at the end of the folder name to not have all the "The" together.

Anyway, the series element is correct for the bluray share as well.

For example,
"Austin Powers" gets imported, but not "Austin Powers - Goldmember" or "Austin Powers - The Spy Who Shagged Me".

So the example I gave previously is actually wrong.

To sum it up, here are my conclusions, true for both the 4K bluray and the bluray shares:

- If a title is NOT part of a series, it is ALWAYS imported
- If a title is part of a series, the FIRST installment is usually imported, it's usually the next installments that aren't.
- However, this is NOT always true, some series are imported correctly
- Whether there is a " - " in the folder name or any other special character doesn't seem to matter, wich is why "John Wick 2" isn't imported.
- The behaviour of the bug is the same for 4K or bluray series

Maybe @Jespermart could look into his missing titles and confirm the above observations? Or maybe find more common elements?

You should be able to reproduce this easily if you rip a few 4K or blurays series into BD folders, with one installment per folder (each title in its own folder).

Your missing rate should be a lot higher than 10%, probably 50% or more based on the results here. I've changed the title of the thread as well to reflect this.

If you rip the following series, the missed rate should be 100% (I'm in the UK and these are not imports in case it matters):

4K UHD Bluray:
John Wick (1-2 boxset and 3)
Ghostbusters collection (4-film boxset)

Bluray:
Austin Powers Trilogy

Please note that all these titles are 1:1 bd folders rips from my original discs. Usually made with the fox, more occasionally with the fab.

At least you don't have to rip hundreds of titles to reproduce and fix :)
Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: mattkhan on July 27, 2023, 08:28:13 am
I use BD folder rips stored on an unraid server & do not have these issues

I verified by dumping this to a file

Code: [Select]
find /my/share -name index.bdmv > files.txt

then wrote a script which gets all entries via mcws using these searches

[Filename (name)]="index.bdmv"
[Filename (name)]="index.bluray3d"
[Filename (name)]="index.bluray"

and then looked for files on disk that aren't in those results

there's nothing missing & I have 636 BD folders in my library & I have some of those same sets (e.g. john wick)

Your missing rate should be a lot higher than 10%, probably 50% or more based on the results here. I've changed the title of the thread as well to reflect this.
my results can't prove there is no bug present but it does mean it's something more specific than just the symptoms reported (if there is a bug)

Title: Re: BD Folders import in library: 10% of the titles are missed
Post by: Manni on July 27, 2023, 08:35:46 am
I use BD folder rips stored on an unraid server & do not have these issues

I verified by dumping this to a file

Code: [Select]
find /my/share -name index.bdmv > files.txt

then wrote a script which gets all entries via mcws using these searches

[Filename (name)]="index.bdmv"
[Filename (name)]="index.bluray3d"
[Filename (name)]="index.bluray"

and then looked for files on disk that aren't in those results

there's nothing missing & I have 636 BD folders in my library & I have some of those same sets (e.g. john wick)
my results can't prove there is no bug present but it does mean it's something more specific than just the symptoms reported (if there is a bug)

Thanks. This only happens with series though, so you can discard all the titles that are NOT series, they import fine here too.

As I said, it's not true for ALL series. Some series import fine. The problem might be present on the 4K UHD version and not on the bluray version. Also the folder structure might matter as well. All the titles are in the share, with one folder per title. No subfolders for each installment in a "master" folder for example.

It would definitely help if @Jespermart could take a look at their collection and let us know if it confirms or not my findings.

Hearing from those who have the same issue is more useful than from those who don't :)

Here are the 4K UHD Blurays I mention (the blurays are in storage somewhere) if the same edition has to be checked:
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: mattkhan on July 27, 2023, 08:39:35 am
Hearing from those who have the same issue is more useful than from those who don't :)
no that's not how to diagnose a bug, you need all information to narrow it down

the main point missed in this case is that the provided logs clearly and irrefutably show that MC's import did find the item and did process it, the Q to be answered is therefore *where is that item in your library*
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 08:42:01 am
no that's not how to diagnose a bug, you need all information to narrow it down

the main point missed in this case is that the provided logs clearly and irrefutably show that MC's import did find the item and did process it, the Q to be answered is therefore *where is that item in your library*

I agree, but you mention 636 bd folders, most of them are irrelevant.

Please confirm how many of these titles are volume 2 and up of a series franchise, and provide the exact title, format and edition, as well as the folder structure used to store them, and the software used to rip them from original discs as 1:1 BD folders.

Then we'll have more useful info from someone who doesn't experience the issue.
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: mattkhan on July 27, 2023, 08:50:46 am
Define series more precisely please

You're still ignoring the point about your logs showing it processing the entry, you can't keep ignoring that in favour of your chosen root cause
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Hendrik on July 27, 2023, 08:52:35 am
Media Center does not have any inherent concepts of series or collections. Such things are all applied only based on metadata on the view layer. Which makes it seem likely again that a view is somehow misbehaving in your setup.
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 08:58:10 am
Media Center does not have any inherent concepts of series or collections. Such things are all applied only based on metadata on the view layer. Which makes it seem likely again that a view is somehow misbehaving in your setup.

I am using the default views. I've even cleared the library. So please explain how to show all the titles from a series if it's a view issue.

MC is still only reporting 222 titles out of 246, or 201 out of 209, and it only shows the imported titles in the default view.

Anyway, I really don't care about the library.

I'm going to do a simple test:

I'm going to re-import my 4K share with black bar detection enabled.

If what you say is true, black bar detection should work on all the titles (I'll play them from CMC, so I don't need to see the titles).

If black bar detection doesn't work only for the missing titles, then it will point to the direction of a bug and not a view problem, correct?
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 09:04:29 am
I'll wait for the new build that fixes the index.bluray bug to do this test.
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 09:13:01 am
Define series more precisely please

You're still ignoring the point about your logs showing it processing the entry, you can't keep ignoring that in favour of your chosen root cause

Series in this context (films only) is a franchise with more than one installment.

I've given enough examples and made it clear it was about films, not TV series, but you're right that it doesn't hurt to make it even clearer.
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: mattkhan on July 27, 2023, 09:27:05 am
no fixed definition then but things that are part of a series  I can see include

21/22 Jump Street
Alien (1-3)
Ant Man (+ sequel)
Avengers (infinity war, age of ultron)
Back to the Future (all 3)
Beauty & the Beast + remake
Blade Runner + 2049
Captain America (all 3)

that's quite a few already, not going through the rest as there are plenty there (skimming the rest I can see things like Harry Potter, Star Trek, John Wick + more)

so if there is a bug it's definitely more specific than noted so far

Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: jespermart on July 27, 2023, 09:28:03 am
I also have problems with series, but not all series have problems. John Wick 1 is found 2, 3, and 4 are not - Jack Reacher 1 is found 2 is not - Red is found 2 is not - and I am also missing titles that are not part of a series, but mainly it seems that series are most often affected. When I started with MC 31 I don't remember having the problem, but when I cleared my library due to a problem with subtitles when using black bar detection, and imported my movies again with MC 31.0.38, then I could see I was missing a lot of movies especially series.
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: mattkhan on July 27, 2023, 09:32:58 am
@jespermart pls share logs as well (as per the earlier post) so we can see if your behaviour is same in the logs
Title: Re: BD Folders import in library: installments missed
Post by: Manni on July 27, 2023, 09:33:44 am
no fixed definition then but things that are part of a series  I can see include

21/22 Jump Street
Alien (1-3)
Ant Man (+ sequel)
Avengers (infinity war, age of ultron)
Back to the Future (all 3)
Beauty & the Beast + remake
Blade Runner + 2049
Captain America (all 3)

that's quite a few already, not going through the rest as there are plenty there (skimming the rest I can see things like Harry Potter, Star Trek, John Wick + more)

so if there is a bug it's definitely more specific than noted so far

Thanks.

Please could you back up your library, clear all the titles and re-import a share with at least some of these with MC 31 build 38, especially John Wick and let us know if they are all imported?
Title: Re: BD Folders import in library: installments missed
Post by: JimH on July 27, 2023, 09:56:35 am
You're still ignoring the point about your logs showing it processing the entry, you can't keep ignoring that in favour of your chosen root cause
Please reply to mattkhan's suggestion.

It's possible the entries are located in a different top level view.  Documents, for example.  Search your entire library.

There are some files that are containers that can be audio or video.

In the future, you might consider using makeMKV to rip.
Title: Re: BD Folders import in library: installments missed
Post by: Manni on July 27, 2023, 10:09:54 am
Please reply to mattkhan's suggestion.

It's possible the entries are located in a different top level view.  Documents, for example.  Search your entire library.

There are some files that are containers that can be audio or video.

In the future, you might consider using makeMKV to rip.

I have already done these checks and reported on this. There is nothing hidden anywhere.

I honestly don't care about the library, if no one thinks there is a bug, it's not my problem if the default view after a clean install doesn't show all the titles imported.

I'm going to check if the black bars are detected for all the titles.

If they are, problem solved (for me) as I've never used the library view and I don't intend to.

I use makemkv when I have to, but it's not my preferred way to rip to folders. I don't think it's relevant here anyway, unless Mattkhan does what I suggest (re-import his shares after deleting the library content with the current build which is what those reporting issues are using) and doesn't experience the issue I and others experience.

If Mattkhan still gets all his titles (including John Wick) after re-importing with MC 31 build 38, then we can look into various ripping differences.

I asked this question to Mattkhan (tool to rip their original physical disks 1:1 to bd folders) and they didn't reply.

If the library re-import test is positive for them, I'll try to re-rip John Wick using the tool they've used and see if it makes a difference.

But I'll only do this if black bars are not detected for all titles, as that's the only case use for which the library and library views matter for me.
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 27, 2023, 10:50:52 am
I missed that, the question is what do I use to rip? Recently makemkv backup (NB not via MC, via standalone makemkv), previously anydvd.

I can try a clean import on another machine later.
Title: Re: BD Folders import in library: installments missed (films)
Post by: BryanC on July 27, 2023, 11:00:19 am
This is the correct answer: https://yabb.jriver.com/interact/index.php/topic,136599.msg946931.html#msg946931

The files are somewhere in MC, they just aren't getting tagged properly on import. Auto-import (Carnac) generally works well but sometimes it can get tripped up and assign improper Media Sub Types. If you share some of these directory and filename layouts with the JRiver team I'm sure they'll do their best to improve the auto-import logic.

The easiest thing for you to do in the interim is to find the files in the MC library and properly tag them so that they show up in the Movies view with the rest of your Movies. Once you find them, you can batch highlight and edit (Alt+Enter) the [Media Sub Type] to "Movie" and they'll show up in your Movies view.

Another option is to avoid the Sub Type tag altogether and use a view based on filepaths, that is also very easy to implement and should work automatically moving forward.

Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 27, 2023, 11:21:57 am
Did you check for presence of a sidecar file? This would also reinforce it being a view problem
Title: Re: BD Folders import in library: installments missed (films)
Post by: Manni on July 27, 2023, 12:18:21 pm
I missed that, the question is what do I use to rip? Recently makemkv backup (NB not via MC, via standalone makemkv), previously anydvd.

I can try a clean import on another machine later.

Thanks. Please let us know what the results are. Make sure you use the latest 31.0.38, as those having the issue didn't have it with previous versions. I certainly never had this issues with version up to 29, whenever I tested the library (which is not often I admit).
Title: Re: BD Folders import in library: installments missed (films)
Post by: Manni on July 27, 2023, 12:19:21 pm
Did you check for presence of a sidecar file? This would also reinforce it being a view problem
Sidecar files are not created during import, even though according the settings they should be.
Title: Re: BD Folders import in library: installments missed (films)
Post by: Manni on July 27, 2023, 12:22:00 pm
This is the correct answer: https://yabb.jriver.com/interact/index.php/topic,136599.msg946931.html#msg946931

The files are somewhere in MC, they just aren't getting tagged properly on import. Auto-import (Carnac) generally works well but sometimes it can get tripped up and assign improper Media Sub Types. If you share some of these directory and filename layouts with the JRiver team I'm sure they'll do their best to improve the auto-import logic.

The easiest thing for you to do in the interim is to find the files in the MC library and properly tag them so that they show up in the Movies view with the rest of your Movies. Once you find them, you can batch highlight and edit (Alt+Enter) the [Media Sub Type] to "Movie" and they'll show up in your Movies view.

Another option is to avoid the Sub Type tag altogether and use a view based on filepaths, that is also very easy to implement and should work automatically moving forward.
The missed titles are nowhere else. Also, if they were, it would be a BUG because I always add a manual Movie tag to the import. So there is no reason for any imported files to go anywhere else. MC doesn't have to guess. I tell it "all these are movies". I only enable "video" types (all sub-types checked, as per default) and I disable all the other types (music, pictures).
Title: Re: BUG: BD Folders import in library: installement 2 and up of series often missed
Post by: Manni on July 27, 2023, 12:24:49 pm
Media Center does not have any inherent concepts of series or collections. Such things are all applied only based on metadata on the view layer. Which makes it seem likely again that a view is somehow misbehaving in your setup.

FYI, I'm currently re-importing the same share (246 titles, only 222 detected). MC is analysing the video for black bars, and it's only analysing 222 titles.

This confirms that it's an import issue, not a view issue. If it was a view issue, it would analyse all 246 titles. I'll attach the new log. [Sorry, I can't, log wasn't enabled on that client].

I'm getting the same results on two different clients (HTPC and laptop). I had just done a full uninstall / clean / reinstall on the laptop, with default settings.
Title: Re: BD Folders import in library: installments missed (films)
Post by: JimH on July 27, 2023, 12:33:48 pm
Manni,
I'd appreciate it if you'd keep your opinion on where the problem lies to yourself until we know more.  You're repeating yourself, with little hard information.  Nobody is ignoring you.

We've seen stranger problems with NAS drives, for example.

You could also take the caps lock off.
Title: BD Folders import in library: installments missed (films)
Post by: Manni on July 27, 2023, 12:56:59 pm
Manni,
I'd appreciate it if you'd keep your opinion on where the problem lies to yourself until we know more.  You're repeating yourself, with little hard information.  Nobody is ignoring you.

We've seen stranger problems with NAS drives, for example.

You could also take the caps lock off.
I have reinstalled MC 28 build 91 and imported the same share.

All the Ghostbusters and John Wick titles are imported correctly. It detects 254 titles (I assume because a few have more than one disc or bonuses), vs 222 with MC 30 (and I believe MC30).

I suspect MattKhan will confirm that he also has some issues when he reimports his shares with MC 31.0.38.

I attach the log and a screenshot of the import results with MC29.
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 27, 2023, 02:50:23 pm
Mattkhan to reimport their shares with MC31 and tell us the results
can confirm the reported problem

output of my script

Code: [Select]
pipenv run python bdmv_verify.py /home/matt/Downloads/files.txt 1.2.3.4 myuser mypass
NOT FOUND : /media/films/Back to the Future Part III/BDMV/index.bdmv
NOT FOUND : /media/films/Close Encounters of the Third Kind (Bonus)/BDMV/index.bdmv
NOT FOUND : /media/films/Die Hard 2/BDMV/index.bdmv
NOT FOUND : /media/films/Die Hard_ With a Vengeance/BDMV/index.bdmv
NOT FOUND : /media/films/Harry Potter and the Half-Blood Prince (Disc 2)/BDMV/index.bdmv
NOT FOUND : /media/films/Jurassic World_ Fallen Kingdom/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible - Ghost Protocol/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible II/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible III/BDMV/index.bdmv
NOT FOUND : /media/films/Scream 2022/BDMV/index.bdmv
NOT FOUND : /media/films/Sing Street/BDMV/index.bdmv
NOT FOUND : /media/films/Spy Kids 2/BDMV/index.bdmv
NOT FOUND : /media/films/Star Trek Beyond 3d/BDMV/index.bdmv
NOT FOUND : /media/films/TRON_ Legacy/BDMV/index.bdmv
NOT FOUND : /media/films/The Godfather_ Part II/BDMV/index.bdmv
NOT FOUND : /media/films/The Godfather_ Part III/BDMV/index.bdmv
NOT FOUND : /media/films/The Hunger Games_ Catching Fire/BDMV/index.bdmv
NOT FOUND : /media/films/The Hunger Games_ Mockingjay - Part 1/BDMV/index.bdmv
NOT FOUND : /media/films/The Smurfs 2/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England '88/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England '90/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England 86/BDMV/index.bdmv
NOT FOUND : /media/films/Toy Story 4/BDMV/index.bdmv
Found 613 NOT 23

reported pattern (something directory name related) appears to dominate the gaps

percentage of missed folders slightly smaller in my case but that's highly likely to be an artifact of my content rather than anything fundamental
Title: Re: BD Folders import in library: installments missed (films)
Post by: jespermart on July 27, 2023, 02:52:05 pm
Dismissive and patronising tone weakens yours.

Anyway, I’ve provided all the info I could, I suggest that we wait for:

Mattkhan to reimport their shares with MC31 and tell us the results
Jespermart to reimport with MC29 and see if it solves the issue, as it did for me.

Over and out.
I can confirm Manni's findings. Reimporting all my shares with MC29.0.91 solved all problems
Title: Re: BD Folders import in library: installments missed (films)
Post by: jespermart on July 27, 2023, 03:06:30 pm
The reason why I haven't discovered the problem before is probably due to the fact, that I have always had a backup of my library lying around and every time I have reinstalled windows and jriver MC, I have simply restored my library from the backup copy. Only when I had to restore my library without black bar detection did I discover that I was missing 40 movies, mainly series in my 4K folder and about 20 movies in my BD folder.
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 27, 2023, 03:13:50 pm
I reran import and no new items added

I then picked one of the missing folders at random and did a single folder import, it imported fine

there's clearly a bug in the bulk import somewhere but v hard to say where looking from the outside

I could speculate though :) e.g. it's clear that it chunks the load up into multiple tasks so some error in that chunking is feasible (and the no of missing files is close to the no of chunks which is highly likely to be a coincidence though could point towards a particularly simple bug)

Title: Re: BD Folders import in library: installments missed (films)
Post by: JimH on July 27, 2023, 03:24:47 pm
I can confirm Manni's findings. Reimporting all my shares with MC29.0.91 solved all problems
Thanks.
Title: Re: BD Folders import in library: installments missed (films)
Post by: Hendrik on July 27, 2023, 05:26:07 pm
can confirm the reported problem

How did you do the bulk import? Just pick the top level folder for import? Or setup auto-import?

Was this with the fix in .39? I could see import getting confused due to the inconsistency of the name.
If it wasn't with 39, can you re-run it?

If it persists, can you post a log of the original import on a previously empty library, and outline which ones ended up missing again (or at least a few)? It would take me some time to setup a bunch of ripped Blu-ray folders for testing. I had a set of maybe 20 and those went fine.

Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 28, 2023, 01:23:42 am
I used auto import with a single drive/folder added (W:/) and ran it with 38, I'll retry with 39
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 28, 2023, 05:45:58 am
results with .39

Code: [Select]
NOT FOUND : /media/films/Back to the Future Part III/BDMV/index.bdmv
NOT FOUND : /media/films/Close Encounters of the Third Kind (Bonus)/BDMV/index.bdmv
NOT FOUND : /media/films/Die Hard 2/BDMV/index.bdmv
NOT FOUND : /media/films/Die Hard_ With a Vengeance/BDMV/index.bdmv
NOT FOUND : /media/films/Harry Potter and the Half-Blood Prince (Disc 2)/BDMV/index.bdmv
NOT FOUND : /media/films/Jurassic World_ Fallen Kingdom/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible - Ghost Protocol/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible II/BDMV/index.bdmv
NOT FOUND : /media/films/Mission_ Impossible III/BDMV/index.bdmv
NOT FOUND : /media/films/Night at the Museum_ Battle of the Smithsonian/BDMV/index.bdmv
NOT FOUND : /media/films/Night at the Museum_ Secret of the Tomb/BDMV/index.bdmv
NOT FOUND : /media/films/Scream 2022/BDMV/index.bdmv
NOT FOUND : /media/films/Sing Street/BDMV/index.bdmv
NOT FOUND : /media/films/Spy Kids 2/BDMV/index.bdmv
NOT FOUND : /media/films/Star Trek Beyond 3d/BDMV/index.bdmv
NOT FOUND : /media/films/TRON_ Legacy/BDMV/index.bdmv
NOT FOUND : /media/films/The Godfather_ Part II/BDMV/index.bdmv
NOT FOUND : /media/films/The Godfather_ Part III/BDMV/index.bdmv
NOT FOUND : /media/films/The Hunger Games_ Catching Fire/BDMV/index.bdmv
NOT FOUND : /media/films/The Hunger Games_ Mockingjay - Part 1/BDMV/index.bdmv
NOT FOUND : /media/films/The Smurfs 2/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England '88/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England '90/BDMV/index.bdmv
NOT FOUND : /media/films/This Is England 86/BDMV/index.bdmv
NOT FOUND : /media/films/Toy Story 4/BDMV/index.bdmv
Found 611 NOT 25

added 2 new misses this time but otherwise the same, new misses are

Code: [Select]
NOT FOUND : /media/films/Night at the Museum_ Battle of the Smithsonian/BDMV/index.bdmv
NOT FOUND : /media/films/Night at the Museum_ Secret of the Tomb/BDMV/index.bdmv

log sent via email, lmk if anything else needed
Title: Re: BD Folders import in library: installments missed (films)
Post by: Hendrik on July 28, 2023, 07:03:02 am
Thanks, I figured it out, it should be fixed in the next build.
Title: Re: BD Folders import in library: installments missed (films)
Post by: Matt on July 28, 2023, 10:05:41 am
Please try this build and let us know.  Thanks.
https://files.jriver-cdn.com/mediacenter/channels/v31/latest/MediaCenter310040-x64.exe
Title: Re: BD Folders import in library: installments missed (films)
Post by: JimH on July 28, 2023, 02:13:38 pm
Manni was right.  Here's the fix:

1. Fixed: Importing several Blu-ray structures with a common name could result in some being skipped.

Thanks to mattkhan for careful testing that put Hendrik on the right track to fix it.
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 28, 2023, 04:20:10 pm
.40 is more broken, it certainly fixed the previous issue but introduced some new bug as I now have 190 missing BDs in this library (sent list separately)

also note that there is nothing named index.bdmv, they're all named index.bluray;1 . Is that intentional? I suspect yes but it's certainly a change which I guess could be a breaking one?

Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 28, 2023, 04:30:57 pm
ran import again on top of the partial import and now it has loaded everything, I didn't do anything different that time so not sure what happened. I'll try another clean import tomorrow to see if same thing recurs or if it imports everything in one go.
Title: Re: BD Folders import in library: installments missed (films)
Post by: Hendrik on July 28, 2023, 05:26:41 pm
The change was very simple and isolated, finding it was the real problem. Once there, all the behavior made perfect sense. So I don't expect any side-effects from it. Essentially, when it was grouping up things for a single Blu-ray aggregate (as part of the changes to improve this in MC30), it was checking for things to start with W:\Movie, instead of W:\Movie\, hence "movie series" being affected, as they shared a common path fragment. This change alone should not have any other impact.

The naming of index.bluray is intentional and how Blu-ray structures have historically always been named - except for a brief period where a bug with the extended video analysis overwrote the name to index.bdmv.
Title: Re: BD Folders import in library: installments missed (films)
Post by: mattkhan on July 29, 2023, 03:57:51 am
retried the import from fresh, completed normally this time and did load everything with no gap
Title: Re: BD Folders import in library: installments missed (films)
Post by: Manni on July 29, 2023, 08:05:18 am
Thanks to Matttkhan and Jespermart for confirming my findings.
Thanks to Hendrik for fixing this bug.

I can confirm that the bug is fixed here. All 252 titles are imported right away with MC31 build 40 and are visible with the default view, as was the case with MC28.