INTERACT FORUM
More => Old Versions => JRiver Media Center 27 for Windows => Topic started by: comox on November 16, 2020, 12:16:27 pm
-
I have for several years been experiencing two different MC freezes. I recently fixed the first freeze and it occurred to me that my fix may provide a clue for JRiver to fix the second freeze.
I define here "freeze" as a period of 10-30 seconds when MC is unresponsive. During a freeze task manager shows nothing abnormal with CPU/disk usage and other Windows applications continue to function normally. MC always eventually resumes normally from a freeze and I have never experienced a data loss because of a freeze.
The first freeze occurred several times per session when tagging files making MC very unpleasant to use. Lately it has been getting worse making it difficult to use MC so I decided I needed to fix the problem.
My system is a high end Windows 10 desktop with multiple drives. Nothing exotic. All media is on local drives. No RAID, just simple NTFS drives. All MC program and data files are on a fast SSD. I use vanilla Windows Defender virus protection.
I did not see others complaining about the freeze so I knew it had to be something specific to my configuration. In addition, the freeze existed before and after I upgraded my motherboard and cpu so I assumed it was not a hardware problem.
The MC freeze felt similar to the freeze that occurs when Windows is waiting for a hard drive to spin up, or waiting for an optical drive to mount, or when scanning a network, so I started my hunt looking external to MC.
I methodically changed one thing at a time and then used MC for a period to see if the freeze was fixed.
I ruled out the following candidates:
1. Changed setting so my hard drives never went to sleep.
2. Added MC exclusions to Windows Defender.
3. Disabled Windows Defender.
4. Unplugged network devices like Chromecast that MC might talk to.
5. Windows Network and Sharing Center\Advanced sharing settings\turned off automatic setup of network connected devices.
6. Uninstalled MC TiVo Server plugin.
7. Disabled MC "Lookup lyrics automatically".
8. Disabled JRiver Features:
- ASIO Driver
- Burning
- DVD Ripping
- Media Network
- OpenSubtitles.org
- Podcasts
- Streaming
- Upload to Cloud
- Precision Zone Sync
By now I had concluded that the freeze only occurs when the MC tag action window is open so I shifted my focus to the configuration of my tag action window. I methodically deleted items from my configuration until I isolated the problem.
The freeze was caused by me using the Custom(TagDump) expression on [Media Type] = [Video].
As soon as I deleted this expression from the tag action window MC stopped freezing and was a pleasure to use again.
As mentioned above, I have also been experiencing a long standing 2nd freeze that occurs when I navigate via Drives & Devices to a folder with files that have not yet been imported into MC. The freeze occurs maybe 25-50% of the time that unimported files are first touched. Once MC resumes from this freeze it never occurs again on that batch of files. Only when I navigate to a different folder with unimported files does the freeze reoccur.
I have a hunch that the two freezes are related. My guess is that there is a bug in the code that reads tags from mp4 and mkv video files.
The first freeze presumably occurred regularly because the Custom(TagDump) expression was frequently executing when the tag action window was open.
The second freeze only occurs on "first touch" of a file because that is the only time MC extracts tags (unless we manually tell it to update the library from tags).
I'm hoping the JRiver team might use these clues to fix the 2nd freeze.
-
Custom(TagDump) is heavy since it needs to read the entire tags.
If you have particular files that seem overly slow, please provide a copy to matt at jriver dot com and I'll see if there's anything we could do.
Thanks.
-
You can reproduce the problem with any random collection of mp4/mkv files. I've seen it with 10's of thousands of video files.
-
You're right, it can be a little slow.
I'm not seeing anything ever come back on a video, so I might just turn it off for video files.
Thanks.
-
Thanks.
The freeze is not, I think, just slow code. Sometimes there is no delay, and sometimes MC totally locks up for 30+ seconds on my screaming fast Ryzen 7 3700X.
If you add Custom(TagDump) for [Media Type] = [Video] to your tag action window and then use MC for a few days with the tag action window open, I think you will see what I am talking about.
I suspect the reason no one else reported this problem is that the out of the box profiles for the tag action window only use Custom(TagDump) for media types Image and Audio.
-
Thanks for the thorough report.
What version of MC are you running?
A delay like that is often a drive that's unavailable. Usually a network drive.
Check your settings for File Location. Try looking at your files by location and look for drives that don't exist.
-
I forgot to say that the first thing I did was check all of my File Location settings for an unavailable drive.
The problems have existed for several years up to and including 27.0.26.
My guess is the first freeze started when the new tag action window was introduced and I customized it in a way that no else did. I think it has gotten worse in the last month or two which motivated me to get to the bottom of it.
With regard to the 2nd freeze, over the years I have seen occasional similar reports from other users that (I think) have never been resolved. It's impossible to reliably reproduce, and it's not so bad that most people probably just live with it, as I have for many (8+?) years.
I think it's the only thing left in your product that irritates me so it would be very nice to fix it.
-
Try the Weird Problems thread in my signature.
-
As mentioned above, I have also been experiencing a long standing 2nd freeze that occurs when I navigate via Drives & Devices to a folder with files that have not yet been imported into MC. The freeze occurs maybe 25-50% of the time that unimported files are first touched. Once MC resumes from this freeze it never occurs again on that batch of files. Only when I navigate to a different folder with unimported files does the freeze reoccur.
I suspect that this is not a freeze, but it can certainly be made into a freeze, or even a crash. I just crashed MC by pushing it around too much. Sadly, I didn't have logging turned on.
Just to confirm though, you are talking about the Explorer function under Drives & Devices in MC, correct?
When you open Explorer be very aware of the options at the top, because they have a large effect on how Explorer works. For example, if you select a Mode of "All Files" and tick the "Show Files In Subfolders" box, MC will read every file in the directory or drive you have selected, and import them all into a temporary database in its Library. All files!
Typically, MC shows the "Working" message in Explorer, but it isn't obvious how hard it is working. Windows Task Manager doesn't always assign much disk activity to MC, as MC offloads that work to Windows, so the activity is shown mostly under "System" using Windows Resource Monitor. [I just caused a freeze in Windows Resource Monitor while thrashing Explorer! ::)] But some of the time Task manager will show a disk is being thrashed, because Explorer reads every file, and every tag in those files, in order to import them temporarily and show the files and their tags in Explorer.
Explorer is very handy. It can even temporarily import files, show you the tags, and allow you to update those tags, but never have the file imported into the MC Main database. Neat.
But it must be used with care. If Explorer says it is "Working" you pretty much can't do anything else on the Explorer tab in MC, other than wait. If you click on anything else in the MC Explorer interface, you are likely to see a freeze. If you keep clicking around, and a lot of files are being processed, you are likely to see a crash. You can scroll around the Tree, and go to another tab, and do things. But not click on another folder in the Explorer tab, for example.
Maybe this could be improved. But when you are loading a drive at 100%, you really just have to wait.
Note that the temporary database does get purged between MC runs, I believe. EDIT: Confirmed. Purged when MC is restarted. So while you can go back and look at a folder and its sub-folders without a second wait, if you close MC and try again, you will have to wait again.
You can set up a View to show what is in the Explorer Database. The rule for file display is just "Limit database to: Explorer", or "~d=e" in short form.
I just loaded a folder which contains 412 sub-folder, 72,680 files, and 653Gb of data, mostly images, video files and Premiere Pro projects. I didn't time it, but I think it took Explorer about 15 minutes to finish working. Oh, Explorer builds thumbnails as well. Thumbnails for 71,389 jpg images from Time Lapse video creation. Also, it seems I had Audio Analysis turned on for videos, which I normally do, and Explorer used that setting. That adds some load and time.
To avoid all of the above, and still be able to use Explorer and drill down to a folder you want to look at, select a Mode of "All Files" and leave the "Show Files In Subfolders" box unticked. When you get to where you want to be in the drive structure, tick the box to view all fils below that level. Or change the Mode to a setting that suits your activity.
-
Thank you for digging into this.
I am aware of caution that must be used with MC Explorer. What you describe is not the problem I am reporting.
I am talking about navigating via MC Explorer to a flat folder with a few video files (often 1, rarely more than 10) that are not yet imported into MC. This folder is excluded from auto import so MC has not yet "seen" the files.
The freeze occurs as soon as the folder is opened (about 25-50% of the time), and usually lasts about 5-10 seconds during which MC is completely unresponsive. The other 50-75% of the time when the folder is opened there is no freeze.
After MC returns from the freeze, all subsequent import, audio analyzing, tagging, and file renaming operations always perform normally without any further freezes.
I believe that MC reads tags from video files to populate its fields the first time a file is "seen". This makes me suspect that the cause of the first freeze I solved above is related to this MC Explorer freeze.
I suspect a bug in the code that reads tags from video files, which may be triggered by the presence or absence of certain tags in video files.
Unlike the world of mp3 tags which most applications handle reasonably consistently, the world of video tags seems to be a complete mess, so it would not surprise me if there was a problem due to no fault of JRiver.
I always assume video tags are garbage in the files I download so have not spent any time correlating the freeze to specific tags.
-
I am talking about navigating via MC Explorer to a flat folder with a few video files (often 1, rarely more than 10) that are not yet imported into MC. This folder is excluded from auto import so MC has not yet "seen" the files.
The freeze occurs as soon as the folder is opened (about 25-50% of the time), and usually lasts about 5-10 seconds during which MC is completely unresponsive.
I don't think I have ever seen that issue, for folders with so few files, unless I try to move around Explorer while it is "Working". But maybe I haven't looked hard enough, or I'm being too understanding. Certainly, I notice that the Explorer startup and analysis can be quite slow. Just not a lockup... Unless I click elsewhere in Explorer while it is "Working".
Also, I just turned off most of the "Tasks" in Auto Import, which is probably used as the settings for the temporary Explorer import, and the process did speed up quite a bit.
MC doesn't do much in the way of reading or writing video files. I think it only really reads the Title tag from MP4 and MKV files, but I haven't investigated, and it isn't documented anywhere. All other fields in MC come from the filename, as interpreted by CARNAC, and metadata lookups on the internet. Then MC field values can get stored in Sidecar files.
However, it is possible that trying to read video file tags is a slow process for MC. But I don't know what tags it tries to read, other than Title.
Anyway, only JRiver developers can make informed comments. I'm just going by observations and usage.
-
Yes, the world of reading and writing video tags is murky in every application I have used.
I don't think JRiver needs to spend a lot of time on this.
I would be more than happy if MC simply stopped reading video tags.
I'm pretty sure they already don't write any video tags.
-
Just to be clear. This is not a new problem. It has been an irritation for most of the 17 years I have been managing my media with JRiver.
-
Did you ever try logging? There's a topic on the wiki. A delay like that should show up. I'm guessing that MC is waiting for something at the OS level.
You might try building a test library from scratch.
-
Attached is a log I captured from a freeze using the following procedure:
1) Start MC.
2) Reset and enable logging.
3) Navigate with MC Explorer to folder with 3 random video files that MC has not yet seen.
4) After MC returns from a 30+ second freeze, save the log.
-
Hmmm. Interesting MC environment you are running there.
Anyway...
It looks like the directory you put the three files in was H:\Test1\.
You still have Analyse Audio turned on, and these three video files were analysed. That can be quite slow.
Two files, "Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4" and "The Story of Capital Punishment - 2011.mp4" are only analysed using the analysis parameter "/AnalyzeJRVid".
The third file, "Primary_ JFK's Road to the White House - 1960.mkv", was also analyzed using a different analysis parameter "/AnalyzeDX", which I assume is because it is an MKV file with different video and audio formats in it. More tags as well, but the look.
Thumbnailing took much longer for one file, "The Story of Capital Punishment - 2011.mp4", which took 30.146 seconds to process.
Thumbnailing for the other two files only took 1.661 seconds each to process.
The total directory load looks like it took a little over 60 seconds to process.
Analysing Audio for video files can be quite slow, and varies quite a lot for different file and audio format. My understanding is that when analysing audio in video files, it is best to only analyse one file at a time.
Turn off the first five Tasks in the bottom section of the Auto Import configuration. There is no reason to analyse audio for any file when only viewing them in MC Explorer.
Repeat the test for the same files. MC will not remember these files after a restart, so there is no need to use a "fresh" unseen set of files. Repeating for the same files give you a like for like test.
Share the results.
Maybe MC Explorer should have its own set of Auto Import settings, where many things can be turned off.
Most relevent log lines.
Line 179: 0010874: 2440: General: CMainUIWnd::SetMCView: View info name: Test1
Line 196: 0010878: 18868: Import: JRAnalyzer::AddFile: Filename: H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4
Line 198: 0010878: 9456: Import: JRAnalyzer::AddFile: Filename: H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv
Line 203: 0010878: 276: Import: JRAnalyzer::AddFile: Filename: H:\Test1\The Story of Capital Punishment - 2011.mp4
Line 205: 0010892: 18868: Import: JRAnalyzer::AddFileJRWorkerExe: Parameters: /AnalyzeJRVid "H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 18868.xml"
Line 207: 0010892: 18868: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /AnalyzeJRVid "H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 18868.xml"
Line 211: 0010903: 276: Import: JRAnalyzer::AddFileJRWorkerExe: Parameters: /AnalyzeJRVid "H:\Test1\The Story of Capital Punishment - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 276.xml"
Line 213: 0010903: 276: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /AnalyzeJRVid "H:\Test1\The Story of Capital Punishment - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 276.xml"
Line 219: 0010908: 9456: Import: JRAnalyzer::AddFileJRWorkerExe: Parameters: /AnalyzeJRVid "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 9456.xml"
Line 221: 0010908: 9456: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /AnalyzeJRVid "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 9456.xml"
Line 238: 0000020: 5348: Playback: CJRVideoEngine::Open: Opening file H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4
Line 240: 0000021: 10084: Playback: CJRVideoEngine::Open: Opening file H:\Test1\The Story of Capital Punishment - 2011.mp4
Line 333: 0040998: 9456: Import: JRAnalyzer::AddFileJRWorkerExe: Parameters: /AnalyzeDX "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 9456.xml"
Line 335: 0040998: 9456: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /AnalyzeDX "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Analyze - 9456.xml"
Line 341: 0000004: 4600: Import: CJRVideoAnalyzeHelper::AnalyzeFileDX: Filename: H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv, File Type: mkv
Line 410: 0041139: 6868: Database: Thumbnail Thread (H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4): Start
Line 412: 0041139: 6868: General: CThumbnailCacheInfo::CreateThumbnail: Filename: H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4
Line 415: 0041139: 15272: Database: Thumbnail Thread (H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv): Start
Line 418: 0041139: 15272: General: CThumbnailCacheInfo::CreateThumbnail: Filename: H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv
Line 420: 0041139: 2592: Database: Thumbnail Thread (H:\Test1\The Story of Capital Punishment - 2011.mp4): Start
Line 422: 0041139: 2592: General: CThumbnailCacheInfo::CreateThumbnail: Filename: H:\Test1\The Story of Capital Punishment - 2011.mp4
Line 424: 0041140: 6868: General: CThumbnailCacheInfo::CreateThumbnailUsingCommandLineGrabber: Command line: /Thumbnail "H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 6868.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 6868.xml" 15000
Line 426: 0041140: 6868: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /Thumbnail "H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 6868.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 6868.xml" 15000
Line 427: 0041141: 2592: General: CThumbnailCacheInfo::CreateThumbnailUsingCommandLineGrabber: Command line: /Thumbnail "H:\Test1\The Story of Capital Punishment - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 2592.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 2592.xml" 15000
Line 428: 0041141: 15272: General: CThumbnailCacheInfo::CreateThumbnailUsingCommandLineGrabber: Command line: /Thumbnail "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 15272.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 15272.xml" 15000
Line 431: 0041141: 2592: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /Thumbnail "H:\Test1\The Story of Capital Punishment - 2011.mp4" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 2592.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 2592.xml" 15000
Line 432: 0041141: 15272: General: RunProgram: Filename: C:\Program Files\J River\Media Center 27\JRWorker.exe / Parameters: /Thumbnail "H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Image - 15272.png" "C:\Users\Rob\AppData\Roaming\J River\Media Center 27\Temp\Video Metadata - 15272.xml" 15000
Line 455: 0000022: 14940: Import: 0x3a5c CThumbnailGrabberWnd::GetThumbnail: Grabbing thumbnail from H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv at 15 sec
Line 493: 0000026: 14020: Import: 0x36c4 CThumbnailGrabberWnd::GetThumbnail: Grabbing thumbnail from H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4 at 15 sec
Line 938: 0042801: 15272: Database: Thumbnail Thread (H:\Test1\Primary_ JFK's Road to the White House - 1960.mkv): Finish (1661 ms)
Line 940: 0042801: 6868: Database: Thumbnail Thread (H:\Test1\Hans Litten vs. Adolf Hitler_ To Stop a Tyrant - 2011.mp4): Finish (1661 ms)
Line 1125: 0071286: 2592: Database: Thumbnail Thread (H:\Test1\The Story of Capital Punishment - 2011.mp4): Finish (30146 ms)
-
Thanks for interpreting the log for me.
Yes I navigated to a folder named Test1.
I checked my auto-import settings:
1) Analyze audio and video are both turned off. I always manually initiate analysis.
If analysis started by navigating to the folder I think that is a bug.
In addition, if the analysis did automatically run, the results were not saved because initiating a manual analysis shows the files still need to be analyzed.
A manual analysis of the 3 files in parallel takes 23 seconds so this suggests the culprit may be thumbnailing, but I still hope Matt looks into my video tag reading hypothesis.
FYI, I have a fast 8 core system so I always analyze 8 files in parallel.
2) Build thumbnails is turned on.
I will turn this off, do some tests, and report back.
One very strange thing (that I already reported to Matt) is that if I rename folder Test1 to something else and rerun the test I can experience no freeze. In other words the same video files produce inconsistent results which is one of the reasons I have just lived with this problem for many years.
-
I turned off build thumbnails in the auto import settings are re-ran my tests.
I'm still seeing the same very long freezes.
-
Although maybe my analysis is wrong...
I just viewed a directory containing three short (17 min, 3 min, 33 seconds) virtually identical MP4 action camera videos, with most of the Auto Import settings turned off, and it took MC Explorer about 30 seconds to display the contents, confirmed in a log.
All three files were still analyzed using "/AnalyzeJRVid", and the second 3 minute file was also analyzed using "/AnalyzeDX". No thumbnails were built. No tags read.
Once again, the file analysed using "/AnalyzeDX" took about 30 seconds, but the other two took less than 2 seconds.
There is a Timeout in the thread for the problem file when it is running the "/AnalyzeJRVid" analysis. The JRWorker process times out. The run time for the failed process is 30.155 seconds. MC then launches another JRWorker process to do the analysis using "/AnalyzeDX", which finishes in 0.127 seconds.
Comox, your log has two 30 second timeouts for JRWorker processes. One for the thumbnailing process for "The Story of Capital Punishment - 2011.mp4" file, and one for the "/AnalyzeJRVid" analysis for "Primary_ JFK's Road to the White House - 1960.mkv" file. So that explains the 60 second elapsed time.
Maybe MC can't have more than two JRWorker processes running the same function at the same time? Although that seems unlikely.
If the timeout is the issue, it sounds like a fixable problem. I can't see a reason for the timeouts yet.
-
Actually, the reason for the timeouts isn't logged, so any analysis of that would have to be done in a debugging environment.
-
MC will not remember these files after a restart, so there is no need to use a "fresh" unseen set of files. Repeating for the same files give you a like for like test.
I may have been wrong about that. I just re-ran my test after restarting MC and the response was instant. I shall look further.
EDIT1: Nope. The Explore Database is cleared after restarting MC. So my latest test must not have suffered a timeout. Checking log...
EDIT2: No timeouts. All "/AnalyzeJRVid" analysis processes completed successfully in about 0.3 seconds.
Maybe the underlying Windows file processes are an issue? The file gets locked by MC before it gets opened? I'm guessing. :)
-
Excellent! You are seeing the same problem with the same inconsistencies that I am.
Now if you go and add TagDump for file type video in the tag action window, and navigate around some video files with the tag action window open, I think you will see why I think the two issues have the same underlying cause.
-
Do you mean navigate around in MC Explorer, or just in general Views?
I did have the TagDump included in the Tag Window for movies until recently. I'll have a look again.
-
I don't think it matters. Just click on different videos to cause TagDump to execute.
-
Do not use the latest beta because Matt has disabled TagDump for video.
-
I noticed the Beta change, since I update the Wiki. ;D
It did cause some long pauses, which you might call a freeze, in my test Library. I even crashed MC by continuing to click around while it was trying to work. Those sorts of things shouldn't happen, but they do.
I need to try the Video TagDump on my real Library, which is larger.
I'm not sure how this TagDump issue would be causing the timeouts I see in the MC Explorer logs though. Trying to read video tags may take some time, but 30 seconds per file, and causing a timeout for JRWorker, seems unlikely. Not impossible I guess.
Also, the inconsistency we are seeing may have something to do with Windows caching of files, even though MC has been restarted before my tests. Not sure.
-
The question, I think, is why does reading a piece of the file take 30+ seconds one time, and is near instantaneous the next time?
Perhaps it's not about the reading the tags. Perhaps it's about opening the file to do anything (build a thumbnail, get the tags, determine the codec types, etc.).
-
The good news is you do not have to worry about the freeze corrupting your database.
Over many years it has always returned from the freeze with everything intact.
-
I don't think it takes 30 seconds to read a file. I think something fails, and it takes 30 seconds for MC to timeout the JRWorker process it was waiting on.
-
You were right Comox.
The same error, in an overall sense, is causing the freeze in MC Explorer and when navigating video files with the Tag Window open, and the TagDump group displayed.
I used MC27.0.33 to produce the freeze on my Workstation with the small Library, since I can analyse what is happening so much easier there. I could reliably cause the freeze by clicking a few different episodes of a TV Show.
The logs show that what looks like the main MC thread is waiting for a JRWorker to analyse the video file. In the case I looked at in detail, the analysis using both the "/AnalyzeJRVid" and "/AnalyzeDX" parameter timed out after 30 seconds, allowing the main MC process to continue.
As previously, any analysis of why the JRWorker process times out would have to be done in a debugging environment. There are no obvious resource spikes that I can see using Task Manager and Resource Monitor.
I couldn't produce the freeze once I removed the TagDump group from the Tag Window.
Putting the Tag Dump group back immediately caused a freeze.
I tried upgrading to the MC version with the fix, and was prevented, as two JRWorker processes were stuck in Task Manager. It looks like they not only timed out, but MC failed to kill the processes. Resource Monitor said the processes were running normally, and they didn't have any process/threads they were waiting for, in the Wait Chain. I created dump files of the two JRWorker processes, after I had closed MC and attempted to upgrade, if anyone wants to see them.
I killed the JRWorker process and installed the MC update with the fix. I could no longer cause the freeze with the Tag Dump area displayed. So I guess the code behind those two JRWorker parameters, "/AnalyzeJRVid" and "/AnalyzeDX", had issues. It certainly shouldn't take more than 30 seconds to read the tag area of a video file, and so those operations should have either completed and returned nothing, or failed cleanly.
I wonder if this was the only function that was causing all those elusive freezes/lockups in MC. If so, you have done everyone a great service Comox. The problem in the past was always being able to repeatably produce the lockups. The analysis of the issue, as far as I can go, isn't hard once a lockup can be produced on demand.
Kudos.
-
Build with the tag dump off for videos here:
https://yabb.jriver.com/interact/index.php/topic,127629.0.html
RoderickGI said this build was good on his system.
-
Thanks RoderickGI for investigating and thanks Matt for trying to fix it.
1) I suspect the reason you had a couple stuck JRWorker processes is that you clicked around MC during the freeze which can cause a crash. If when a freeze starts you are patient and don't click around, MC will eventually return and there will be no stuck processes.
2) 27.0.34 only fixes the freeze that occurs when a video is clicked with the tag action window open and TagDump active. It does not fix the freeze that occurs when a folder containing one or more previously unseen videos is opened with MC Explorer.
I expect the freeze problem is located in code shared by TagDump and MC Explorer for opening and reading the contents of a video file.
-
I hadn't actually caused a crash in this latest testing on MC27.0.33, but the two JRWorker process were left behind anyway. However, I just reinstalled that version and ran the test three times, and JRWorker was eventually cleaned up in those tests.
I hadn't checked the MC Explorer issue in MC27.0.34, but just tested, and yes, the issue is still there. As I said earlier, it looks like it is in the code behind those two JRWorker parameters, "/AnalyzeJRVid" and "/AnalyzeDX".
-
An interesting question is, why do so few people report this problem given how easy it is to reproduce, and given that it's been present for many years?
My guess is that most people use auto-import to import video files, which if true, and assuming the freeze also occurs during auto-import, then it will happen in a background thread that is not visible or annoying.
I have noticed that the time it takes for auto-import to complete varies widely from one manual execution to the next.
-
The problem is actually reported widely, but with a different description. It is usually reported as the MC GUI expands, goes blurry, and perhaps gets a white cast. Much like when the Windows message pops up that says MC has stopped responding, do you want to wait or close it? I think that is the same problem; the MC main thread is waiting for a process that has failed, and the main thread won't respond until that process times out.
There are also a lot of reports of freezing, with mostly no apparent reason. Not all, or maybe even many, of the freezes are caused by trying to read tags from video file, but many are never resolved.
As I said earlier, the real reason those problems haven't been fixed is that they were very difficult to reproduce on demand. This is almost the first time a reproducible freeze, by multiple people, has been reported.
Personally, I've always known that the MC Explorer function is doing a lot of work, so I've given MC some slack and not complained. Also, I rarely have the Tag Window open and then navigate around MC Views. I mostly do tag editing in the Views themselves, so did't see the TagDump issue. Perhaps I should have complained, or done some detailed analysis. But, well, I'm not JRiver quality control or software tester, and it hasn't affected me often. I think most people feel the same and just dismiss the problem and move on, sometimes to other software altogether.
Auto Import may be another issue, as doing Audio Analysis of video files can take a long time, and is dependent on the video size, format, etc. So if users have Audio Analysis for video files turned on in Auto Import, as I do most of the time, then the import can be very slow and variable. It also depends on how many files are analysed at a time (a global setting), how many CPU cores and CPU Threads are supported on the hardware, the performance of the drive on which the files are stored, and so on.
-
Good points.
Auto-import may or may not be a separate issue. I have used pretty much every day for 17 years auto-import exclusively as a system and database integrity check. All of my media files are in folders scanned by auto-import and they are all imported, analyzed, and have their thumbnails built. If auto-import completes and reports zero files changed, added, or removed, then I feel confident that nothing in my file system or MC database has screwed up.
I would therefore expect these daily manual runs of auto-import to take roughly the same length of time, but instead they vary widely from the normal blinding fast 1-2 seconds, to an occasional 15 -30 seconds.
-
There is probably some dependence on what Windows is doing, and if you haven't excluded your media file locations from security software, what it is doing. Maybe some other applications. If something else locked a file at a critical time, MC would have to wait. Maybe such a lock would see the first attempt at checking a file fail, so a timeout would be involved.
The Windows cache manager can make a huge difference to file reads.
-
You might be right.
All of my media is excluded from Windows Defender, and no background file apps like Dropbox are permitted to touch my media.
All I'm saying is it's suspicious based on thousands of observations across many hardware and operating systems.
A good argument against my speculation is that auto-import does not need to open any video files when I manually execute it because it has already seen all the files. So if the occasional extra time is related to the freeze, then the freeze would have to be caused by something other than opening a video file, which seems improbable given what we learned with TagDump.
-
MC will sometimes read a file again even if it has been seen before. For example, if for some reason the file's Date Modified has changed by something outside MC. So it would be possible for a video file to still be causing the issue.
You could test to see if video files are involved simply by excluding all video files from your manual runs of Auto Import, even if temporarily. If you still get the slowdowns when video files are excluded, then there is something else going on. I imagine that you would have to run the same test for some time to gather data on whether slowdowns occur though, given that the slowdowns are intermittent.
Note that MP4 files are included with Audio files as well as Video files in the Auto Import configuration. So you would have to turn off MP4 under Audio as well.
You could also just log all future manual runs of Auto Import and search for "timeout" in them. Just reset the log before manually running Auto Import so that you get a nice short log. From there, it isn't hard to see which file caused any timeout.
I use Notepad++ to view logs files, which makes the analysis much easier than using Notepad, for example.
The search dialogue includes a "Count" function, which quickly shows if there is an occurrence of "timeout".
Then I use the "Find All in Current Document" function to get a summary of instances with line numbers.
That gives me the "Process ID" related to the timeout message(s), which I then do a Count of as a sanity check. If there are a lot of lines, then it may have been the main thread in MC that timed out. If there are only a few, it would probably be a sub-process, probably from a JRWorker process.
Then use the "Find All in Current Document" function on the "Process ID" to get all lines related to it. (Double-clicking on a line in the summary displayed takes you to the line in the log file, if you need to see what is around it. Not usually necessary, and can be confusing.)
It isn't hard then to work backwards within the summary display from the line number of the timeout message to find the file that caused the issue.
It helps to know that in the log, the numbers at the left represent, in order:
The Line Number in the log. Purely a Notepad++ number. Not actually in the file.
The timecode of the line, in milliseconds, starting from when logging was initiated.
The "Process ID" associated with the message.
I don't know what all the log line types or content means, but many can be worked out, or guessed.
See, easy! ;)
The attached image shows an example of a summary display for a "Process ID" from a timeout analysis. Any non-zero "Result" is a failure. Both the log message and the timecode of the lines show the failure took ~30 seconds.
-
Excellent tips for analyzing log files. Thank you very much!
Sorry I was unclear on how I use auto-import. I never modify my media files after I have imported, analyzed, tagged, and renamed them. So if a media file changes (or one is added or deleted) then that means either I made a mistake, or some other app is misbehaving, or my file system has a problem, or something got screwed up in the MC database. Auto-import reports no changes 99.99% of the time. When auto-import does reports a change, I immediately know a problem exists.
Suggest we drop this line of inquiry. I don't consider the occasional auto-import delay to be a problem. I was just highlighting it as something suspicious that might be related to the freeze because it has about the same length of delay, and is also inconsistent.
-
I'm happy to leave it here. I was interested to document what we found and did so that if someone comes up with another reliably repeatable cause of freezes in MC, I could test using the same process, and look for the same issue.
Cheers.
-
Same as comox, I live with these freezes since years. It happens when a view includes several sleeping drives with movies.
MC tries to access the files which are visible in my view and fails if several drives have to be started at the same time. As long only one drive is involved, this works in background and UI keeps responding.
There would be a very simple solution: Do not access the media file unless playback or any manual tagging operation is requested.
This works fine. If I disconnect the drives, everything is handled by database and thumbnail files. On start playing I have to insert the requested drive. It is not that comfortable, but less frustrating than these GUI freezes.
My movie collection takes more than 40TB. There is no single drive big enough to hold them all. So they are stored on 8 drives. Some are accessed less than once a month. So they will go sleep for sure.
-
Some of the freezes occurs when MC is trying to load image for video from video folder. If video is on NAS which is sleeping the UI will freeze even though one is just browsing in the theather view.
I dont't know why MC needs file access to NAS when there is local thumbnail available but it does as for movies the problem went away with ZRatings, which can move posters to specified folder i.e. away from NAS but problem remains for TV shows. To help with this I have suggested that also video files should have same kind of specific cover art folder possibility than the audio already have.
https://yabb.jriver.com/interact/index.php/topic,126160.msg885320.html#msg885320
-
Some of the freezes occurs when MC is trying to load image for video from video folder. If video is on NAS which is sleeping the UI will freeze even though one is just browsing in the theater view.
Configure Windows Defender. Read InflateableMouse's post here: https://yabb.jriver.com/interact/index.php/topic,127841.msg886994.html#msg886994
-
Remember that the default rule in the statusbar will have to spin a drive up, so change that if that's a problem:
[Name] (FixCase([File Type], 3) - [Size] - [File Size]) If(IsMissing(), /(file not found/),).
Also, make sure the option Tree & View > Advanced > Display missing file image in lists is off.
-
Configure Windows Defender. Read InflateableMouse's post here: https://yabb.jriver.com/interact/index.php/topic,127841.msg886994.html#msg886994
Come on... :) I just told that for movies it was fixed by third party SW which moves cover images out of NAS. (Defender is configured and anyway it is MC which is requesting cover images. Defender check would just be another slowdown)
In my case also statusbar and missing file image is configured (This is the point actually. MC is fetching image file from NAS which is sleeping, when it could use either thumbnail or if the cover image path could be set outside NAS when thumbnail is not big enough)
Image loading is also related to view refresh. After freezing for ~30s MC works properly for a while.
-
I can confirm Iepa's post.
ProcessMonitor shows that MediaCenter accesses EVERY file which is shown by the Movie View. Even theses Movies were imported long ago and all Thumbnails built.
I'm not asking you for difficult things. I just ask not to do something. Not to access media files unless Play, Reindex, Tagging, Move or Rename is requested.
Do I have to put that on your "Things we should have done because they were so simple" list ?
-
Remember that the default rule in the statusbar will have to spin a drive up, so change that if that's a problem:
[Name] (FixCase([File Type], 3) - [Size] - [File Size]) If(IsMissing(), /(file not found/),).
Also, make sure the option Tree & View > Advanced > Display missing file image in lists is off.
Both is checked and fine!
I 'think', that spinning up one single drive does not lead to these blockings. My views can show Movies from different drives. Spinning up several of them seems to block the UI.
-
I think rolf-eigenheer is discussing a different freeze. As explained above, I eliminated sleeping drives as a possible cause of the freeze I observe.
Last night I observed a new clue. I imported and edited an M4B audiobook and MC froze for 30ish seconds almost continuously. M4B is another of those formats for which reading and writing tags is murky. As soon as I converted it to MP3 and deleted the M4B the freezing stopped.
-
Here is a refined hypothesis on what is causing the freeze I observe.
The freeze occurs when tags are read and/or written to certain file types. (I don't know if it occurs on read or write or both.)
Something fails in the tag reading/writing code until it times out in about 30 seconds. MC is unresponsive during the freeze.
Let's define 3 groups of file types:
Group1: no tags
This group includes file types like AVI, IFO, TXT, SRT, etc. for which it is not possible to store tags in the file. MC makes no attempt to read or write tags and a freeze never occurs with this group.
Group2: reliable tags
This group includes file types like MP3 and JPG for which reading and writing tags is always reliable. Freezing never occurs with this group.
Group3: unreliable tags
This group includes file types like MP4, MKV, M4B for which it is possible to read and write tags but doing so is not reliable and sometimes a freeze occurs until the code times out.
When Matt disabled Tag Dump in the Tag Action Window for video files we successfully papered over a portion of the problem by disabling tag reading/writing for Group3 video files when using the Tag Action Window to read and write tags.
But this fix did not solve the problem for non-video in Group3, like M4B.
Nor did this fix solve the problem when tag reading/writing is triggered by means other than Tag Dump in the Tag Action Window. For example, when a previously unseen file is selected using the MC File Explorer.
-
Here is a refined hypothesis on what is causing the freeze I observe.
I agree. Something to do with tag reading is broken.
-
If this is a difficult problem to solve, I would be more than happy with a quick and dirty fix that disables tag reading/writing for all Group3 file types.
-
Another clue. Don't know if it's related but it's suspicious since it involves reading tags.
MC has for several years struggled to correctly identity the compression type for some video files.
Today I imported a 1080p x265 mkv file and MC reported the compression as "mkv video (video: HEVC, audio: LAV AAC ADTS)".
I knew from experience that this was wrong so I clicked "Update Library (from tags)" to refresh the compression detection and MC correctly reported "mkv video (video:hevc, audio: aac)".
The same file caused a 30 second freeze on import (as described above), and the "Update Library (from tags)" operation also (sometimes but not always) caused a 30 second freeze and so I wondered if the two problems are related.
-
After a couple days of testing I am optimistic that 27.0.61 fixed this freeze.
We'll probably need another couple weeks to know for sure but it looks good.
I'm assuming it was this change:
"Fixed: The video analyzer could sometimes take a long time."