INTERACT FORUM

More => Old Versions => Media Center 14 (Development Ended) => Topic started by: rpalmer68 on July 21, 2009, 11:48:03 pm

Title: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on July 21, 2009, 11:48:03 pm
In recent builds I've noticed that after starting MC it is almost totally unusable for about 7 minutes after running for about 1 or 2 minutes.

I can't navigate around in theater view as it's so slow and video playback/TV playback is choppy.

Upon investigation JWorker.exe sits around 50 to 70% cpu in task manager during this time and looking at the logs it seems to be while thumbnails are build biult after the first auto import.   (Which currently imports about 25 video files)


If I turn off the option to build thumbnails on auto import, that improves things but as soon as I start navigating around in Theater View and browse to my views with my Video recordings (that are imported with auto import) MC grinds to a halt while the thumbnails build.

I'm running XP SP3 on a Quad Core Intel processor @ 2.66Ghz so really don't expect to see this level of slow down from MC.

Have update DirectX to latest version and the latest ATI drivers for my 4850 GPU.

I thought thumbnails were being done in their own core now on multi-core processors?

Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: fitbrit on July 21, 2009, 11:54:04 pm
I do notice a slow thumbnail building too. I considered that this may be due to the cover art being stored on an unraid server, and that the disks may need to be spun up. However, it's still pretty slow even factoring those details.
Title: Re: MC almost unusable while thumbnails are built.
Post by: raym on July 22, 2009, 12:40:18 am
I do notice a slow thumbnail building too.

Thought it was just me.. Pretty sure it's thumbnail related in my case too. Sometimes I'll start MC at it'll be unresponsive for 5-20 seconds. Havn't seen this kind of problem in MC for ages.

I'm using a shared (networked) db so that may have something to do with it. I can't say I've seen this issue with a local instance.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 22, 2009, 01:09:52 am
Thought it was just me.. Pretty sure it's thumbnail related in my case too. Sometimes I'll start MC at it'll be unresponsive for 5-20 seconds. Havn't seen this kind of problem in MC for ages.

I'm using a shared (networked) db so that may have something to do with it. I can't say I've seen this issue with a local instance.

I can have MC unresponsive for 5 to 10  seconds on load too, but this issue is for like 7 minutes, MC still responds but VERY slowly making navigation/use hopeless until it finished building thumbnails for the newly imported files.

The files being imported are on a local drive, but the library is on a NAS although opened read only.

R
Title: Re: MC almost unusable while thumbnails are built.
Post by: Matt on July 22, 2009, 08:48:40 am
What file types are being thumbnailed?  Does it need to use some fancy filters to render a little bit of the movie?  How does thumbnailing in Windows Explorer compare?

Since thumbnails happen in a low priority thread that runs a low priority process for videos, it seems like it should never interfere with the main program.
Title: Re: MC almost unusable while thumbnails are built.
Post by: leezer3 on July 22, 2009, 09:08:51 am
This may or may not be related, but I've posted in the past about removing files from the Recently Imported view.
More often than not, these are videos with the thumnailing in process, and it quite regularly causes 10- 20second hangs.

The database is local, but pretty much any file getting newly imported is on a network server.
Am I seeing a pattern whereby one part or the other is located on a network drive?

-Leezer-
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 22, 2009, 10:16:08 am
What file types are being thumbnailed?  Does it need to use some fancy filters to render a little bit of the movie?  How does thumbnailing in Windows Explorer compare?

Since thumbnails happen in a low priority thread that runs a low priority process for videos, it seems like it should never interfere with the main program.

They are MPEG (.mpg) files.

Even with MC otherwise idle, Media Center14.exe and JWorker take the CPU to over 88% while the  thumbnails are generated.

Here is a log from MC being launched until things settle down. 

During these several minutes my total CPU ranged from 31% to 100% with Media Center 14.exe peaking at about 25% and up to three JWorker.exe processes each taking up to 25% as well!

No wonder the system grinds to a halt!

I've only recently started to notice it being this bad, and in fact in the last week or so I've noticed a noticable increase in the number of times video will stutter (I think from high CPU spikes) during playback even after the system has settled down.

Richard




Title: Re: MC almost unusable while thumbnails are built.
Post by: Matt on July 22, 2009, 10:25:00 am
During these several minutes my total CPU ranged from 31% to 100% with Media Center 14.exe peaking at about 25% and up to three JWorker.exe processes each taking up to 25% as well!

Normally if you start a low priority thread and work like crazy in it, it won't be noticeable because the system only gives the thread cycles when there's nothing else to do.

So it should be alright for a background process to use even 100% of the CPU.

However, it seems like the priority part of the thumbnailing isn't right.  We set the process priority of JRWorker.exe to the lowest, but DirectShow sets its own thread priorities when rendering the file, and perhaps this is the problem?  It's also possible that hardware acceleration is getting invoked or video memory allocated which could cause a weird slowdown?
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 22, 2009, 10:30:03 am
Normally if you start a low priority thread and work like crazy in it, it won't be noticeable because the system only gives the thread cycles when there's nothing else to do.

So it should be alright for a background process to use even 100% of the CPU.

However, it seems like the priority part of the thumbnailing isn't right.  We set the process priority of JRWorker.exe to the lowest, but DirectShow sets its own thread priorities when rendering the file, and perhaps this is the problem?  It's also possible that hardware acceleration is getting invoked or video memory allocated which could cause a weird slowdown?

The strange thing is I'd don't ever remember seeing this issue in the past (like a month ago) ... I'll have to go back and try MC13 with the same set of files, but I'm sure it didn't do this.

Cheers
Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: glynor on July 22, 2009, 11:02:23 am
Just a note... I used to have all KINDS of problems with this when it was building video thumbnails on my system.  Even a quite powerful PC (Q9550 @ 3.6GHz with 4GB RAM) was brought to it's knees when MC would try to thumbnail a folder full of new 720p x264 MKVs.  The issue has been present for a long, long time with multiple versions of MC.  I pleaded for help on this repeatedly, asking that all thumbnailing activities be suspended at least while full-screen playback is active (to not interfere with smooth playback while you are obviously busy watching something in the foreground).  But, no action was ever taken on this request...

However, I was able to solve it completely with a simple, cheap, purchase that has been really helpful in a number of other ways: CoreAVC (http://www.coreavc.com/).  I think FFDSHOW just doesn't respect priorities well, and doesn't react well to generating multiple thumbnails of HD content the way MC wants to do it.  With CoreAVC installed and selected as the default MPEG-4 decoder on the system, my issues vanished completely.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 22, 2009, 05:45:52 pm
 I pleaded for help on this repeatedly, asking that all thumbnailing activities be suspended at least while full-screen playback is active (to not interfere with smooth playback while you are obviously busy watching something in the foreground).  But, no action was ever taken on this request...

I'd second this request.
Title: Re: MC almost unusable while thumbnails are built.
Post by: raym on July 22, 2009, 05:49:54 pm
I'd just prefer performance of the thumbnailing is restored to how it used to be. As you've mentioned already, this has only been an issue recently.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 22, 2009, 06:02:49 pm
I'd just prefer performance of the thumbnailing is restored to how it used to be. As you've mentioned already, this has only been an issue recently.

I think the issue is that several jworker.exe processes are now being used to generate the thumbnails, so it's done a lot quicker but at the expense of system performance.   I'd like to see thumbnailing go back to just one process, but also think stopping thumbnailing in the background during video playback would fix the last of the playback stutters I get from time to time.

Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: raym on July 22, 2009, 07:30:55 pm
but also think stopping thumbnailing in the background during video playback would fix the last of the playback stutters I get from time to time.

I guess the point I'm trying to make is I'd prefer background thumbnailing to stay but the associated performance degradation to be resolved because unless it is resolved, we're likely to just experience related issues when trying to navigate around in Theater View (for example). Preventing thumbnailing during fullscreen playback will avoid some of your video stuttering issues (possibly) but it's not a real solution to the problem IMO.

Cheers.
Title: Re: MC almost unusable while thumbnails are built.
Post by: glynor on July 24, 2009, 03:41:06 pm
I guess the point I'm trying to make is I'd prefer background thumbnailing to stay but the associated performance degradation to be resolved because unless it is resolved, we're likely to just experience related issues when trying to navigate around in Theater View (for example). Preventing thumbnailing during fullscreen playback will avoid some of your video stuttering issues (possibly) but it's not a real solution to the problem IMO.

I agree.  Unfortunately, I doubt that there is much they can do about the performance of a third-party decoder like FFDSHOW, which is really the root of this issue.

Some things you can do to help reduce (but probably not eliminate) the issue:
1. Turn off any and all Video postprocessing (deinterlacing, post, sharpen, filtering, etc) done in FFDSHOW by default.  If you want these to be done on-demand when watching files, add them to a non-default profile, and activate it on-the-fly when needed.
2. Change FFSHOW Video decoder "multiple instances" control to "No Limitations".  This is in the FFDSHOW Video Decoder Configuration utility, under DirectShow Control.
3. Try enabling Queue support in FFDSHOW.  This enables some more intelligent queuing of decode jobs, but isn't widely supported by some hardware configurations.  I don't know if it will work with MC well, but it seemed to with my limited testing.  NOTE: This will have the largest impact if you have a dual-core or better CPU that has multiple thread hardware support.  To enable Queuing support, follow these steps:
3a. In the FFDSHOW Video Decoder configuration utility, go to the Queue and Misc "tab" (from the tree on the left-side).
3b. Check the Queue output samples box.
3c. UN-check the Use queue only in: box (or add MC to the list)
3d. Enable queue in VMR9-YV12 (this may cause problems with Intel integrated graphics hardware)
4. Make sure all SMID instructions support is enabled (under Info & CPU in the FFDSHOW Video Decoder configuration utility).

Again, if you want to really SOLVE the problem once and for all, simply buy CoreAVC, which is available for only $15 US.  For me, after testing out the trial, the $15 spent on CoreAVC was WELL WORTH my time and effort spent trying to solve this and other performance problems with HD MPEG-4 ASP and AVC playback.  It also solved a problem I'd been having with BeyondTV completely, and I really feel that the quality is much better as well.

It has far fewer Post-Processing capabilities, but you could always route through FFDSHOW for post-processing if you wanted on-demand.  I, personally, haven't seen the need as quality has been very good out-of-the-box.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 24, 2009, 04:23:11 pm
I'm not using FFDSHOW  for .mpg files, plus they are not HD recordings so coreAVC won't actually decode them anyway will it?

If MC limited thumbnail building to just one jworker.exe process that would also help I would think.

R
Title: Re: MC almost unusable while thumbnails are built.
Post by: glynor on July 24, 2009, 04:37:33 pm
Wow... You're seeing issues with vanilla MPEG-1 files?  What kind of ancient machine are you running it on?  If it isn't ancient, you probably have other issues, or aren't seeing the same issue that I am.

EDIT: Sorry, hadn't read every single post above very well, and skimmed most of it.  I don't think you should see issues like this with MPEG playback, unless something else is going on.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 29, 2009, 03:29:46 am
Wow... You're seeing issues with vanilla MPEG-1 files?  What kind of ancient machine are you running it on?  If it isn't ancient, you probably have other issues, or aren't seeing the same issue that I am.

EDIT: Sorry, hadn't read every single post above very well, and skimmed most of it.  I don't think you should see issues like this with MPEG playback, unless something else is going on.

LOL, it's the machine you helped me overclock.... and she's going great thanks!

Some of my files are HD some SD so maybe it's the HD ones slowing things down. But it was never a problem with MC13.

I think becuase multiple jworker threads are being started in MC14 now they are pulling too much CPU, sure it could alsobe  filter related but if we just had one thumbnailing thread I'm sur my quad core could cope!  Let's face it MC as to cope with lots of flter combinations we can't say "hey if you want MC to be usable while it's building thumbnails you have to buy a special filter"

I got the frame rate display up in theater view during this slowness at it drops down to 8 FSP (from 60 when everything is normal).

Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: glynor on July 29, 2009, 09:12:25 am
If they're HD MPEG files, they can't be "typical" MPEG-1 files.  While officially the MPEG-1 standard can encode up to 4095x4095@60fps, typical "MPEG-1" files used are actually of the MPEG-1 CPB subtype (Constrained Parameters Bitstream), which is limited to the MPEG-1 SIF (Source Input Format) which is 352x240@30fps (NTSC) or 352x288@25fps (PAL).  Going any higher than that on a MPEG-1 stream is really a huge waste of resolution because the MPEG-1 compression will turn it into a blocky mess no matter what resolution you feed in and out of the codec.

I've actually NEVER seen an MPEG-1 encoded stream at higher resolution than SIF.  Ever.  It just isn't done, and the encoder software apps don't support it.

It is far more likely that they are actually MPEG-2 video streams in a MPG wrapper.  HD MPEG-2 streams are far more simple to decode than, say, H264 or VC1 streams, but still require an external decoder (probably FFDSHOW) and loading multiple instances of FFDSHOW is almost certainly your problem.  That, or they're actually mislabeled MPEG-2 TS containers with even H264 (MPEG-4 AVC) content inside.  Open one of them with MediaInfo (http://mediainfo.sourceforge.net/en) and see what the actual stream is inside of the file.

Did you try any of the FFDSHOW tweaks I mentioned above?

That said, I agree fully that the Thumbnailing in MC is far too aggressive by default.  I think perhaps you should be able to "turn it up" if you have a machine and filters that can handle it, but by default it should be much slower and more careful about system resources.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 29, 2009, 09:24:35 am

Did you try any of the FFDSHOW tweaks I mentioned above?

Are you suggesting that FFDSHOW is being used to thumbnail even though I'm using The Cyberlink Video Decoder... must go back and check the logs...?

I don't know what the mpeg files really are to be honest, MC reports them as 1920x1080 and a 2 hour show is about 13GB.

The thing is this had gotten worse in recent times using the same codecs for some reason, in fact I was watching a recorded show tonight and got a couple of major "stutters" during the 1 hour show. Both around the time other recordings would have been finishing. 

I'll have to try different codecs I guess, but I've domne a LOT of testing to get the almost perfect smooth playback I now have and know different codecs won't give me the best result.

Cheers
Richard

Title: Re: MC almost unusable while thumbnails are built.
Post by: glynor on July 29, 2009, 12:07:41 pm
Are you suggesting that FFDSHOW is being used to thumbnail even though I'm using The Cyberlink Video Decoder... must go back and check the logs...?

Naah.  Made a bad assumption.  I've had problems in the past with the Cyberlink decoders (but that was long, long ago).  It is likely that they just aren't playing nice with multiple instances being loaded.  I'd be almost certain that they're MPEG-2 streams in there, assuming they are actually MPG containers and not misnamed.

Either way, there are many, many many filters that don't play well with multiple instances running simultaneously.  Yet one more reason that we need the background auto-thumbnailing routine to suspend while full-screen playback is active.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 29, 2009, 04:32:10 pm

Either way, there are many, many many filters that don't play well with multiple instances running simultaneously.  Yet one more reason that we need the background auto-thumbnailing routine to suspend while full-screen playback is active.

Even disabling while video is playing doesn't help general usability when running theater view, I can't use MC at all for about 7 minutes after it loads due to all the jworker threads taking over.

Maybe we could have two thumbnailing options  
1) Build Thubmnails + # of processes (So a dropdown list No, Yes - 1 process, Yes - 2 processes etc)
2) Suspend thumbnailing when playing fullscreen video


Matt - could/would this be possible please?

Richard

Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on July 31, 2009, 06:35:18 am
 It isn't only thumbnailing, other background processes are spawing multiple jworker.exe threads slowing the system down.  I assume it's probably auto-import as this also loads the video files to confirm what type they are I think.  But while these multiple jworker.exe threads are hogging 60 or 70% cpu navigating Theater view is basically impossible.


Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: MrHaugen on July 31, 2009, 01:20:07 pm
Just a note... I used to have all KINDS of problems with this when it was building video thumbnails on my system.  Even a quite powerful PC (Q9550 @ 3.6GHz with 4GB RAM) was brought to it's knees when MC would try to thumbnail a folder full of new 720p x264 MKVs.  The issue has been present for a long, long time with multiple versions of MC.  I pleaded for help on this repeatedly, asking that all thumbnailing activities be suspended at least while full-screen playback is active (to not interfere with smooth playback while you are obviously busy watching something in the foreground).

I've had the same problem with HD content as well. I think that even before I had any HD content (and the version was 13 and 12), the Full Screen playback would skip at times. I've messed up some times when moving movies into the right directories before I'm watching a movie. I'd vote for a "no thumb building when video watched". Would sort out some problems. I think my problems have ben less since I've started using CoreAVC my self though.
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on August 24, 2009, 07:45:18 am
Matt - Any thoughts further on this please?

Tonight while watching a TV recording twice my playback stuttered badly for about 5 seconds both times, and both times were when a background recording would have recently finished.

Both the recordings were SD TV recordings so no HD content issues here.

MC13 used to have very minor stutters but the MC14 ones are really bad.  I've tried different codecs/filters and  I haven't seen any real improvement.

Could we PLEASE have a way of limiting the number of JWORKER.exe threads or halt thumbnailing during full screen playaback???

Thanks
Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on August 26, 2009, 06:02:29 pm
Sorry to go on like a broken record, but this is still an issue for me......

To further try to narrow this down I have setup my two PCs as close to the same as I can.

I'm running the same Cyberlink video/audio decoder and VMR7 on both. (YES I have tried different decoders)

HTPC1 is an Intel quad core running at 2.66Mhz and 4GB RAM
HTPC2 is an old P4 3.2Ghz with 2GB RAM

Both machines are auto-importing from the same shared folder (T:\) and have build thumbnails turned ON.

Both machines open the library read ony, so the files to be autoimported are NOT in the library being opened.

There are 57 .mpg TV recordimngs in T: most are SD with a few being HD (but still .mpg files so no idea what they really are, but they are a LOT larger then the SD files!)

If I load MC on both machines here is what I find;



In general daily use, when playing back a video HTPC2 will have a VERY slight stutter (maybe a couple of frames) when a new file is imported, while HTPC1 will have a major stutter lasting a couple of seconds or more when it imports the same file during playback.

Would somebody else running a multi-core processor be able to test this for me please?
1) Setup a new folder for auto-import to monitor and ensure Build thumbails is enabled in the import options.
2) Close MC
3) Dump 50 .mpg files into the new folder
4) Launch MC and try to navigate around for a few minutes, you should find MC becomes totally unusable during this process (I hope)

Many thanks
Richard
Title: Re: MC almost unusable while thumbnails are built.
Post by: JimH on August 26, 2009, 06:27:41 pm
I'm running the same Cyberlink video/audio decoder and VMR7 on both. (YES I have tried different decoders)
I had a problem a few days ago (sorry I don't remember the details) that was solved by moving to MS filters.

Shared folder on a network drive?
Title: Re: MC almost unusable while thumbnails are built.
Post by: rpalmer68 on August 26, 2009, 10:07:00 pm
I had a problem a few days ago (sorry I don't remember the details) that was solved by moving to MS filters.

Shared folder on a network drive?

Was it the same symptoms or a different problem? 

T: is actually a local drive on HTPC1 shared so HTPC2 can access it.

Title: Re: MC almost unusable while thumbnails are built.
Post by: raldo on August 27, 2009, 01:29:21 am
Matt,

Adding a System.Thread.Sleep(10) (or whichever equivalent you're using) in the thumbnailing loop may improve the situation. I don't think this'll make thumbnailing significantly slower.

I've done this a few times without really knowing why I had to do it.

And yes I know, this kind of defies the whole point of thread priority, but if it works, it works...
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 08, 2009, 04:54:21 pm
This is STILL a big problem here.

Last night I was watching a movie with my wife, and duruing this time three TV recordings finished, so three times the movie stuttered for a few seconds and we had to rewind to hear what was said.

This is not happening anywhere near as badly on my older Pentium4 PC or for that matter with MC13 on my multi-processor system.  In fact I'd be confident in saying it started to stutter this badly when the mult-core changes were made for auto-import a while back.

Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 08, 2009, 05:10:07 pm
rpalmer,
Do you have cyberlink filters installed?

Jim
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 08, 2009, 06:03:23 pm
rpalmer,
Do you have cyberlink filters installed?

Jim


Jim, if you read my previous posts you'd know the answer to this.   

Yes, I use the cyberlink filters as they produce the best visual results on my system.  I have also pointed out that different filters do the ame thing.

Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 08, 2009, 06:20:39 pm
Jim, if you read my previous posts you'd know the answer to this.   
I don't read the entire thread each time I answer.  Sorry.

I have had stuttering problems for about a month.  It's about the same period of time that I've had some CyberLink software installed.  I installed it when I added a Blu-Ray player to my PC.  It may not be relevant.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 02:01:33 am
I have had stuttering problems for about a month.  It's about the same period of time that I've had some CyberLink software installed.  I installed it when I added a Blu-Ray player to my PC.  It may not be relevant.

It may be related to having the Cyberlink decoders installed, but I somehow doubt it.

I have done further testing with the latest build, and I can confirm that it makes no difference if I have the Cyberlink, Dscaler or ffdshow filters selected for .mpg playback the slowness while my recordings are being imported is exactly the same.

And I don't just mean stuttering during video playback, it can take several seconds to change a view in Theater View or even get a menu up in standard view... the CPU is basically maxing out during the import!

Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 06:35:11 am
It may be related to having the Cyberlink decoders installed, but I somehow doubt it.

I have done further testing with the latest build, and I can confirm that it makes no difference if I have the Cyberlink, Dscaler or ffdshow filters selected for .mpg playback the slowness while my recordings are being imported is exactly the same.

And I don't just mean stuttering during video playback, it can take several seconds to change a view in Theater View or even get a menu up in standard view... the CPU is basically maxing out during the import!
It may be our problem, but I don't see how.  Even for you, it works on other machines, right?

Look again at what else is running in the background.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 05:31:58 pm
It may be our problem, but I don't see how.  Even for you, it works on other machines, right?

Look again at what else is running in the background.

Nothing else is running in the background, it can't! JWorker.exe is taking over all the CPU cycles.  I can have up to four of them (jworker.exe) running all taking at least 20% CPU.

Yes correct it is all working fine on the other machine, but as pointed out the machine having problems is the ONLY machine I have with quad cores, the other machine is single core, and this problem only started when MC was changed to use multiple cores for auto import etc.  

There has always been a  slight stutter when auto-import/thumbnails were built previously and I agree with others posts that thumbnailing should be suspended during full screen video playback, but the stutter is minor compared to what is happening now.

So I still think there is an issue with the multi-core background processing somewhere. Sure it might be a directshow issue not folowing the priority setting rather than a direct MC issue, but MC needs to cope with this as we all have to run directshow so it's not going to just go away on it's own.

I have deatiled how to test this, so until somebody else can tell me they have done the same as I'm doing and it all works fine, I'm going to have to still believe it's a MC issue rather than a machine specific issue with my machine.


Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 05:34:06 pm
If it is a DirectShow filter causing the problem, I don't think there is anything that MC can do.

Without reading the entire thread again ... have you tried to see if you can narrow down the problem to a single filetype?
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 05:40:35 pm
If it is a DirectShow filter causing the problem, I don't think there is anything that MC can do.


Yes there is, go back to the way it USED to work without the multi-core support, it was MUCH better! )Sure auto-import is much fatser... but at what real cost in terms of the user experience?)

Or at least let us select how many cores MC will take over and disable thumbnailing during full screen video playback.

Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 05:42:27 pm
Yes there is, go back to the way it USED to work without the multi-core support, it was MUCH better!

Or at least let us select how many cores MC will take over!

Richard
Richard,
I really think you're barking up the wrong tree.   See my previous post (edited).

Jim
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 05:49:40 pm
Richard,
I really think you're barking up the wrong tree.   See my previous post (edited).

Jim


All my recordings are .mpg files and I can't change this, so it's a bit hard to test importing 30 files of a different file type.

Also remember that just browsing theater view or using the PC in general  is VERY SLOW while the import is happening so it's not just during video playback.

Also note that the SAME files using the SAME filters on the SAME machine under Mc13 DO NOT have the same problem, it's ONLY MC14 with builds after multi-core support for auto-import was added.

I can go and try to convert all my recordings to another format I guess and try that, but would really lile to hear Matt's opinions on all of this before I spend lots of time converting files.

Cheers
Richard

Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 05:55:49 pm
All my recordings are .mpg files and I can't change this, so it's a bit hard to test importing 30 files of a different file type.

Also remember that just browsing theater view or using the PC in general  is VERY SLOW while the import is happening so it's not just during video playback.

Also note that the SAME files using the SAME filters on the SAME machine under Mc13 DO NOT have the same problem, it's ONLY MC14 with builds after multi-core support for auto-import was added.

I can go and try to convert all my recordings to another format I guess and try that, but would really lile to hear Matt's opinions on all of this before I spend lots of time converting files.

Cheers
Richard


Try thumbnailing another type of video or try changing the filters for mpg.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 06:04:21 pm
Try thumbnailing another type of video or try changing the filters for mpg.

This is going nowhere, as I have already answered both of these in previous posts.

Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 06:14:04 pm
I'll let you work on this a while.   Everything you say points to problems with the filters.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 06:21:40 pm
I'll let you work on this a while.   Everything you say points to problems with the filters.

OK Jim you do that.

But before I spend more of my time testing and trying to debug your software for free, please explain the following;
1) why MC13 does NOT have the same problem with the SAME files, SAME filters on the SAME machine?  If it was a filter problem MC13 would also have this problem would it not?
2) Why do ALL filters I have tried, Dscaler, NVIDIA, FFDSHOW, Cyberlink ALL do the same thing?   IF ALL filters have the same problem, then  it's something MC is going to have to deal with a little better than it currently does.



Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 06:25:45 pm
Richard,
If I could explain what happens on computers, I would be drinking a beer and fishing on a beach in Australia (http://www.pix01.com/gallery/875E075F-D3DC-424C-AF33-226F3A9E423E/The_Fishing_Trip/) instead of trying to help you with this problem.

Jim
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 06:35:48 pm
Richard,
If I could explain what happens on computers, I would be drinking a beer and fishing on a beach in Australia instead of trying to help you with this problem.

Jim

Very true, but I don't know what else I can test/change that will show you it's not somethig I believe I can fix at this end, MC is starting too many threads during an import and it's bringing the machine to it's knees.  Sure it might be the way directshow filters work, but directshow isn;t going away ay time soon so MC is going to have to work with the issue somehow (like going back to not using multiple treads for importing).

It's like a car, it's not the car's fault people drive into trees and other cars, but they had to introduce seat belts to try to help minimise the "impact" of it happening.

I'll spend the day converting all my files to .avi files and testig with them, but I honestly don't think it will prove anything new.

Regards
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 09, 2009, 06:37:23 pm
avi is a container format and won't prove anything.  Find some files on the Internet to test.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 09, 2009, 06:42:59 pm
avi is a container format and won't prove anything.  Find some files on the Internet to test.

I have a lot of files from the internet, (TV series etc) but they are all .avi files, what format would you suggest then?
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Daydream on September 10, 2009, 12:12:24 am
Can we stay a second more with your MPG files? What's the source for them? DVD rips, broadcasts, home made, something else? What are the filters used when you play one of these files? I'm aiming not only for the Cyberlink Video decoder but also what is the splitter filter used. If you use a mixed combination certain filters may work together but not very well, resulting in unnecessary long time to parse the files and decode them properly.

I have tried to replicate your described behavior but couldn't really. Maybe there is a penalty when files are indexed/thumbnails are created but that's only if I lock an arrow key and force the program to scroll at max speed for no good use. It's light years away from the slowness you describe. Something's not OK. I have seen this behavior with MC struggling to create thumbnails for certain files (with CPU % shot to hell) but that was only when the DirectShow decoding chain wasn't the best. That's why I'd like to know what are all filters used on your side for MPG.

As a rule of thumb I'd use filters from the same source - all Microsoft, all Cyberlink (and unrelated to your problem I have zero confidence in these ones) or, my fav choice, the filters from MPC-Home Cinema (all of them, file source, splitters, decoders, etc). But that's just me. :)
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: raym on September 10, 2009, 03:22:27 am
Richard, I know we have similar setups but my htpcs are dual cores only. I too see what you describe but very rarely and no where near as severe as you describe.

As a work-around, you could disable auto import and script a job to import files only when mc is idle.  I have some ideas on this if you're interested.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Chili-Jam on September 10, 2009, 07:35:57 am
I have had one issue in Theaterview on my HTPC: browsing could get completely stuck and i had to wait for say 15 sec to change views.
Finally i found what caused this. There is the option to select your graphic card perfomance (can't look at MC at the moment so don't know the exact menu items).
I was able to test all versions of anti-aliasing BUT as soon as i switched the "sync-option" on
navigation got stuck.

I understand it may have nothing to do with your problems but then it's only a small click to switch synch framerate off and maybe it helps.

Good luck.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 07:48:12 am
Can we stay a second more with your MPG files? What's the source for them? DVD rips, broadcasts, home made, something else? What are the filters used when you play one of these files? I'm aiming not only for the Cyberlink Video decoder but also what is the splitter filter used. If you use a mixed combination certain filters may work together but not very well, resulting in unnecessary long time to parse the files and decode them properly.

I have tried to replicate your described behavior but couldn't really. Maybe there is a penalty when files are indexed/thumbnails are created but that's only if I lock an arrow key and force the program to scroll at max speed for no good use. It's light years away from the slowness you describe. Something's not OK. I have seen this behavior with MC struggling to create thumbnails for certain files (with CPU % shot to hell) but that was only when the DirectShow decoding chain wasn't the best. That's why I'd like to know what are all filters used on your side for MPG.

As a rule of thumb I'd use filters from the same source - all Microsoft, all Cyberlink (and unrelated to your problem I have zero confidence in these ones) or, my fav choice, the filters from MPC-Home Cinema (all of them, file source, splitters, decoders, etc). But that's just me. :)

I've not done anythig with the filesource or splitter as I really wouldn't know what to use, I'm using the cyberlink filter and VMR7 renderer as this gives the smootest playabck for me.  Having said that I'm not fussed what I end up using as long as the results are good.

The files are recordings from DVB-T broadcasts in either SD (the majority) or HD (although still .mpg files so no REAL HD I guess).

Here's the graph playing back a file,

Code: [Select]
Filter Graph Info:

    Filter 'Video Renderer'
        CLSID: {B87BEB7B-8D29-423F-AE4D-6582C10175AC}
        Host: C:\WINDOWS\system32\quartz.dll
        Input Pin 'VMR Input0'
            Connected to pin 'Video Out' of filter 'CyberLink Video/SP Decoder (PDVD7)'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {1B81BE0C-A0C7-11D3-B984-00C04F2E73C5}, Format type FORMAT_VideoInfo2
        Input Pin 'VMR Input1'

    Filter 'ReClock Audio Renderer'
        CLSID: {9DC15360-914C-46B8-B9DF-BFE67FD36C6A}
        Host: C:\Program Files\ReClock\ReClock.dll
        Input Pin 'In'
            Connected to pin 'Out' of filter 'AC3Filter'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

    Filter 'CyberLink Video/SP Decoder (PDVD7)'
        CLSID: {8ACD52ED-9C2D-4008-9129-DCE955D86065}
        Host: C:\Program Files\CyberLink\PowerDVD\VideoFilter\CLVsd.ax
        Input Pin 'Video In'
            Connected to pin 'Video' of filter 'MPEG-2 Demultiplexer'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_MPEG2_VIDEO, Format type FORMAT_MPEG2_VIDEO
        Output Pin 'Video Out'
            Connected to pin 'VMR Input0' of filter 'Video Renderer'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {1B81BE0C-A0C7-11D3-B984-00C04F2E73C5}, Format type FORMAT_VideoInfo2
        Input Pin 'SubPicture In'
        Output Pin '~Closed Caption Out'

    Filter 'AC3Filter'
        CLSID: {A753A1EC-973E-4718-AF8E-A3F554D45C44}
        Host: C:\Program Files\AC3Filter\ac3filter.ax
        Input Pin 'In'
            Connected to pin 'Mpeg-1' of filter 'MPEG-2 Demultiplexer'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_MPEG2_AUDIO, Format type FORMAT_WaveFormatEx
        Output Pin 'Out'
            Connected to pin 'In' of filter 'ReClock Audio Renderer'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

    Filter 'MPEG-2 Demultiplexer'
        CLSID: {AFB6C280-2C41-11D3-8A60-0000F81E0E4A}
        Host: C:\WINDOWS\system32\mpg2splt.ax
        Input Pin 'MPEG-2 Stream'
            Connected to pin 'Output' of filter 'T:\general\(2009-08-27 19-55) Catalyst ABC1.mpg'
            Major type MEDIATYPE_Stream  Sub type MEDIASUBTYPE_MPEG2_PROGRAM, Format type TIME_FORMAT_NONE
        Output Pin 'Mpeg-1'
            Connected to pin 'In' of filter 'AC3Filter'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_MPEG2_AUDIO, Format type FORMAT_WaveFormatEx
        Output Pin 'Video'
            Connected to pin 'Video In' of filter 'CyberLink Video/SP Decoder (PDVD7)'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_MPEG2_VIDEO, Format type FORMAT_MPEG2_VIDEO

    Filter 'T:\general\(2009-08-27 19-55) Catalyst ABC1.mpg'
        CLSID: {E436EBB5-524F-11CE-9F53-0020AF0BA770}
        Host: C:\WINDOWS\system32\quartz.dll
        Output Pin 'Output'
            Connected to pin 'MPEG-2 Stream' of filter 'MPEG-2 Demultiplexer'
            Major type MEDIATYPE_Stream  Sub type MEDIASUBTYPE_MPEG2_PROGRAM, Format type TIME_FORMAT_NONE

I'm all ears on any suggestions!

Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 08:05:40 am
Richard, I know we have similar setups but my htpcs are dual cores only. I too see what you describe but very rarely and no where near as severe as you describe.

As a work-around, you could disable auto import and script a job to import files only when mc is idle.  I have some ideas on this if you're interested.

 I could do that, but would prefer not to unless I can't find another solution.

Can you remind me what filters/splitters etc you're running for your webscheduler recording playback?  Oh but you're using dvr-ms files still arent you?   For some reason I cant use them because they will only play back on the PC they are recorded on, they just stutter terribly if I try to play then on any other PC... never have worked out why!

Sigh... audio playback was so simple :)

Cheers
Richard


Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Matt on September 10, 2009, 09:45:35 am
I could do that, but would prefer not to unless I can't find another solution.

Can you remind me what filters/splitters etc you're running for your webscheduler recording playback?  Oh but you're using dvr-ms files still arent you?   For some reason I cant use them because they will only play back on the PC they are recorded on, they just stutter terribly if I try to play then on any other PC... never have worked out why!

Sigh... audio playback was so simple :)

Cheers
Richard




We run JRWorker.exe, which analyzes or thumbnails videos, as a separate program with the lowest process priority.  The system should never give it cycles when playback, which is a higher priority, needs the cycles.

However, it seems like there's still some resource that JRWorker and playback conflict over.  Perhaps it's GPU related instead of CPU related, or physical memory related?

We see a couple options:
1) Better make JRWorker.exe work at a very low priority -- but I'm not sure if there's anymore we can do
OR
2) Disable auto-import work during Display View playback

Thoughts?
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Matt on September 10, 2009, 09:52:23 am
Just another little note.

Version 14 of Media Center imports two files at a time instead of one.  This means two JRWorker.exe processes if two videos are importing.

Thumbnailing uses a more advanced quota system to decide how many concurrent processes to spin up.  It normally works out to two thumbnails at a time for video.  However, auto-import will only ever do one at a time since it's designed to be a background task.

I wonder if the amount of parallelism in the import engine should scale based on whether it's a user-requested (foreground) or auto-import (background) task?  Of course, if thread and process priorities work, you shouldn't really have to be afraid of extra threads.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: glynor on September 10, 2009, 12:54:44 pm
2) Disable auto-import work during Display View playback

Thoughts?

Yes.  On both really, if #1 is possible, but mostly #2.  And Thumbnailing too.

In my case, I don't think it really is so much the Auto-Import that is slowing things down as the Thumbnailing.  I suspect the problem is that the Filters used to decode the videos don't always respect the thread priority of JRWorker.exe, or at least don't play nicely with multiple instances of each other running, particularly in "startup" (or perhaps it is simple disk throughput or something).

One easy to replicate example I can think of is this:

1. Set Media Player Classic to allow multiple instances started from file.  View --> Options.  Then Choose Player in the tree on the left, and change Open options to "Open a new player for each media file played".
2. Open a high-quality file in MPC.  Preferably one that uses Haali and FFDSHOW to decode.  Something like a x264 MKV would be perfect.  Let it play for a few seconds.
3. Then open a few more other (similar) files in second, third, and fourth copies of MPC.

If you do this, at least on all of my systems, the video playback in the first copy of MPC will "pause" for a second or two while the new instance is loading.  Once they are loaded, then everything "springs" back to life and they can play concurrently.  This happens to me even if I'm loading the multiple files off of different physical disks, though it does seem to be slightly worse if they are all on the same disk (which makes some sense as the drive head seeks to find the new spot).

I think this is a bit of a microcosm of what we are seeing in MC when it is launching multiple thumbnail worker threads.  Once they are launched, they could "play" fine.  The problem is that they don't really "play" they just render a frame and then quit and launch a new thread, and I think it is sending the Windows scheduler into a tizzy.  Even if they all have low priority, this constant loading and unloading of filter resources (not to mention disk head seeking) is bound to draw some substantial resources.

That's all fine, so long as you aren't doing something that has zero tolerance for latency, like full-screen video playback does.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Daydream on September 10, 2009, 01:32:44 pm
One easy to replicate example I can think of is this:

1. Set Media Player Classic to allow multiple instances started from file.  View --> Options.  Then Choose Player in the tree on the left, and change Open options to "Open a new player for each media file played".
2. Open a high-quality file in MPC.  Preferably one that uses Haali and FFDSHOW to decode.  Something like a x264 MKV would be perfect.  Let it play for a few seconds.
3. Then open a few more other (similar) files in second, third, and fourth copies of MPC.


May I point that the OP talked about MPG files. With MPG files I can run a test like above (of course without Haali and ffdshow) and nothing fails, stutters or stumble (in MPC). The first file keeps on playing fluently as the rest open and play. Overall this discussion may prove that parallelism is somewhat taxing but that may become noticeable just because a not-optimal DirectShow decoder chain is being used, so a certain lag gets multiplied.

I mean looking at the above filter graph, the splitter is from MS, the audio decoder/renderer is a combination of AC3 Filter and Reclock, the video decoder is from Cyberlink... I'm not saying that it shouldn't work but would it work fast, in parallel? Hmmm...
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 04:10:52 pm
May I point that the OP talked about MPG files. With MPG files I can run a test like above (of course without Haali and ffdshow) and nothing fails, stutters or stumble (in MPC). The first file keeps on playing fluently as the rest open and play. Overall this discussion may prove that parallelism is somewhat taxing but that may become noticeable just because a not-optimal DirectShow decoder chain is being used, so a certain lag gets multiplied.

I mean looking at the above filter graph, the splitter is from MS, the audio decoder/renderer is a combination of AC3 Filter and Reclock, the video decoder is from Cyberlink... I'm not saying that it shouldn't work but would it work fast, in parallel? Hmmm...

So can you suggest a DirestShow chain I can test/try as an alternative that you think might render better performance results?

R
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 04:54:53 pm
Thanks everybody, some interesting thoughts/points being raised here.

I will just mention a couple of things.


1) Initial Auto-import on MC Load
When I first launch MC14 and start browsing in theater view it can take several seconds for MC to action my key presses.  So I can press UP and then wait 3 or 4 seconds until the cursor actually moves, then it will take a few more seconds until the next key press is actioned.  This goes on for several (6 or more) minutes (importing 50 files or so).
During this time there are usually 4 JWorker.exe threads running, and they float between 5 and 23% utilization each.
This is obviously while auto-import is importing the 50 or so .mpg TV recordings I have on the local D: drive.
I can improve things by disabling the thumbnailing during the import, then things import and perform a lot better, BUT when browsing my views in theater View I get very stuttery movement while MC builds the thumbnails for each view I go into.

2) Stuttering during plaback
I'll get a couple of seconds of BAD stutter when a new file (.mpg) is imported in the background during video playback.  Audio/video will get out of sync for several seconds after this and then it all settles down and plays normally again.
It doesn't matter if I'm using the same or different filters for the playback.  For example, the graph below can be in use for video playkack and when a .mlg file is imported (using my previous graph posted) I'll still get the big stutter.
Code: [Select]
Filter Graph Info:

    Filter 'Video Renderer'
        CLSID: {B87BEB7B-8D29-423F-AE4D-6582C10175AC}
        Host: C:\WINDOWS\system32\quartz.dll
        Input Pin 'VMR Input0'
            Connected to pin 'Out' of filter 'ffdshow Video Decoder'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_YUY2, Format type FORMAT_VideoInfo2
        Input Pin 'VMR Input1'

    Filter 'ReClock Audio Renderer'
        CLSID: {9DC15360-914C-46B8-B9DF-BFE67FD36C6A}
        Host: C:\Program Files\ReClock\ReClock.dll
        Input Pin 'In'
            Connected to pin 'Out' of filter 'ffdshow Audio Decoder'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

    Filter 'ffdshow Video Decoder'
        CLSID: {04FE9017-F873-410E-871E-AB91661A4EF7}
        Host: C:\Program Files\Combined Community Codec Pack\Filters\FFDShow\ffdshow.ax
        Input Pin 'In'
            Connected to pin 'Stream 00' of filter 'AVI Splitter'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {44495658-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'Out'
            Connected to pin 'VMR Input0' of filter 'Video Renderer'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_YUY2, Format type FORMAT_VideoInfo2
        Input Pin 'In Text'

    Filter 'ffdshow Audio Decoder'
        CLSID: {0F40E1E5-4F79-4988-B1A9-CC98794E6B55}
        Host: C:\Program Files\Combined Community Codec Pack\Filters\FFDShow\ffdshow.ax
        Output Pin 'Out'
            Connected to pin 'In' of filter 'ReClock Audio Renderer'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx
        Input Pin 'In'
            Connected to pin 'Stream 01' of filter 'AVI Splitter'
            Major type MEDIATYPE_Audio  Sub type Unknown GUID Name: {00000055-0000-0010-8000-00AA00389B71}, Format type FORMAT_WaveFormatEx

    Filter 'AVI Splitter'
        CLSID: {1B544C20-FD0B-11CE-8C63-00AA0044B51E}
        Host: C:\WINDOWS\system32\quartz.dll
        Output Pin 'Stream 00'
            Connected to pin 'In' of filter 'ffdshow Video Decoder'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {44495658-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'Stream 01'
            Connected to pin 'In' of filter 'ffdshow Audio Decoder'
            Major type MEDIATYPE_Audio  Sub type Unknown GUID Name: {00000055-0000-0010-8000-00AA00389B71}, Format type FORMAT_WaveFormatEx
        Input Pin 'input pin'
            Connected to pin 'Output' of filter 'X:\Video\TV Series\Frasier\Season 2\S02E01 - Slow Tango in South Seattle.avi'
            Major type MEDIATYPE_Stream  Sub type MEDIASUBTYPE_Avi, Format type TIME_FORMAT_NONE

    Filter 'X:\Video\TV Series\Frasier\Season 2\S02E01 - Slow Tango in South Seattle.avi'
        CLSID: {E436EBB5-524F-11CE-9F53-0020AF0BA770}
        Host: C:\WINDOWS\system32\quartz.dll
        Output Pin 'Output'
            Connected to pin 'input pin' of filter 'AVI Splitter'
            Major type MEDIATYPE_Stream  Sub type MEDIASUBTYPE_Avi, Format type TIME_FORMAT_NONE


So stopping background importing etc during playback would solve #2, but it obviously won't resolve #1 which basically makes MC unusable for 5 or 6 minutes after its initially loaded, and often when people are wanting to use it.

Maybe Glynor is right, it may be a windows scheduling/queuing issue (along with filters etc I'm sure), I really don't know, or maybe MC does need to limit the number of threads back to just one file being imported at a time... not sure.


Matt, when MC imports does it build just the video side of the graph to determing  details and create thumbnails or does it build the audio side too?  Would this make any difference?  Just thinking it would at least remove the audio filters etc out of the equation...


Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Alex B on September 10, 2009, 05:00:31 pm
Quote
   Filter 'ReClock Audio Renderer'
        CLSID: {9DC15360-914C-46B8-B9DF-BFE67FD36C6A}
        Host: C:\Program Files\ReClock\ReClock.dll
        Input Pin 'In'
            Connected to pin 'Out' of filter 'AC3Filter'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

ReClock might well be causing the problem (or making it more severe) if it really gets loaded always when JRWorker accesses a video file. ReClock always analyses the file format when playback starts and depending on the result it can change the frame rate and start audio resampler. This causes additional resource usage and delay.

ReClock is a nice filter for playback (essential for some users), but it is useless for thumbnailing. You can set JRWorker as "never load" in ReClock's options if you have not already done that.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 05:23:16 pm
ReClock is a nice filter for playback (essential for some users), but it is useless for thumbnailing. You can set JRWorker as "never load" in ReClock's options if you have not already done that.

Thanks Alex,
I find reclock invaluable over here in PAL land as I'm always having to deal with different video frame rates when watching non-PAL sources and I use it to switch my monitor refresh rate to match the video source as well.

I have set Jworker.exe to "Never Load" in the reclock options, but if JWorker is loading it itself then maybe you're right... hence the question on if the audio part of the graph is or needs to be loaded for impoirt/thumbnailing.

I'll try taking reclock out sometime today to see if it makes any difference...
 
Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Daydream on September 10, 2009, 05:52:32 pm
So can you suggest a DirestShow chain I can test/try as an alternative that you think might render better performance results?

R

If you're willing let's try this:
- get the MPC-HC standalone filters from here (http://sourceforge.net/projects/mpc-hc/files/) (MPC-Standalone Filters.1.3.1249.0.(x86).zip) - keeping it simple, 32bit versions
- open the zip somewhere and register (only): mpadecfilter.ax, mpcvideodec.ax, mpeg2decfilter.ax, mpegsplitter.ax (registering is done with regsvr32 x:\[path to filter]\[filter file] from Run..., from a commandline window, etc)
- set up MC to use only  them since you already have other filters. That would be the MS source filter (sorry can't recall with what name exactly is coming up, it's the quartz.dll file), OpenSource MPEG splitter as the splitter filter, Gabest Audio decoder as the audio decoder, OpenSource MPEG2Dec Filter for playing MPEG, MPEG-2 and whatever else MC lists in that category. No Reclock, nothing else. JRiver bitrate monitor can stay, that's fine.
- Make sure the files play and there are no x filters to do the same job, daisy chained (i.e. if you see Gabest Audio decoder AND Ac3Filter showing up together it's not good, go in Ac3Filter and disable its use for everything from its own control panel, System tab). We wanna make sure the playback it's working with minimum filters. Will check for subtleties later.
- Index some more stuff with make thumbnails on. See if there is any difference.

If there is a difference than the decoding chain is the problem (or possible something else we haven't accounted for). If there is no difference I guess we can concentrate our attention to parallelism, although personally I don't see anything wrong with it (but my indexing and thumb-nailing technique might differ significantly from other users).
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 10, 2009, 07:23:12 pm
OK - I tested as well - I normally have Auto Import Off, so I turned it on and within 1 Minute got a "MC has Stopped Working" error in Win7-64 Bit Ultimate on an i7 Box (see attached log).  I turned on and off the Auto Import, added and removed items in a test folder and saw big halts in MC with Task Mgr saying "Not Responding" for up to 10-sec at a time but the more tested the less frequent these periods became.  It seems to me to be more related to the JRAnalyser than JRWorker (if they are different that is!) and I especially notice this when going into "Drives and Devices" - something is being loaded / checked that halts all other things.  In my final test of watching a DVB-T CH, Auto Importing a bunch of MPG files, navigating around MC, watching Thumbs build in the Recently Imported list...it all worked perfectly.


Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 10, 2009, 07:50:06 pm
Don't you both live in Australia?  And raym?
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 08:44:53 pm
Don't you both live in Australia?  And raym?

Correct.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 10, 2009, 09:27:16 pm
If you're willing let's try this:
- get the MPC-HC standalone filters from here (http://sourceforge.net/projects/mpc-hc/files/) (MPC-Standalone Filters.1.3.1249.0.(x86).zip) - keeping it simple, 32bit versions
- open the zip somewhere and register (only): mpadecfilter.ax, mpcvideodec.ax, mpeg2decfilter.ax, mpegsplitter.ax (registering is done with regsvr32 x:\[path to filter]\[filter file] from Run..., from a commandline window, etc)
- set up MC to use only  them since you already have other filters. That would be the MS source filter (sorry can't recall with what name exactly is coming up, it's the quartz.dll file), OpenSource MPEG splitter as the splitter filter, Gabest Audio decoder as the audio decoder, OpenSource MPEG2Dec Filter for playing MPEG, MPEG-2 and whatever else MC lists in that category. No Reclock, nothing else. JRiver bitrate monitor can stay, that's fine.
- Make sure the files play and there are no x filters to do the same job, daisy chained (i.e. if you see Gabest Audio decoder AND Ac3Filter showing up together it's not good, go in Ac3Filter and disable its use for everything from its own control panel, System tab). We wanna make sure the playback it's working with minimum filters. Will check for subtleties later.
- Index some more stuff with make thumbnails on. See if there is any difference.

If there is a difference than the decoding chain is the problem (or possible something else we haven't accounted for). If there is no difference I guess we can concentrate our attention to parallelism, although personally I don't see anything wrong with it (but my indexing and thumb-nailing technique might differ significantly from other users).

Have tried to follow your directions, registered the filters OK but am not sure if I have the correct ones selected for mpg playback in MC as the names are a little confusing.

When I try to playback a recorded mpeg file MC crashes with an application error, in module quartz.dll, so something isn't right!

When you get a change can you tell me the correct names to have ticked/selected please?
What should I have as the "Source Filter" and what should be ticked under Other Filters.  Also which renderer should I use on XP?

Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Daydream on September 11, 2009, 01:19:00 am

When you get a change can you tell me the correct names to have ticked/selected please?

Here you go:

(http://thumbnails7.imagebam.com/4848/79176648478361.gif) (http://www.imagebam.com/image/79176648478361/) (http://thumbnails10.imagebam.com/4848/225da748478368.gif) (http://www.imagebam.com/image/225da748478368/)

Made a change in the suggestions, use the source filter from MPC also (no MS-stuff quartz.dll; mpegsplitter.ax does both functions, source and splitter). For the renderer EVR if you have .Net Framework 3.0 installed or VMR9 if you don't. See in the second pic how the filter list should look when you play an MPG file.

Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 04:28:51 am
Here you go:


Made a change in the suggestions, use the source filter from MPC also (no MS-stuff quartz.dll; mpegsplitter.ax does both functions, source and splitter). For the renderer EVR if you have .Net Framework 3.0 installed or VMR9 if you don't. See in the second pic how the filter list should look when you play an MPG file.



Thanks!

Have setup mpg playback exactly the same, except EVR gave me a black screen with audio, VMR7 the same and then crashed, VMR9 seems to play back OK.  Strange as I do have .NEt 3.0 and 3.5 installed, anyway the test is fine as long as one renderer works at this stage.

So I have played back some files OK so have then proceeded to do a test.

I closed MC and opened it again, this will cause all my mpg files to be impoirted again by auto-import.

Navigating in Theater view is still VERY sluggish while the import is happening, with 4 jworker.exex processes running and CPU between 20 and 75%.
What is very interesting is that once I manage to navigate to an mpg file and play it, CPU drops back to 8% and the jworker.exe processes vanish.
As soon as I stop playback cpu goes back up and JWorker.exes come back and the PC gets sluggish again.

This doesn't happen with my original filters (even without reclock), the jworker.exe processes keep running during playback..... it's like everything stops for playback with the MPC filters...

Anyway, not sure what this tells us, and I'll have to do some recordings while watching somethig to see if I still get stuttering during playback  but it's interesting....


[EDIT] As an experiment I tried to play back an avi file (currently using ffdshow filters) while the import was running (with MPC filters still enabled).  Video playback was all stuttery and unwatchable, CPU stayed high and JWorker.exe processes were still running.

Will report back when I test some more...

Cheers
Richard


Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 07:24:19 am
I just had two big stutters while watching an .mpg recorded file and two recordings that recently finished were imported, so we still don't have a solution....


Oh dear, I've just noticed that the source filter (MPC_Mpeg Source) isn't loading and the graph is still using "File Source (Async)" and the Mpeg_2 Demultiplexer.  Looking in the logs the MPC one isnt even being tried, quartz.dll is loading first....  Hmm.  The filter is registered (just did it again to be safe) and it is selected in MC... any ideas?

The other MPC filters are loading ok.

log file attached.

Richard.

Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: glynor on September 11, 2009, 10:22:26 am
Hmmm.... I'm definitely not seeing problems that severe.  While I see issues, it is generally to a far smaller degree (and, like I said, has been dramatically better since I went with CoreAVC).  Generally my issues are isolated to high def videos with x264 encoding and high-ish bitrates.

I suspect that Jim might be right.  You sound like you have some filter issues.  Have you worked through these tips?

http://www.cccp-project.net/wiki/index.php?title=Troubleshooting_Guide

Before going in and playing with a bunch of individually applied and installed filters, I'd get the system in a "known good" configuration, to eliminate other problems as a possibility.  If you get your filters all set up through CCCP, with no "tweaks" or "individually installed" filters, then you'll have a good starting point.  Rip the "guts" of your DirectShow playback filters out, and start over fresh (including removing anything like Nero filters, PowerDVD filters, and all the rest).  If that solves things then you know it was something you did manually or some software you added.

If not, then we need to look elsewhere...

I'd also be interested to see a graph of your results in HD Tach for both your system drive and your media drive, assuming they're not the same drive.  I'm wondering if you aren't having some sort of Hard Drive throughput issue.  Try grabbing HD Tach and run the Long Test on your drives: http://www.simplisoftware.com/Public/index.php?request=HdTach

This is an Intel Q6600 (or something like that) machine, right?  On a P45 chipset, if I'm not mistaken.  Perhaps a un/re-install of your Chipset Drivers and Intel Storage drivers might make a difference.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Daydream on September 11, 2009, 10:53:56 am
Richard, it looks like there is problem with the merit of certain filters on your system. For whatever reason some (unwanted) filters are kicking in even if you chose otherwise. There are filter managers that can change the merits but that's a rather annoying/advanced way of dealing with the problems.

If you install CCCP (like Glynor suggests, and most of the people over here swear by that package) that may set the merits right. One note though: if you just click away thought the installing steps of CCCP package and don't change anything you might end up with to much stuff (including audio decoding) assigned to ffdshow which is something I wouldn't chose.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: glynor on September 11, 2009, 11:00:42 am
If you install CCCP (like Glynor suggests, and most of the people over here swear by that package) that may set the merits right. One note though: if you just click away thought the installing steps of CCCP package and don't change anything you might end up with to much stuff (including audio decoding) assigned to ffdshow which is something I wouldn't chose.

I'm not saying keep it like that forever.  I'm just saying get there to eliminate other things as potential issues.  Then step through new installs and tweaks (for quality) ONE AT A TIME, testing all the way.

Systematic, my friend.  Systematic.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 03:31:29 pm
Thanks gents, I do have CCCP installed and have re-installed it only a couple of days ago.

What software should I use to try to change the merit of things to force the MCP source filter to be used?

Seems strange, I would have thought that if I specify it in the MC settings, MC would at leat try to use it, or am I missreading the log files?

R
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 03:36:40 pm
Richard, what prog are you using to record with and is it on the same PC and the one you are using for playback?
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 03:45:04 pm
Thanks gents, I do have CCCP installed and have re-installed it only a couple of days ago.

What software should I use to try to change the merit of things to force the MCP source filter to be used?

Seems strange, I would have thought that if I specify it in the MC settings, MC would at leat try to use it, or am I missreading the log files?

R

I used to use http://www.videohelp.com/tools/RadLight_Filter_Manager to muck around with changing merits in XP but MC should do it for you.  I hate to say it but I've really had a Much Better time with DirectShow when I moved to Vista (and now to Win7).....do you have the ability to try a Win7 (I think you can still get a "free" Win7-RC key from MS...)
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 04:03:29 pm
Richard, what prog are you using to record with and is it on the same PC and the one you are using for playback?

I'm using Webscheduler and yes it is on the same PC, although it hasn't been a problem until recently, and running auto-import without webscheduler recording still slows the PC right down.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 04:06:43 pm
Richard - I've have a look at some of your logs and just another idea (if you have not already tested) is to take the Network out of the equation - have you tested to see if you get the stutter when playback / thumnailing / library etc is all off the local C: Drive isntead of over the network (I had stutter issues that I thought was bandwidth related but it was caused by a cheap switch I'd recently added (http://yabb.jriver.com/interact/index.php?topic=53728.0 )).

Thanks
Nathan
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 11, 2009, 04:10:34 pm
A disk in PIO mode could also cause problems. 

Here's what Microsoft says:
Quote
For repeated DMA errors. Windows XP will turn off DMA mode for a device after encountering certain errors during data transfer operations. If more that six DMA transfer timeouts occur, Windows will turn off DMA and use only PIO mode on that device.

http://www.microsoft.com/whdc/device/storage/IDE-DMA.mspx

This was one of the worst ideas ever devised by Microsoft.  It caused me a lot of trouble a few years ago.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 04:12:54 pm
Richard - I've have a look at some of your logs and just another idea (if you have not already tested) is to take the Network out of the equation - have you tested to see if you get the stutter when playback / thumnailing / library etc is all off the local C: Drive isntead of over the network (I had stutter issues that I thought was bandwidth related but it was caused by a cheap switch I'd recently added (http://yabb.jriver.com/interact/index.php?topic=53728.0 )).

Thanks
Nathan

My T: drive is in fact my local D: drive just mapped to T: so all machines use the same drive letter stored in the library.

Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 04:37:59 pm
I have changed the merit of the Filesource (async) filter to Do NOT USE, and increased the MPC filter to a higher merit yet the filesource filter is still being loaded in MC instead of the one specified in the MC settings.


R
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 04:50:56 pm
*sigh* why can't things just work as they should!

It's the "fun" of HTPC (makes you want to just buy a HW player some times.....)

I just run another test of AutoImporting a bunch of MPG files with Resource Monitor going (see below) and.... it played nice again.  I normally get 3 or 4 JRWorker threads going on my i7 Win7-64 PC and while they seem to take a long time (eg up to 10sec for some files ...which is probably also filter related to build a thumb) none of these threads ever exceed around 5% of CPU each.  My Disc IO / Memory etc was also fine.  If there is anything else you want me to check....
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Alex B on September 11, 2009, 04:51:47 pm
My T: drive is in fact my local D: drive just mapped to T: so all machines use the same drive letter stored in the library.

As a test, you could disconnect the mapped T: drive and use the SUBST command instead.

For instance, if the shared folder that you have mapped is D:\Media run a command prompt and write

SUBST T: D:\Media

If that helps anything you can create a .bat file that contains the same command line and put it in the startup folder.

I have run some tests and found that a locally mapped "network" drive connection is slower than SUBST (which does not differ from standard disk access), though the difference is quite small.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 04:56:03 pm
Richard - do you get the problems if you uncheck Tools --> Options --> Tree & View --> Thumbnails --> Create thumbnails for videos ?

This should stop thumbs from being built during Auto Import (but you would then have to Build missing thumbnails manually at some other time).
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: jmone on September 11, 2009, 04:58:02 pm
As a test, you could disconnect the mapped T: drive and use the SUBST command instead.

For instance, if the shared folder that you have mapped is D:\Media run a command prompt and write

SUBST T: D:\Media

If that helps anything you can create a .bat file that contains the same command line and put it in the startup folder.

I have run some tests and found that a locally mapped "network" drive connection is slower than SUBST (which does not differ from standard disk access), though the difference is quite small.

Or...just change the Drive Letter (in Disk Manger) to be T:
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: glynor on September 11, 2009, 05:06:32 pm
Or...just change the Drive Letter (in Disk Manger) to be T:

That's the real answer, unless you need it to be both T and D for some wacky reason.

Do the disk throughput test.  This all seems like a filter problem, but it could be a disk throughput issue too.  Running HD Tach will only take a few minutes, and it will answer the question.

And, I didn't mean just install CCCP.  I meant work through the CCCP troubleshooter to ensure that you have ONLY CCCP on there, and then systematically add anything additional that you need.  The troubleshooter has a bunch of important steps and the Insurgent app will help you identify issues and conflicting filters.

I have changed the merit of the Filesource (async) filter to Do NOT USE, and increased the MPC filter to a higher merit yet the filesource filter is still being loaded in MC instead of the one specified in the MC settings.

Even if you set it to DO NOT USE, it will still fail back to it if the render graph fails at the higher Merit scores.  Whatever you are choosing as your default is NOT WORKING.  It quite likely is NOT that the render graph is "defaulting" to the Filesource filter.

Test the render graph manually using G-Spot or GraphEdit.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Yaobing on September 11, 2009, 05:09:46 pm
Sorry I have not said anything during last few days.  I now have to do some catching up reading these posts.

Regarding audio filters during thumbnailing, we build the graph initially letting DirectShow render all streams, including audio, then we remove the audio branch.  Therefore audio filters should not be a problem, at least not during actually process of grabbing an image.  But if an audio filter gives us trouble even during the process of building the DirectShow graph, we have to re-consider how we do it.

Matt has made some changes which will appear in a future build.  I hope they will alleviate the problem:

8. Changed: Adding files to the library will run one-at-a-time during a background import, and only two-at-a-time during a manual import.
9. Optimized: Auto-import builds thumbnails two-at-a-time when run manually.
10. Changed: Auto-import will no longer respond to external file system changes while playing in Display View, but instead wait to reconcile changes until exiting Display View.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 11, 2009, 05:22:31 pm

8. Changed: Adding files to the library will run one-at-a-time during a background import, and only two-at-a-time during a manual import.
9. Optimized: Auto-import builds thumbnails two-at-a-time when run manually.
10. Changed: Auto-import will no longer respond to external file system changes while playing in Display View, but instead wait to reconcile changes until exiting Display View.

For the sake of clarity, I think these may help or even eliminate the symptoms, but I also believe there is an underlying problem with the hardware/software combination rpalmer is using.

It would be nice to get to the bottom of the problem.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 05:28:15 pm
For the sake of clarity, I think these may help or even eliminate the symptoms, but I also believe there is an underlying problem with the hardware/software combination rpalmer is using.

It would be nice to get to the bottom of the problem.

Thanks Yaobing, Matt and Jim

I'll await these changes and report back on how things improve...

In the meantime I'll try to debug my filters, as yes I agree Jim I'd like to get to the bottom of it too :)

Cheers
Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 11, 2009, 05:32:41 pm
Thanks Yaobing, Matt and Jim

I'll await these changes and report back on how things improve...

In the meantime I'll try to debug my filters, as yes I agree Jim I'd like to get to the bottom of it too :)
Please don't forget the drive possibilties above.  PIO mode could explain everything.  Maybe you know that and have elimitated it as a possible cause.  glynor's drive test would probably catch that and other problems.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 05:39:49 pm
Please don't forget the drive possibilties above.  PIO mode could explain everything.  Maybe you know that and have elimitated it as a possible cause.  glynor's drive test would probably catch that and other problems.

The drive is an SATA drive not an IDE but yes I will run the HD test to check.

Cheers
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: JimH on September 11, 2009, 05:49:53 pm
I believe SATA drives can also have the PIO problem.  It's Windows deciding to whack the drive for being bad.
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: rpalmer68 on September 11, 2009, 05:52:31 pm
I believe SATA drives can also have the PIO problem.  It's Windows deciding to whack the drive for being bad.

Here's the results from the HD Tach.

Will take a lot longer to troubleshoot the filters as I have the in-laws here this week... so better not spend the whole week under the bonnet so to speak :)

Richard
Title: Re: MC almost unusable during auto-import and thumbnails are built.
Post by: Alex B on September 11, 2009, 06:38:57 pm
Or...just change the Drive Letter (in Disk Manger) to be T:

That's the real answer, unless you need it to be both T and D for some wacky reason.

The SUBST command is fine to use. I have never had problems with it.

A virtual drive letter makes possible to access a shared folder from any PC on the LAN through the same path.

When a folder path is SUBSTituted locally on the PC that holds the media files file access doesn't happen through the network subsystem and is as fast as standard disk access.

On client PCs the same shared folder can be mapped with the same letter so that a copy of the same main library can be used. In addition, the new feature that makes possible to play library server files directly if they are found from the file path can be used.

I have normally three SUBST commands in a .bat file that I have in the startup folder, for instance:

SUBST X: D:\music
SUBST Y: E:\video_images
SUBST Z: F:\new_stuff

My main library has been imported from the virtual X:, Y: and Z: drives for years even though the physical location has varied a lot.

I also have similar base folders and a subset of the media files on an external 1 TB USB drive. When I use it outside home with a laptop I can create the same drive letters easily with SUBST and once again use the same library (naturally the files that are not included are then offline, but otherwise the library works without any tweaking).