Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rjm]

I have a Core i5 3570K.

When analyzing 4 files at a time (JRiver's max) it uses 50% of my cpu.

My favorite video encoder uses 100% of my cpu and the system is plenty responsive for other tasks.

This suggests JRiver could double its audio analysis throughput which would be helpful for large libraries.

[Matt]

You're probably disk bound.

[rjm]

Task manager reports disk @ 3-8% (around 25 MB/s for JRiver).

[Matt]

Task manager reports disk @ 3-8% (around 25 MB/s for JRiver).

Interesting.  What types of files?

It would be easy enough to allow 8 files at once.

I'll do a little testing on our side and follow-up in a few days.

[rjm]

Right now I am analyzing video (mp4, mkv, avi).

Can provide data on any file type that interests you.

[Hendrik]

Analyzing too many files at once would hit the disc with a lot of random IO instead of sequential reads, which are not nearly as fast anymore.

[6233638]

I have suggested it a number of times, but the analyzer really needs to become smarter about how it works.

Rather than sorting files in the order they were displayed in the current view, if I have files from multiple drives selected (uncommon with music, common with film) it should try to assign a drive to each core.
The way MC currently does things, it is I/O bound the majority of the time when analyzing large groups of files. If it were to assign a drive to each core, this should happen far less often.

I also wonder if MC could recognize when it's I/O limited and when it's CPU limited.
If there are more drives than cores (assuming files have each been assigned to a core) and CPU usage is low, perhaps it would be best to analyze more than four files at once. (on a quad-core system)

I think it would also be useful to have some kind of logic that would get the most files analyzed in the maximum amount of time, rather than analyzing them as presented.
E.g. focus on analyzing stereo files before multichannel ones, analyze the shortest albums first, lower bit-depth and sample rate files first, prioritize lossy/lossless files in order of decoding/analysis speed etc.

It would also be really useful to have the option of pausing analysis, rather than having to stop it when you need to use the disk for something else during the middle of a large group of files being analyzed.

[rjm]

Analyzing too many files at once would hit the disc with a lot of random IO instead of sequential reads, which are not nearly as fast anymore.

That makes sense.

[Matt]

Next build:
Changed: Analyze Audio allows analyzing up to 8 files at a time instead of 4.

As Hendrik mentioned, this might help and might not depending on your disk situation.

I also noticed contention on a lock in the player core that's keeping it from pushing the CPU as hard as I'd like while analyzing.  I'll put this on the someday list, because we have to tread lightly here (since all audio handling is built on that core).

[rjm]

thanks!

[rjm]

Analyzing too many files at once would hit the disc with a lot of random IO instead of sequential reads, which are not nearly as fast anymore.

I was thinking some more about this and something doesn't add up.

As I change the number of analysis threads the cpu load changes linearly: 1=12%, 2=25%, 3=36% 4=50%.

This suggests we are not constrained by random disk IO. It suggests more cpu should be applied to each thread.

Why is it possible for one video encoding task to consume 100% cpu whereas one MC audio analysis task can only consume 12%?

[InflatableMouse]

I think the reason you're capping 100% with 1 videofile is because MC actually needs to decode the entire file, I don't think its able to decode audio only for analysis.

I have the same CPU as you in my HTPC.

In my case, analyzing audio, 1 file averages 35%. 2 56%, 3 75% and 4 ~92% (totalling 100% with varying percentages for other threads).

In my case this averages 10MB/s for 1 file, reads on disk, 21MB/s for 2 files, 32MB/s for 3 files and 40MB/s for 4 files. Other counters like IO p/s and disk queue lengths are all on the low side too.

[Hendrik]

As I change the number of analysis threads the cpu load changes linearly: 1=12%, 2=25%, 3=36% 4=50%.

Thats because audio decoding and analysis is not a multithreaded process, so one file can only use one core (and with 4+4 cores, one core is about 12.5% CPU usage)
Its a very simple linear operation, and as such not many opportunities present themself to multi-thread the analysis. While it would be possible, the code complexity would probably exceed the potential gain.

Your CPU has 4 cores (and 4 virtual cores), so with 4 threads its already quite busy, even if Windows says its only 50%. Adding 4 more threads would make Windows say 100%, but the speed would only increase by about 20-30% (thats the bonus you get from Hyper-Threading). Matt did increase the limit, so you can run it on 8 threads if you wish, but it won't make it twice as fast on your CPU. Sadly Windows isn't smart enough to handle that sanely, so in short, the CPU usage does not scale linear with performance.

You cannot compare this to video encoding, which is a very complex thing to do, with many tasks that can be performed in parallel, and use more CPU cores in the process.

PS:
It only decodes audio, if you also had to decode video, it would take much much longer.

[rjm]

Thats because audio decoding and analysis is not a multithreaded process....

Thanks for the illumination. With 4 cores I expected a busy thread to consume 25%, not 12%. I was not aware of the effect of virtual cores. Also did not know that video encoding is easier to multi-thread then audio.

[InflatableMouse]

A Core i5 doesn't have hyperthreading, its just 4 cores, no virtual ones.

If you have an SSD in your PC, maybe you can try copying a bunch of files to it and analyze them then.

I find it strange that my machine would be twice as fast as yours while my disks aren't doing that much.

[Hendrik]

Hm you're right, his CPU doesn't have hyper threading. Odd that it would use *exactly* half the CPU then, that seems rather odd (and is what confused me, i guess)

[rjm]

A Core i5 doesn't have hyperthreading, its just 4 cores, no virtual ones.

If you have an SSD in your PC, maybe you can try copying a bunch of files to it and analyze them then.

I find it strange that my machine would be twice as fast as yours while my disks aren't doing that much.

Hmm... starting to wonder if something is wrong with my system.

I analyzed some video files on my SSD and observed no change in CPU (50%).

Big question that is worrying me is why are you with same processor able to consume 100% cpu?

I poked around in the bios cpu settings. I had virtualization disabled so I enabled it but it made no difference. Did not see any other settings that I thought I should change. Let me know if you have any ideas.

May I ask you to run the MC benchmark and report your results? Here are mine:

=== Running Benchmarks (please do not interrupt) ===

Running 'Math' benchmark...
    Single-threaded integer math... 3.752 seconds
    Single-threaded floating point math... 2.497 seconds
    Multi-threaded integer math... 1.938 seconds
    Multi-threaded mixed math... 1.300 seconds
Score: 2003

Running 'Image' benchmark...
    Image creation / destruction... 0.302 seconds
    Flood filling... 0.393 seconds
    Direct copying... 0.433 seconds
    Small renders... 1.185 seconds
    Bilinear rendering... 0.750 seconds
    Bicubic rendering... 0.497 seconds
Score: 6181

Running 'Database' benchmark...
    Create database... 0.552 seconds
    Populate database... 1.130 seconds
    Save database... 0.127 seconds
    Reload database... 0.032 seconds
    Search database... 0.971 seconds
    Sort database... 0.980 seconds
    Group database... 0.570 seconds
Score: 4930

JRMark (version 19.0.40): 4371

[InflatableMouse]

=== Running Benchmarks (please do not interrupt) ===

Running 'Math' benchmark...
    Single-threaded integer math... 3,663 seconds
    Single-threaded floating point math... 2,431 seconds
    Multi-threaded integer math... 1,917 seconds
    Multi-threaded mixed math... 1,291 seconds
Score: 2043

Running 'Image' benchmark...
    Image creation / destruction... 0,472 seconds
    Flood filling... 0,388 seconds
    Direct copying... 0,443 seconds
    Small renders... 1,160 seconds
    Bilinear rendering... 0,762 seconds
    Bicubic rendering... 0,510 seconds
Score: 5890

Running 'Database' benchmark...
    Create database... 0,398 seconds
    Populate database... 1,233 seconds
    Save database... 0,354 seconds
    Reload database... 0,062 seconds
    Search database... 0,988 seconds
    Sort database... 0,953 seconds
    Group database... 0,573 seconds
Score: 4712

JRMark (version 19.0.41): 4215


Sorry I just realized you're only talking about analyzing video files. I thought video was going fine but analyzing music files were slow.

For analyzing video files (mkv, 720p):
1: 17%
2: 34%
3: 51%
4: 68%

With 4 threads, disk is doing 70-100MB/s.

Next I analyzed 4 video files on separate disks. I'm getting 100% cpu and cumulative disk throughput averages 140MB/s.

If your video files are on the same disk, I think you're hitting a disk limit as Matt suggested. I think my disk is slightly faster which is why my results are slightly higher.

[rjm]

I feel better now...

Copied some audio files to my SSD and cpu went to 100% when analyzing.

So it would seem I am disk bound when I see only 50% cpu.

Thanks to both of you for your help.

[6233638]

With 4 threads, disk is doing 70-100MB/s.
Next I analyzed 4 video files on separate disks. I'm getting 100% cpu and cumulative disk throughput averages 140MB/s.

This is why I wish the analyzer did some sort of load balancing when analyzing a lot of files. It's not too much of an issue for me now, as I have analyzed the majority of my library at this point, but it would have made things go much quicker.

[Matt]

Thats because audio decoding and analysis is not a multithreaded process

Actually audio analysis uses two threads: one to decode and one to analyze.

[Matt]

Actually audio analysis uses two threads: one to decode and one to analyze.

Like I mentioned above, the two threads were waiting on each other more than they needed to for audio files.

Next build:
Faster: Improved thread parallelism when analyzing audio (and to a lesser extent regular playback).

It's around 20% better at utilizing multiple CPU cores because of this change.

[InflatableMouse]

Nice!

[Hendrik]

Like I mentioned above, the two threads were waiting on each other more than they needed to for audio files.

That would explain why it only looked like one most of the time.

[6233638]

Wow, it looks like you're really hitting the CPU hard now. As I just found out, it looks like I had not analyzed all of my DSD tracks with MC19 yet.

I started analysis on four multichannel tracks, and I could no longer control my mouse. It would lag seconds behind any movement I made, moved extremely slowly across the screen, and took me about two minutes before I was actually able to select MC and stop the analysis.

There really needs to be some kind of limiter to prevent this from happening.

[rjm]

Wow, it looks like you're really hitting the CPU hard now. As I just found out, it looks like I had not analyzed all of my DSD tracks with MC19 yet.

I started analysis on four multichannel tracks, and I could no longer control my mouse. It would lag seconds behind any movement I made, moved extremely slowly across the screen, and took me about two minutes before I was actually able to select MC and stop the analysis.

There really needs to be some kind of limiter to prevent this from happening.

I did not see this. My system was responsive while analyzing. But I don't know what multi-channel DSD is so I probably don't have any.