INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: [Request] Improved multi-threading  (Read 1654 times)

RD James

  • Citizen of the Universe
  • *****
  • Posts: 1871
[Request] Improved multi-threading
« on: March 11, 2018, 08:51:18 am »

I think there are a few ways that Media Center's performance could be improved with additional multi-threading options.
CPUs with >8 logical cores are now available for less than $200, so I would like to see the limit on things like audio analysis and conversions increased from 8 simultaneous files to 16 or more.
 
I ran some tests using a Ryzen 1700X recently, and dBpoweramp was completing conversions 70% faster than Media Center by converting 16 files at once rather than 8.
Though my CPU has 16 logical cores, there are now Threadripper CPUs for enthusiast builds with 32 logical cores. I believe that dBpoweramp is currently limited to 16 simultaneous conversions, so there is the potential to leapfrog them here.
Upcoming CPUs next year are rumored to go beyond 32 logical cores, so if the options are being scaled up I might suggest a limit higher than 32.
 
DSD decoding is limited to a maximum of four threads.
This is not ideal, since arguably the main reason to still use SACD today is the multichannel content. Ideally you would have at least one core per channel (6) as DST compression is quite CPU-intensive.
 
There is another issue with the way that DSD decoding is handled.
Since it uses four threads per file, audio analysis and conversion does not work well with DSD files. Analysis of DSD tracks runs much better when limited to 1/4 as many tracks as logical cores.
 
I've also seen people on the forum here selecting the wrong number of files for analysis - after all, why would they not want to process as many files as possible?
What I'd like to suggest would be an 'auto' mode for things like audio analysis and conversions.
This would convert/analyze one file per logical core that your CPU has for most files, and 1/4 as many files if it is handling DSD.

Video analysis is somewhat trickier. If the files are stored on an SSD, analysis scales well.
In a test of eight 30-minute videos, analyzing one file at a time took 90 seconds, while 8 at a time cut that down to 15 seconds.
With HDDs on the other-hand, analyzing multiple files at once can cause the disk to thrash and analysis can run slower than if you analyzed one or two files at a time. It's generally not an issue for short videos, but can be for longer videos. I'm not really sure that I have an answer for what would be best here. Perhaps like DSD 1/4 is a safe estimate.
Alternatively moving the analysis options into the options window, and having separate controls for Audio, DSD, and Video might be better than an 'auto' option, so that people can choose what is best for their setup.
 
Though there are threading options for thumbnail generation, it seems like thumbnails are only being built for one file at a time. When I clear the thumbnails and enter a new view, they appear to be generated sequentially rather than in batches of 4, 8, 16 etc.
Auto-import also seems to process one file at a time. I assume that this is because it's always running in the background, but with 16 logical cores, that's a bit too conservative. I would suggest changing this from one file, to one file for every four threads that the CPU has. That way a typical quad-core CPU will be unaffected by this change, while things will run faster on CPUs with more cores.
Logged

millst

  • Galactic Citizen
  • ****
  • Posts: 256
Re: [Request] Improved multi-threading
« Reply #1 on: March 11, 2018, 12:09:39 pm »

I like most of that, but disagree that background tasks should necessarily consume more CPU. I don't run MC on a dedicated machine and don't want it grinding to a halt because somebody dropped some pictures on the NAS. That's low priority and can finish whenever.

Even a dedicated machine might have issues if watching a movie with ROHQ and MC gets too aggressive with the background work. JRiver would have to be careful with the implementation. Maybe provide an option for those that want enable more aggressive background tasks.

-tm
Logged

RD James

  • Citizen of the Universe
  • *****
  • Posts: 1871
Re: [Request] Improved multi-threading
« Reply #2 on: March 11, 2018, 04:43:02 pm »

I like most of that, but disagree that background tasks should necessarily consume more CPU. I don't run MC on a dedicated machine and don't want it grinding to a halt because somebody dropped some pictures on the NAS. That's low priority and can finish whenever.

Even a dedicated machine might have issues if watching a movie with ROHQ and MC gets too aggressive with the background work. JRiver would have to be careful with the implementation. Maybe provide an option for those that want enable more aggressive background tasks.

-tm
That's why I proposed that auto-import processes 1/4 as many files as the system has logical cores (rounded).
If you have a typical dual or quad-core processor, there would be absolutely no change to the auto-import behavior.
If you have a quad-core processor with 8 logical cores (i7), or something with more then four physical cores, it starts to take advantage of the additional hardware to speed up the process, while ensuring that resource usage is kept very low.
Logged
Pages: [1]   Go Up