Topic: Speed issues (Read 2048 times)

MrU · « **on:** October 10, 2021, 11:00:33 am »

Why is the updating of library [adding files to the library] taking soooo long.

I'm generally replacing my MP3 files with FLAC files as I find them.
however my collection is large [approx. 4TB]. But I am doing the update as I go along, slowly.

Before my updating has been much quicker but for some reason it has slowed down enormously.
I imagine its something to do with the option settings but without some help I'm stuck at the slow speed.
Any help would be appreciated as this is becoming a very slow job.

marko · « **Reply #1 on:** October 10, 2021, 11:20:12 am »

I don't know.

More info regarding your setup and the files being imported could help others to help you...

comox · « **Reply #2 on:** October 12, 2021, 02:41:30 am »

After updating from 28.0.66 to 28.0.73 my sync to a handheld device local folder has significantly slowed down.

I have a fast computer and this operation has always been a grunt for MC but the time to complete has now more than quadrupled to an unacceptable couple minutes.

I see you made some changes to expression evaluation. Could one of these changes have slowed things down?

Here are the expressions I use for my audio and video paths:

Audio Path: Audio\ifelse(isequal([Filename (path)],Audiobook,8),Audiobook\[Artist] - [Album],isequal([Filename (path)],Comedy,8),Comedy\[Artist] - [Album],isequal([Filename (path)],Course,8),Course\[Artist] - [Album],isequal([Type],Single,8),Music\[Artist Filename] - [Year] - [Album],isequal([Type],Various,8),Music\$Various Artists - [Album],isequal([Filename (path)],Podcast,8),Podcast\[Artist] - [Name],isequal([Filename (path)],RadioPlay,8),RadioPlay\[Artist] - [Album],1,Other\[Artist] - [Album])\

Video Path: ifelse(isequal([Filename (path)],Documentary,8),ifelse(isequal([Album],[Name],1),Video\Documentary,1,Video\Documentary\[Album] - [Year]),isequal([Filename (path)],Course,8),ifelse(isequal([Album],[Name],1),Video\Course,1,Video\Course\[Artist] - [Album]),isequal([Filename (path)],Podcast,8),ifelse(isequal([Artist],[Album],1),Video\Podcast,1,Video\Podcast\[Artist] - [Album]),isequal([Genre],Movie,1),Video\Movie,isequal([Genre],TV,1),Video\TV\[Series],isequal([Filename (path)],Music /(video/),8),Video\Music,isequal([Genre],MyImage,1),Memories,1,Video\Other\[Artist] - [Album])\

JimH · « **Reply #3 on:** October 12, 2021, 12:07:21 pm »

Turn off audio analysis, lyrics lookup, cover art, thumbnailing, etc. to see if it makes a difference.

Antivirus can slow things way down.

External drives also.

comox · « **Reply #4 on:** October 12, 2021, 01:10:21 pm »

To eliminate other possible explanations for the slow down I changed nothing in my library and reverted to 28.0.66, and took some measurements to quantify my perception.

Time to complete handheld sync to local folder:
27.0.66 69 seconds
27.0.73 604 seconds

27.0.73 is 10 times slower.

Edit: see below for correction, I made a typo on the version numbers

JimH · « **Reply #5 on:** October 12, 2021, 01:38:57 pm »

Quote from: comox on October 12, 2021, 01:10:21 pm

To eliminate other possible explanations for the slow down I changed nothing in my library and reverted to 28.0.66, and took some measurements to quantify my perception.

Time to complete handheld sync to local folder:
27.0.66 69 seconds
27.0.73 604 seconds

27.0.73 is 10 times slower.

Compare your settings. Conversion, for example. Antivirus.

You could back up in MC28, then load an MC27 library, with settings.

JimH · « **Reply #6 on:** October 12, 2021, 01:53:23 pm »

You can turn on logging (see the wiki) and read the log afterward. The times are shown in milliseconds.

comox · « **Reply #7 on:** October 12, 2021, 02:19:21 pm »

Quote from: JimH on October 12, 2021, 01:38:57 pm

Compare your settings. Conversion, for example. Antivirus.

You could back up in MC28, then load an MC27 library, with settings.

I'm so sorry. I made a typo on the version. I'm using V28. Here are correct results:

Time to complete handheld sync to local folder:
28.0.66 69 seconds
28.0.73 604 seconds

I changed nothing between tests except the installed version of MC.

In both cases, sync reported there was nothing to sync, as was expected.

JimH · « **Reply #8 on:** October 12, 2021, 02:28:05 pm »

Ok. If you go back to the newer build now, what happens?

comox · « **Reply #9 on:** October 12, 2021, 03:21:08 pm »

Quote from: JimH on October 12, 2021, 02:28:05 pm

Ok. If you go back to the newer build now, what happens?

Reinstalled 28.0.73, time to sync: 603 seconds
Reverted again to 28.0.66, time to sync: 70 seconds

Results are repeatable.

28.0.73 is 10 times slower than 28.0.66.

JimH · « **Reply #10 on:** October 12, 2021, 04:56:52 pm »

What about antivirus? If you're using Windows, did you configure Windows Defender?

Matt · « **Reply #11 on:** October 12, 2021, 05:07:59 pm »

Did you say it's not actually syncing any files? So what is it spending the time doing? Logging might offer a clue.

comox · « **Reply #12 on:** October 12, 2021, 05:16:49 pm »

Quote from: JimH on October 12, 2021, 04:56:52 pm

What about antivirus? If you're using Windows, did you configure Windows Defender?

Yes, I disabled antivirus for MC many years ago. Nothing has changed except the version of MC.

comox · « **Reply #13 on:** October 12, 2021, 05:35:22 pm »

Quote from: Matt on October 12, 2021, 05:07:59 pm

Did you say it's not actually syncing any files? So what is it spending the time doing? Logging might offer a clue.

I have a "handheld device" which I've configured to be a folder on a fast local hard drive.

MC is checking to see if any of the 12,000 files on the handheld device have changed, or if any new files have been added to the handheld device.

It takes 600 seconds to tell me nothing has changed on 28.0.73 and 60 seconds to tell me nothing has changed on 28.0.66.

If I add some files to sync, both 28.0.73 and 28.0.66 sync the files at the same usual fast speed.

I have a hunch something has slowed down in the changes you made to expression evaluation, which is why I provided my expressions above.

Attached is a screen cap of my options.

One more point, I haven't changed anything in the configuration of my handheld device for about 5 years, and have used this configuration with many versions of MC.

28.0.73 is the first time I've seen a problem.

Matt · « **Reply #14 on:** October 12, 2021, 05:37:58 pm »

Can you just test without expressions? That would let us know. If that's the cause, share the expressions you're using. Thanks.

comox · « **Reply #15 on:** October 12, 2021, 05:45:19 pm »

Quote from: Matt on October 12, 2021, 05:37:58 pm

Can you just test without expressions? That would let us know. If that's the cause, share the expressions you're using. Thanks.

Sorry I don't know how to do that because the expressions are central to how the whole thing works.

I have not changed these expressions for about 5 years. They've worked perfectly for many versions of MC until 28.0.73. And they still work with 28.0.73, just 10 times slower.

Here are the expressions I use for my paths:

Audio Path: Audio\ifelse(isequal([Filename (path)],Audiobook,8),Audiobook\[Artist] - [Album],isequal([Filename (path)],Comedy,8),Comedy\[Artist] - [Album],isequal([Filename (path)],Course,8),Course\[Artist] - [Album],isequal([Type],Single,8),Music\[Artist Filename] - [Year] - [Album],isequal([Type],Various,8),Music\$Various Artists - [Album],isequal([Filename (path)],Podcast,8),Podcast\[Artist] - [Name],isequal([Filename (path)],RadioPlay,8),RadioPlay\[Artist] - [Album],1,Other\[Artist] - [Album])\

Video Path: ifelse(isequal([Filename (path)],Documentary,8),ifelse(isequal([Album],[Name],1),Video\Documentary,1,Video\Documentary\[Album] - [Year]),isequal([Filename (path)],Course,8),ifelse(isequal([Album],[Name],1),Video\Course,1,Video\Course\[Artist] - [Album]),isequal([Filename (path)],Podcast,8),ifelse(isequal([Artist],[Album],1),Video\Podcast,1,Video\Podcast\[Artist] - [Album]),isequal([Genre],Movie,1),Video\Movie,isequal([Genre],TV,1),Video\TV\[Series],isequal([Filename (path)],Music /(video/),8),Video\Music,isequal([Genre],MyImage,1),Memories,1,Video\Other\[Artist] - [Album])\

Images Path: ifelse(isequal([Genre],MyImage,1),Memories,1,Photos)\

Data Path: Documents\ifelse(isequal([Filename (path)],Book,8),Replace(RemoveLeft([Filename (path)],10),_,\),isequal([Filename (path)],Magazine,8),Magazine,isequal([Filename (path)],Paper,8),Replace(RemoveLeft([Filename (path)],10),_,\),isequal([Filename (path)],Podcast,8),Podcast,isequal([Filename (path)],Course,8),Course,1,Other)\

comox · « **Reply #16 on:** October 12, 2021, 05:49:23 pm »

I'm just speculating that the cause might be a change to expression evaluation.

It could be something else.

Did you change anything that might slow down the traversal of hierarchical playlists?

Or drive i/o?

Matt · « **Reply #17 on:** October 12, 2021, 06:00:34 pm »

If it's a virtual device I wonder if you mailed me a library backup? I could test the same on my end. I'm matt at jriver. Thanks 👍

comox · « **Reply #18 on:** October 12, 2021, 06:08:55 pm »

Sorry no can do, private data.

Maybe some people out there with fancy Smartlist expressions can do a few tests to see if they spot anything slowing down?

Matt · « **Reply #19 on:** October 12, 2021, 06:14:17 pm »

Quote from: comox on October 12, 2021, 06:08:55 pm

Sorry no can do, private data.

Maybe some people out there with fancy Smartlist expressions can do a few tests to see if they spot anything slowing down?

That's too bad.

Just sent you an email about downloading test builds to find the exact start.

Thanks.

comox · « **Reply #20 on:** October 12, 2021, 06:19:33 pm »

Got it. Back as quick as I can with results.

comox · « **Reply #21 on:** October 12, 2021, 06:49:58 pm »

In case anyone is curious what I'm doing with this handheld device that is actually a folder on my local hard drive, I've found a very nice way to get media on and off my Android phone.

I use Resilio Sync to monitor and synchronize that folder with a folder on my phone via wi-fi.

Android's USB interface is horribly unreliable and this method sidesteps the whole issue.

To add a file to my phone, I simply add that file to an MC playlist. To remove a file from my phone I simply remove that file from an MC playlist.

Then I synchronize the MC handheld device and the changes magically show up on my phone.

Matt · « **Reply #22 on:** October 12, 2021, 08:02:26 pm »

I worked with Rob and we found it. More tomorrow after a little thinking. Thanks for the help 🙂

comox · « **Reply #23 on:** October 12, 2021, 08:08:57 pm »

Best piece of software ever written.

I have spent a gazillion hours with MC and love the product.

Matt should be very proud.

nathanchavez · « **Reply #24 on:** October 12, 2021, 08:27:29 pm »

Quote from: comox on October 12, 2021, 08:08:57 pm

Best piece of software ever written.

I have spent a gazillion hours with MC and love the product.

Matt should be very proud.

Completely agree!

Matt · « **Reply #25 on:** October 13, 2021, 10:25:07 am »

Should be fixed in the build that's up now. Thanks to everyone for the help.

I changed to always save changes when expressions changed because of another request and the knock on wasn't too good! Figured out a hopefully better solution.

Matt · « **Reply #26 on:** October 14, 2021, 09:38:42 am »

Please make sure you test build 75:
https://yabb.jriver.com/interact/index.php/topic,130944.0.html

Thanks.

INTERACT FORUM

Author Topic: Speed issues (Read 2048 times)