Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[DmitryB]

I'm having serious drag issues when 1) tagging music information (there is a massive delay between me typing and the text actually being entered) and then 2) if I rename or move the source folder and want to reconnect via F6 > Base Path Brose to find the newly renamed/moved folder. When I choose the hard drive folder and go into the general Music folder which contains all of my music in subfolders - it can take up to 2 mins for the folders to appear.

Any idea why such huge delays? Could it be the size of the library/amount of music? I do have a lot of albums and playlists. Is it just not able to keep this much info efficiently? Or is there something I can do to fix this?

I'm on a Mac Mini, the music is stored on an external G-Raid connected via USB-C.

[JimH]

No idea. 

If the RAID array is disconnected, what happens?  You could set up a new "test" library.

What else is unusual about your setup?

[blgentry]

On the Mac, if you have external drives connected, they will eventually go to sleep if you do not access them.  So if you try to read files on a sleeping drive, there will be a delay. 

But there's a more subtle effect also:  Some programs, when you open a file choosing dialog, and choose a drive, the Mac decides that must wake ALL DRIVES before it can show you what you want.  It wakes the drives one by one and waits for them.  I've seen this behavior for several years.  With multiple external drives, there can be a lengthy delay.  Perhaps 45 to 60 seconds on my system.

That may or may not apply to you.

Brian.

[DmitryB]

No idea. 

If the RAID array is disconnected, what happens?  You could set up a new "test" library.

What else is unusual about your setup?

I don't think there's anything else unusual, just that the music is on an external drive. (Which can't be that unusual??)

[DmitryB]

On the Mac, if you have external drives connected, they will eventually go to sleep if you do not access them.  So if you try to read files on a sleeping drive, there will be a delay. 

But there's a more subtle effect also:  Some programs, when you open a file choosing dialog, and choose a drive, the Mac decides that must wake ALL DRIVES before it can show you what you want.  It wakes the drives one by one and waits for them.  I've seen this behavior for several years.  With multiple external drives, there can be a lengthy delay.  Perhaps 45 to 60 seconds on my system.

That may or may not apply to you.

Brian.

I don't think the "drive asleep" is an issue here because it happens even while I'm doing continuous tagging. I decided to reorganize some of my source folders and I'm doing a bunch of it non-stop, so there's not enough time for the external drive to fall asleep.

I thought maybe MC is just having a tough time processing info if the library gets too large? Or if there's a cache that's overwhelmed somewhere? I dunno, but it's frustrating!

[JimH]

It's probably at the disk level, and not MC.

[blgentry]

1.  What format is the file system on your external drive?  It's best to use Mac format for drives attached to Macs.  These days that means APFS .  HFS+ is still acceptable too, but it's "old".
2.  When you open Finder and navigate to that folder on that drive, does it show the contents quickly?
3.  Have you rebooted the system since you noticed this behavior?

These are all just guesses.

Brian.

[DmitryB]

1.  What format is the file system on your external drive?  It's best to use Mac format for drives attached to Macs.  These days that means APFS .  HFS+ is still acceptable too, but it's "old".
2.  When you open Finder and navigate to that folder on that drive, does it show the contents quickly?
3.  Have you rebooted the system since you noticed this behavior?

These are all just guesses.

Brian.

I am apparently on Mac OS Extended (Journaled). This RAID was set up some time ago. I think it's possible to convert to APFS without erasing data?

This problem started a long time ago (notably it did not exist when I first got MC several years ago, but it seems to have gotten worse over time.) However, it's been some time (a couple of years probably!) that it's been a real PIA.

When I navigate to the folders from the desktop, everything always loads quickly. It's only when I try to do it from MC that things come to a screeching halt.

I've re-started the computer many many times (since this is a problem that's been ongoing for well over a year, I just always shrugged it off and sucked it up.)

[JimH]

Bad drive

[blgentry]

I am apparently on Mac OS Extended (Journaled). This RAID was set up some time ago. I think it's possible to convert to APFS without erasing data?

Changing file system types is destructive.  You would need to backup and restore all of your data.  You should probably have a backup anyway.  I have several copies of my music collection on various drives. 

But the difference between HFS (Mac OS Extended) and APFS shouldn't cause big slow downs.  If you had said you were using NTFS or something that's not really meant for Mac, that might have lead us to changing the file system.  But not in this case.

You said it's fast "from the desktop", which I"m guessing means from Finder.  If it's nice and fast there, then the drive is probably ok.  You might run disk utility "first aid" against the drive to check it.  Disk Utility can find "big problems", but it's kind of rare.

I'm can't think of anything obvious (other than a hardware problem) that might be causing this.  If you are certain that it is fast with everything other than MC, then maybe you have some sort of database corruption?

In that case, I would try to back up the MC database and then restore it.  But I'd probably want some more data before I did something like that.

About how big is your collection?  MC should tell you how many songs at the bottom of the display.
I assume you have folders for each album.  I'm reading that HFS+ has (in the past) had problems with more than 2000 to 3000 files or folders in a single folder.  I'm not sure if this applies to your situation or not.

Brian.

[JimH]

If you are certain that it is fast with everything other than MC, then maybe you have some sort of database corruption?

Or the data MC is accessing is on the dying drive.

[blgentry]

I just did a few experiments with an HFS+ (spinning) drive I have attached to a Mac.

1.  Created 3,000 files in a single directory.  Then navigated there with Rename, Move, and Copy Files > Directory > Browse.  It took about 15 seconds for MC to enumerate the files and show them.
  A.  Finder shows the files pretty much instantly.
2.  Created 7800 directories in a single top level directory and navigated to them with RM&C as above. MC shows these instantly.
  A.  Finder shows these instantly.
3.  Created 7800 files in a single directory and repeated.  MC takes about 50 seconds to enumerate.
  A.  Finder shows the files instantly.

I conclude that MC is going to be "slow" enumerating directories with THOUSANDS of files in a single directory.  But it is very fast with THOUSANDS of directories in a single top level directory.

Does the OP have thousands of files in a single directory?

Brian.

[JimH]

Finder may be pre-indexing the files.

An external drive may not perform as well.

[blgentry]

MC is clearly doing a stat() or similar call to determine some information about files and folders.  It's not a big deal.  Having thousands of files in a single folder isn't a very good idea; particularly for music.

Whether or not this applies to the OP is unknown until he responds.

Brian.

[bob]

MC is clearly doing a stat() or similar call to determine some information about files and folders.  It's not a big deal.  Having thousands of files in a single folder isn't a very good idea; particularly for music.

Whether or not this applies to the OP is unknown until he responds.

Brian.

That's a good deduction. I wouldn't be at all surprised if it's doing a stat().
It's really interesting how different filesystems are optimized for various operations.
To me APFS seems really slow on deleting a lot of files.

[DmitryB]

I just did a few experiments with an HFS+ (spinning) drive I have attached to a Mac.

1.  Created 3,000 files in a single directory.  Then navigated there with Rename, Move, and Copy Files > Directory > Browse.  It took about 15 seconds for MC to enumerate the files and show them.
  A.  Finder shows the files pretty much instantly.
2.  Created 7800 directories in a single top level directory and navigated to them with RM&C as above. MC shows these instantly.
  A.  Finder shows these instantly.
3.  Created 7800 files in a single directory and repeated.  MC takes about 50 seconds to enumerate.
  A.  Finder shows the files instantly.

I conclude that MC is going to be "slow" enumerating directories with THOUSANDS of files in a single directory.  But it is very fast with THOUSANDS of directories in a single top level directory.

Does the OP have thousands of files in a single directory?

Brian.

Brian, thanks so much for doing all that! Here's more info on how my system is set up: I have thousands of albums, many of them in high resolution (so ISOs from SACDs and DSFs, but also hi-res FLACs, etc.) There is a folder on the external drive for music, but within it each album is located in its own folder, some times there are subfolders. So let's say I'll have Beethoven Symphonies - Karajan folder and within it more folders of the various recordings of Beethoven by Karajan. Etc. etc. etc. So there are several thousand folders total within the MUSIC folder.

Also one interesting note on the delay in tagging music: the drag only happens if I do it via the separate Tag window. If I just select a category within the main MC window and select F2 to rename - the renaming happens right away. But within the Tag window - it is extremely slow.

As far as the Finder via MC taking extremely long time to open folders - that I don't know what to say. I'm just rearranging and renaming a some folders for easier organization, and from MC - it takes extremely long to open a folder. If I just open any window on the external drive via my regular Finder (clicking in directly into my exteranal drive from the desktop) everything works instantaneously.

I'm attaching a screengrab of what the the F6 window looks like if I am relinking an album to a new location (these options were pre-selected, I did not make these choices.) Is there perhaps a different option I could be using?

[DmitryB]

Finder may be pre-indexing the files.

An external drive may not perform as well.

I'm clearly a luddite on a lot of this stuff! But - is Finder pre-indexing files every single time I open it? Shouldn't it index it once and then retain the information? But this is what happens: I've moved some recordings of Bruckner conducted by Blomstedt into one source folder (they used to be scattered.) So now there's: Bruckner - Blomstedt - and within it are multiple folders of the various recordings he has made. I relink one album to the newly renamed/moved subfolder. Then I go to relink the second album and try to go back to the original Bruckner - Blomstedt source folder (which Finder already opened a minute ago), and it still takes a minute to open it again. So the indexing doesn't seem to be sticking. It seems to re-index every single time, even folders it just opened.

[DmitryB]

To give ya'll a visual idea of what my source folder looks like, here's a small section. Folders with subfolders in them for each individual album. In each subfolder, only one album.

[blgentry]

Your top level music folder, which opens when you do Browse from the Rename, Move, and Copy files screen... It has thousands of folders.  But are there FILES in that top level also, that are all by themselves?

If it is just folders, or just folders and a small number of files, then my experiment does not seem to apply.  I'm not sure where to go from here.

Brian.

[blgentry]

I've done a little more experimenting.  This time I created 26,000 directories (a-z, 1-1000 combined).  Finder loads these in about 15 seconds the first time and instantly thereafter.

MC takes about 3 minutes to display the list of directories.  This is clearly something with what MC is doing for each file or directory before it lists them.

Now the other issue, where changing metadata tags is slow...I don't know why that would be or if it's related, but it would seem to be.

I would suggest that doing file moves might be done better directly with MC.  Tell it where to move your files instead of "relinking them" after the fact.  For simple moves this will be easier as it is one step.  For big moves (hundreds or thousands of files), you are probably best doing the actual moves with Finder (or a similar tool) and then "relinking" in MC as you have been doing.

Bob (or other MC programmer):  Why would changing metadata be slow?  Does MC have to traverse the path of the file, doing a stat() at each folder path along the way?  I wouldn't think so, but why else would it be so slow to do a tag change?

Brian.

[DmitryB]

Your top level music folder, which opens when you do Browse from the Rename, Move, and Copy files screen... It has thousands of folders.  But are there FILES in that top level also, that are all by themselves?

If it is just folders, or just folders and a small number of files, then my experiment does not seem to apply.  I'm not sure where to go from here.

Brian.

Brian, there are no "loose" files in the top level music folder. Only folders with a single album per folder.

And it's odd that while using MC, when using the Tag window - typing words drags. If renaming a category with F2 function in the main window, the renaming is instantaneous. What about the Tag window that's different?

Thank you!

[DmitryB]

I would suggest that doing file moves might be done better directly with MC.  Tell it where to move your files instead of "relinking them" after the fact.  For simple moves this will be easier as it is one step.  For big moves (hundreds or thousands of files), you are probably best doing the actual moves with Finder (or a similar tool) and then "relinking" in MC as you have been doing.

I'll have to explore this option, I haven't used it yet. I'm not moving large numbers of files at a time. More like 2 or 3 albums at a time. Is there a tutorial for this? LOL

[blgentry]

Using RM&C to Rename files is similar to what you've already been doing.  But maybe you should just do what you know works for now and see if the MC team can figure out something helpful.  I may have been a bit too enthusiastic in recommending a process flow change.

If you want to read more about RM&C, it's here:

https://wiki.jriver.com/index.php/Rename,_Move,_and_Copy_Files#:~:text=To%20access%20this%20tool%2C%20select,Rename%2C%20Move%2C%20%26%20Copy%20Files

Brian.

[EnglishTiger]

When you use MC or the OS, i.e Win Explorer or Mac Finder, every time you rename, move or copy files and/or folders, the relationship between individual files and their MC Thumbnails get's broken.

In addition when you use Win Explorer or Mac Finder they will always be quicker than MC at listing folders and the files they contain because they invariably use a File Allocation Table; whilst MC has to check to see if every Folder and the Files within each Folder is/are present, it's building a list of "missing files" every time it is asked to list Folders and Files. Plus, if it's the 1st time it's accessed an individual folder it is probably rebuilding the thumbnails for those files in the background.

When it comes to Moving, Renaming and Copying Files and or Folders, regardless of the number of files involved, after having updated any relevant database/library fields/tags MC hands the task of Moving, Renaming or Copying of those Folders and Files over to the OS; something that is more obvious in Windows than on the Mac.

The decision about using MC or the OS to carry out a R, M & C operation should be based on the Risk Involved and not the Volume/Number of Folders and Files.
If ,for whatever reason, power is removed from the PC while MC is being used to carry out the task you are in danger of having a library/database that is WRONGLY telling you all the folders and files were copied to their "new location" and the only way you can get the folders and files that were not moved, renamed or copied into their "new location" is by using the OS and hope that when you re-open MC there are no missing or damaged/incomplete, files.

MC's R, M & C is a very useful tool but, unfortunately, it contains one major flaw - it updates the library/database Before renaming, moving or copying folders and/or files and not After it's done the task it was given.

[DmitryB]

Thank you EnglishTiger! That's a very interesting breakdown. (And I did notice that the artwork vanished (or dropped to very low resolution) on many albums (though not all. I suspect that's based on the format: FLACs are ok, DFF/DSF were not.)

I can only say that this major slow down in both re-linking (via Finder through MC) and Tagging was not always the case; early on everything was quick. It's only in the past couple of years that things slowed down significantly and I initially attributed it to the size the library. And with Tagging, to repeat, it's only very slow if doing it in the Tag window. If I choose an individual category in the main window and select F2, things adjust immediately.

Also, just to reiterate, if I move multiple folders into one new folder (for example: All U2 albums into one U2 folder) and now have to re-link each album - MC is slow for each album, even though I keep going back to the same U2 source folder. I would have figured MC would build its database the first time I open the new U2 folder, and then to relink each album would be quick. But it's slow every single time I go back to it, even though contents in that folder have not changed since the first time I opened it 30 seconds ago.

But I am getting a new external RAID. The one I've had has been a workhorse for years and I just started getting temperature alerts on it, so I figure I get a new G-Raid before it dies. It will be interesting to see if a fancy new external drive works better. (I'm a professional video editor and edit 4K video off it, so I know the drive itself works super fast.)

[blgentry]

I suspect a new drive/RAID won't help you.  I think you simply have "too many" folders in a single folder and MC is slow to stat() them.

Go to Finder and go into your main music folder.  Look at the status bar at the bottom of Finder. It should say something like "2600 items, 768 GB available".  That's the count of how many files and folders, plus how much free space.

Can you tell us how many "items" you have in your music folder?

Thanks,
Brian.

[DmitryB]

I suspect a new drive/RAID won't help you.  I think you simply have "too many" folders in a single folder and MC is slow to stat() them.

Go to Finder and go into your main music folder.  Look at the status bar at the bottom of Finder. It should say something like "2600 items, 768 GB available".  That's the count of how many files and folders, plus how much free space.

Can you tell us how many "items" you have in your music folder?

This is the current file structure:

(EXTERNAL) HARD DRIVE > MEDIA (folder)

And then within the MEDIA folder the music is divided between 3 folders: HD DSF folder, HD FLAC folder and SACD ISO folder. So the music is divided between 3 folders. That's how I initially set it up to separate the different formats, but am regretting it now. So in part what I'm doing (slowly, it will take a while) is moving everything into just one folder instead of those 3.

And the total number of folders between those 3 is about 5000 folders. BUT remember, there are many subfolders within those many of those 5000. (Artist folder that has several albums by the same artist. But that's inconsistent. Overtime the cataloging got out of hand!)

[blgentry]

It's probably a mistake to make one big folder. 

You should do some sort of organization that makes a smaller number of stop level folders.  It would be really nice to have it be less than 1000.  Maybe Artists could be the top level and then Albums under each artist. 

Unless you are saying that you have 5000 Artist folders.

Anything over about 1000 folders, at any individual level, is probably going to cause some kind of slow down.  The total number of folders going down the structure doesn't really matter.  What matters is how many files and folders MC has to touch at any given level.

This is an educated guess based on my experimentation.  My largest top level folder (Music/FLAC) has about 350 folders in it (they are named for Artist).  From the F6 > Browse dialog, it takes MC about 2 or 3 seconds to show the folders.

Typing in the Tagging Window is instant for me.

I do recognize that your collection is probably 10x or so the size of mine.

Brian.

[DmitryB]

The biggest of the 3 is HD FLAC folder. That has over 3,000 folders in it. And many of the folders have sub-folders. You're saying that 3,000 is too much and should be broken down even further? Let's say have COMPOSERS A-D folder and then COMPOSERS E-J folder, etc.? That would obviously increase the number of top-level folders, but have far fewer in each. I'm not opposed to doing that... but is there an easy way to move this and re-link in bulk? (Well obviously moving is easy, but re-linking........)

And my collection is enormous. As a classic music listener, there are dozens and dozens of recordings of the Beethoven Symphonies, and then Piano Sonatas, and then imagine that for everyone. So yeah, the collection itself is huge. I plan to listen to it all when I retire!

[blgentry]

Because this is new territory for me, I'm reluctant to recommend that you do a huge re-organization.  If the developers can confirm that my theory is correct, then a re-org would be the correct thing to do.

If or when you decide to do this, the RM&C tool can use Expressions.  Expressions can do calculations, like extracting the first letter of the composer name and things like that.  So, you could do some of these moves automatically.  If you were going to do this, I would start very small and do one album.  Then do several more one by one.  Then do some smaller groups of albums.  Work your way up so you can verify as you go.  But that's IF you are going to do this.  My theory might be wrong.

But even with a large classical collection, how can you have so many top level folders?  Are they all bare album names?  Because if it was by composer it would be a relatively small number (less than 50 I would guess).  If by orchestra then it would be probably less than 200.

Does Bob or anyone else from JRiver have any commentary on this?

Brian.

[HaWi]

The biggest of the 3 is HD FLAC folder. That has over 3,000 folders in it. And many of the folders have sub-folders. You're saying that 3,000 is too much and should be broken down even further? Let's say have COMPOSERS A-D folder and then COMPOSERS E-J folder, etc.? That would obviously increase the number of top-level folders, but have far fewer in each. I'm not opposed to doing that... but is there an easy way to move this and re-link in bulk? (Well obviously moving is easy, but re-linking........)

And my collection is enormous. As a classic music listener, there are dozens and dozens of recordings of the Beethoven Symphonies, and then Piano Sonatas, and then imagine that for everyone. So yeah, the collection itself is huge. I plan to listen to it all when I retire!

Sorry to barge in. I just want to mention that my music folder contains 1620 folders, most of them with subfolders for a total of ~41000 tracks of audio. It's organized as music/[Album Artist]/[Album]
The folder is located on a NAS and mounted on my iMac via smb. I do not have dramatic performance issues. Only when I change hundreds of tracks simultaneously it takes a few seconds.

[DmitryB]

BTW, after a recent update - the tagging speed increased significantly! So... thanks!