Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[WinoOutWest]

Background:  I've been on the same database for about 6 years now -starting at around 80gb to currently about 1.4TB.  Server running in the basement for storage, HTPC upstairs as main library and several other PCs sharing a copy of that library - all connected via GB lan.  I am having lots of problems with MC right now (audio playback freezing for 30s, crashing, 10-20 second pauses when adding a track to playing now etc) so I am thinking it is time to clean house.

I just created a new server (W2K3 server, 3TB storage across 9 drives, Athlon 64 X2 5000+ cpu, 1GB ram.  Right now my music collection spans 6 physical drives and I want to transfer that to 2 750GB drives)

I am thinking the easiest way for me to be able to convert my playlists while ensuring I start with a good clean DB would be to create M1, M2...M6 directories on the 2 750s and move the music from drive Music1 to M1, Music2 to M2 etc.  I would then import the collection from scratch creating a new database and then go through the PIA steps of updating my playlists as described in various other posts in here.

My questions are:
1) What are the gotcha's or suggestions and config standards as far as server setup goes to maintain performance with a large audio library. 
2) Will directory structure on my server impact performance at all?
3) Coverart:  does it make a difference in performance how I store cover art?  (ie embedded vs folder.jpg vs directory)
4) Any other advice from others who have experience with large libraries and keeping performance snappy?

Cheers!

[rjm]

I'll bet you a bottle of your favorite beverage that creating a new library will not solve your freeze/crash problems. I personally would chase down and fix that problem first before changing more (quite likely unrelated) variables. Have you tried a different player with your existing hardware to see if it behaves nicely?

1) What are the gotcha's or suggestions and config standards as far as server setup goes to maintain performance with a large audio library.

Someone else with more server experience should answer this.

2) Will directory structure on my server impact performance at all?

A good directory structure will make it easier for you to maintain your library (data backup, capacity expansion, etc.) however I do not think the structure (good or bad) will have any impact on performance. You should avoid a structure that is too deep or too long so that your music files do not bump up against the nasty Windows maximum character length for all nested folders. This can be a real pain when you hit it and you definately want to avoid it.

3) Coverart:  does it make a difference in performance how I store cover art?  (ie embedded vs folder.jpg vs directory)

I am not aware of any difference in performance between the cover art techniques. However from a simplicity and trouble-free perspective you cannot beat embedded. They all work but a search of this forum will uncover hundreds if not thousands of messages from people who had trouble with external artwork because they did not really understand how it worked and inadvertently broke it. Embedded on the other hand, always works and is almost impossible to mess up.

4) Any other advice from others who have experience with large libraries and keeping performance snappy?

My library is large and I have made many design changes to it over the years. My observation has been that the only things that impact performance are:
- number of items in library (I see a gradual decline in performance as my library grows)
- complexity of view schemes (complex schemes take longer to load, simple ones are fast)
- speed of your computer
- craftiness of JRiver developers (they find ways to make MC go faster from time to time)

If anyone has different performance experiences or tips I too would love to hear them.

[Alex B]

I too started my main MC database almost six years ago. I don't have any performance problems. From time to time I have compared my old database with a freshly imported one and I have not seen speed differences even though my old database contains more data.

If your problems are really caused by somehow corrupted database you can export it in XML/MPL format and import all field data to a new database. That should clear any possible structural problems.

You could split the library to a few new playlists and use the Export & Import Playlist tools. The Import Playlist tool can update existing library data from mpl files so you can first import the files from your hard drives and then apply all original field values from mpl playlists.

If you want to mass edit your base folders you can do that in the mpl files (mass edit the Filename fields with a text editor). Move the media files outside MC and import them to a new library. After the media files are initially imported to the database you can import the edited mpl files for returning the original field values (only the Import Playlist tool in the File menu can do this.)

1) What are the gotcha's or suggestions and config standards as far as server setup goes to maintain performance with a large audio library.

You already have a GB LAN so I don't know what else you could add. Personally I don't have any problems with accessing files on a switched 100 MB LAN and I have some big stuff like 500 MB Monkey's Audio disc image files with APL link files. I use mapped network drives and I don't run Auto-Import in the background.

2) Will directory structure on my server impact performance at all?

Keep the directory and file names as short as possible.

For example, I use
X:\artist\album\01 - track name
instead of something like
\\my fabulous server\my shared drive\my music\artist\album\artist - album - 01 - track name

I have configured my system so that the same mapped X:, Y: and Z: drives are available on all PCs. I use these bare drive letters as base folders. From time to time I use junctions on my server for combining physical drives or partitions. This is transparent in MC's database. When I buy a bigger drive I may remove the junctions and gather the folders on the new drive without making any changes to the database.

3) Coverart:  does it make a difference in performance how I store cover art?  (ie embedded vs folder.jpg vs directory)

I think that in theory external cover art should be faster, especially if you have big lossless audio files that store tags in the end of the file, e.g. APE. (It might be best to store the image file in the audio file folder to keep the link short and quickly accessible.) However, I don't think the speed difference can be significant. In view schemes MC uses the local thumbnail cache. The original image file is used only in the main display.

Also, if the embedded image is stored in the beginning of the audio file (e.g. MP3, OGG and FLAC) the process of adding cover art is usually many times slower because the complete audio file must be rewritten.

[John Gateley]

Keep the directory and file names as short as possible.

Does keeping the names short have an efficiency impact?

I'd suggest limiting the number of items in a directory, directory with a long list of files/sub-directories can take a long time to search.

j

[Alex B]

Does keeping the names short have an efficiency impact?

Maybe not in the server's NTFS file system, but at least it lessens the amount of bulk library data. In a very large library the cumulative effect may become observable.

[John Gateley]

Maybe not in the server's NTFS file system, but at least it lessens the amount of bulk library data. In a very large library the cumulative effect may become observable.

Yes, that makes sense, especially when Library Server is in use...

j

[WinoOutWest]

Does keeping the names short have an efficiency impact?

I'd suggest limiting the number of items in a directory, directory with a long list of files/sub-directories can take a long time to search.

j

hmmm... so how exactly can you limit the number of items in a directory when you have 200,000 plus mp3s and want to host that on 2 physical drives?  Right now I planning something like this: 

x:\m1\generalgenreorotherdescriptor\artist-album\song.mp3
(the m1..m6 representing the 6 drives I currently have that I want to move to the new 750s)

as you can see this leaves an awful pile of directories.  Are there any guidelines around this sort of thing for large libraries?

I'll bet you a bottle of your favorite beverage that creating a new library will not solve your freeze/crash problems. I personally would chase down and fix that problem first before changing more (quite likely unrelated) variables. Have you tried a different player with your existing hardware to see if it behaves nicely?

Any guesses what my favorite beverage might be?    I might be be making a bit of a presumption on the database being my problem - I use the HTPC for music and TV/Movies/Pictures and it appears to be solid with the music and JRiver.  Haven't loaded any other players.  I also use the same library on another desktop on the network and have similiar performance issues with JRiver.

[JimH]

Have you tried Library Tools/Rename Files from Properties?  It's a powerful tool to move files.  Try a few before you do very many.

[WinoOutWest]

Have you tried Library Tools/Rename Files from Properties?  It's a powerful tool to move files.  Try a few before you do very many.

But won't this perpetuate any DB problems that might be there?

[John Gateley]

hmmm... so how exactly can you limit the number of items in a directory when you have 200,000 plus mp3s and want to host that on 2 physical drives?  Right now I planning something like this: 

x:\m1\generalgenreorotherdescriptor\artist-album\song.mp3
(the m1..m6 representing the 6 drives I currently have that I want to move to the new 750s)

use m1\genre\artist\album\

I use 256 as a rule of thumb size target, but 1000 is probably okay.
This also depends on the file system implementation, how search is implemented in directories. Some file systems (Reiser?) handle large directories better than others. I'm not sure at all about Windows file systems.

j

[WinoOutWest]

use m1\genre\artist\album\

I use 256 as a rule of thumb size target, but 1000 is probably okay.
This also depends on the file system implementation, how search is implemented in directories. Some file systems (Reiser?) handle large directories better than others. I'm not sure at all about Windows file systems.

j

So if I did this could use the "Rename From Properties.." to accomplish the re-org correct?  And if I did this I would lose my playlists right?  (Or I'd have to manually fix then up with a find and replace somehow)

[John Gateley]

I don't know, give it a try and see. But I wouldn't worry about it too much. Even a thousand should search quickly.

Drinking a surprisingly nice "Cote Du Bone Roan"...

j

[WinoOutWest]

Drinking a surprisingly nice "Cote Du Bone Roan"...

j

Ever try the South African "Goats Do Rome"?

[JimH]

Ever try the South African "Goats Do Rome"?

I'll bet you have.  Very funny.  Thanks.

[John Gateley]

I did try the Goats Do Roam, I loved the name, and the wine was okay. Did you like it?

The Cote du Bone Roan was very good, considering the name, nicely balanced, and hints of subtlety that I can't quite get, but my wife can. And it turns out that Rosenblum Cellars (Chateau La Paws) donates part of the price to training animals for people with disabilities...

j

[WinoOutWest]

yeah its definitely a great label wine - and a great conversation piece.  Quality is so-so - found it getting a$$ier as it became more popular so haven't tried it in the last few years.
My current cheapie:  2006 Petite Syrah from De Bortoli (Australian).  All their wines are seem pretty good for their price but this is one of their cheapest at $12Cdn up here in Vancouver which is one of the more expensive places for wine.  A great cheapie that actually has some character.

[marinero]

have just been browsing through post and the use of properties to re-name caught my attention.  MC's help/explanation for directory and file name management still leaves me struggling (non-techie).  Library properties are as I'd like them but files don't populate Windows Properties columns. Would be grateful for clearer explanation of how best to utilise Directory, Base Path, Rule -  and Filename, Rule sections.    Is this because I've kept all my music as WAV files? Should I create new library and convert all files to some sort of lossless MP3?   Problem at mom is that playback of files through Slim Devices products doesn't show Properties correctly.