Topic: just passed 100k files - performance remains impressive (Read 4723 times)

rjm · « **on:** February 19, 2007, 09:35:39 pm »

Don't see any performance difference from when I had 10k files in my library.

I started out using MusicMatch Jukebox and it died at about 5k files.

Very impressive JRiver. Keep up the good work!

benn600 · « **Reply #1 on:** February 20, 2007, 04:14:04 pm »

Can someone take the opportunity to install iTunes, add 50K, 100K mp3 files or so and then record a screen capture using iTunes. That would be an interesting test and I'd love to see it. I'm only at around 11,200 FLAC files plus lots of DVDs and Pictures which takes me to around 28,000 files and it performs very well. I find that Vista hurts performance a bit so it may not be a great idea to install Vista. I remember my core2 system with XP running MC amazingly well. It runs a lot better when all the cover art is cached so it doesn't spend time caching the thumbnails, thus slowing you down.

benn600 · « **Reply #2 on:** February 20, 2007, 04:15:17 pm »

Interesting:

Quote

I've always wondered whether there was any limit on the number of songs that can be added to an iTunes library. I've just found it.

32,000 songs appears to be the max. I was at 31,992, and added a folder with 12 tunes in it. All of the songs appeared to be added (they showed up in the progress box, and, as iTunes was testing the volume levels, it went through all 12 tunes). But the list is stuck at 32,000 songs, and only the first 8 of the 12 tunes I just added are showing up. I then added another folder with 15 tunes in it, and iTunes went through the motions of adding them to the library, with no change in the displayed list.

It's not just a limit on the number of tunes that can be displayed at one time. If I limit the list by typing in the search box or selecting just the artist that I just added, the recently-added songs still are not present in the list.

I guess I'm going to have to lose something if I want to add anything else to the library.

This appears to be from 2003 and iTunes 3. lol. So it is probably better now.

AoXoMoXoA · « **Reply #3 on:** February 20, 2007, 05:01:18 pm »

I am also at over 100,000 files but seeing some slowdown

However in defense of Media Center 12;
my drives are getting low on free-space, and I have seen this have a drastic effect in the past.
Time for me to do some house-cleaning, or hard-drive shopping.

KingSparta · « **Reply #4 on:** February 20, 2007, 05:04:44 pm »

Quote

my drives are getting low on free-space, and I have seen this have a drastic effect in the past.

why would that have an effect on speed?

AoXoMoXoA · « **Reply #5 on:** February 20, 2007, 05:43:12 pm »

Quote from: KingSparta on February 20, 2007, 05:04:44 pm

why would that have an effect on speed?

No clue.
But last time my free space got tight I saw a noticable slow-down. Increased free-space and it got noticably faster.

benn600 · « **Reply #6 on:** February 20, 2007, 09:39:14 pm »

I could probably map my network server 4 times and import. Then, I'd have 4 times my regular files (all with a different drive letter) and I could experience MC with more than 100 K files. I may do that for fun when I get a minute.

ThoBar · « **Reply #7 on:** February 21, 2007, 12:05:12 am »

Quote

why would that have an effect on speed?

File fragmentation is the most likely cause, although this shouldnt really effect MC's DB (due to its small file sizes). It will become an issue if other regularly accessed files are fragmented, either OS or commonly accessed data files.

If drives aren't defragmented reasonably often, fragmentation rapidly increases on drives that are filling up and have files that are changed frequently.

C.

rjm · « **Reply #8 on:** February 21, 2007, 02:34:53 am »

Assuming you are using the NTFS file system, there does not appear to be any evidence that fragmentation significantly affects performance.

The defrag vendors would like you to believe otherwise.

hit_ny · « **Reply #9 on:** February 21, 2007, 06:51:53 am »

Don't NTFS partitions, get slower to access, once there is less than 10% free of their total capacity ?

rjm · « **Reply #10 on:** February 21, 2007, 11:59:33 am »

hit_ny,

I was not aware of this but thank you for mentioning it. I have noticed a slow down on one of my media drives that has 13% free space.

KingSparta · « **Reply #11 on:** February 21, 2007, 12:07:34 pm »

Quote from: hit_ny on February 21, 2007, 06:51:53 am

Don't NTFS partitions, get slower to access, once there is less than 10% free of their total capacity ?

Why?

InflatableMouse · « **Reply #12 on:** February 21, 2007, 12:38:07 pm »

Quote from: KingSparta on February 21, 2007, 12:07:34 pm

Why?

Without getting into the tedious details and the flawed design of ntfs compared to filesystems used on Unix systems, it is because it becomes harder for the system to find contineous free space to store a file, it needs to break it up and store it in different places on the disk. In the beginning of a disk becoming defragmented, this isn't an imediate problem but believe me, once it starts it goes fast, especially when running out of free diskspace.

The less free space there is, the harder this becomes for the system. Most systems won't suffer badly from a couple of fragmentations, but I have seen systems with hundreds of fragments on a single file. These systems would literally take minutes launching internet explorer or ms word. Clicking the print button would basically hang the system for minutes with a grinding noise from the harddisk.

I am currently dealing with a couple of Windows 2003 file clusters with 500Gb LUNs ('disks' on a SAN) that are running out of space and are heavily fragmented. Believe me, its not funny.

http://www.pcguide.com/ref/hdd/file/ntfs/relFrag-c.html is simple article on NTFS defragmentation and deals with the myth that NTFS doesn't need defragmentation.
A little more technical is http://www.ntfs.com/ntfs_optimization.htm.

MadJewDisaster · « **Reply #13 on:** February 21, 2007, 03:18:12 pm »

Reading this topic with my VERY slow XP running , i just remember than i did not run offline defrag since few months
Did it and now XP running fairly fast.NTSF format here.

Concerning MC, some my drives are full at more than 90% and import to MC from it is really really slow.
But after that , things runing smootly .
Number of files ? Do not know around 500 giga of AVI and 600 giga of mp3-mpc and flac
And, if you play a song not adding any tags like rating or last played etc , do Windows needs to rewrite the file to play it? Therefore creating fragmentation

KingSparta · « **Reply #14 on:** February 21, 2007, 03:24:21 pm »

I have read some articles on how to make NTFS faster (Optimizing NTFS) some of the information comes from Microsoft including patches to the OS to enhance The NTFS from becoming slow.

In Days Long Since Past Symantec's Created The Defrag Utility For MS Dos And Later Losing That Contract To www.diskeeper.com (Do To Every Thing Symantec's Does Ridiculously Slows Down Your Computer) And A Dumbed Down Version Of The Disk-keeper Defragger Is Built Into Vista.

rjm · « **Reply #15 on:** February 21, 2007, 04:41:54 pm »

I checked and my slow media drive is not fragmented. So it must be the fact that it only has 13% free space remaing.

MadJewDisaster · « **Reply #16 on:** February 21, 2007, 04:56:05 pm »

Did you run what they call 'offline defrag'= defrag at boot before OS starting .
I do not think the XP defrag does it - You need a software

rjm · « **Reply #17 on:** February 21, 2007, 07:50:37 pm »

Yes I checked the need for an offline defrag. Thanks for the tip.

hit_ny · « **Reply #18 on:** February 22, 2007, 01:27:39 am »

13% should not make much diff, its only when it falls below 10%, even that isn't a fixed number, i'd say closer to 5% of free space is when you might see some slowdowns with NTFS.

runemail · « **Reply #19 on:** February 22, 2007, 09:43:36 am »

Running 64x Vista and MC12 with around 70k files here. No problems and fast, but you really need a lot of ram and a fast cpu. 2GB is the minumum for 64x.

Ran the vista RC x86 with 1gb and it performed pretty good too. So if you have a new/fast computer MC12 should perform about the same under Vista and XP.

JimH · « **Reply #20 on:** February 25, 2007, 11:33:47 am »

Some posts on hard drives were split here:
http://yabb.jriver.com/interact/index.php?topic=39191.0

I'll lock this thread since it's off topic now. Please start a new thread if necessary.

INTERACT FORUM

Author Topic: just passed 100k files - performance remains impressive (Read 4723 times)