Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[MusicGuy]

I Have a very large Library, over 3 million tracks, looking for a better way to "Load" entire Library.

[blgentry]

MC's import function is the only real way to import your files if MC has never seen them before.  If you have previously imported them into MC, you can make a backup on one MC instance and then restore that backup into another MC instance.

Otherwise, import is the correct supported way to do this.

3 million tracks is pretty insane.  That's about 250,000 albums.  That's over 17 years of continuous play (at 3 minutes per track).  It sounds like you might be a collector of a wide range of things that aren't necessarily music.  Or perhaps you are collecting albums/songs that aren't a regular thing you listen to.

If any of that is true, you might consider partitioning your collection into a main collection and then some auxiliary collections.  You could import these separately.  Either as entirely separate Libraries (not usually a good idea) or as different Views within MC.

In any case, it sounds like you might benefit from considering the overall structure and purpose of your digital music collection.

Best of luck to you.
Brian.

[BryanC]

For initial import I'd suggest disabling all Auto-Import features like Audio Analysis, cover art lookup, building thumbnails, and tag writing (basically, just get the files into the database and get a feeling for performance). From there I'd start triggering the above MC Library features manually and perhaps on subsets of your collection, depending on performance and your hardware.

[HPBEME]

3 million tracks is pretty insane.

Pretty insane?  How about unbelievable?  How does one acquire that many tracks in a single lifetime?  250,000 albums at even a generously low estimate of only $10/each is still $2.5 million dollars!!!  I should be so lucky to have that much discretionary income hahahaha.  More power to you MusicGuy! 

oh… as for the import advice... +1 on everything Brian & Bryan said.  Spending time figuring out your desired structure in advance will pay dividends later.  You probably won't get everything exactly right out of the gate, so doing things in manageable chunks will help you understand the process (vs. committing yourself to one gigantic import).

[MusicGuy]

Guys,

Thanks for your replies, yes, my wife agrees with you, but she said, that I m completely insane, not pretty as you stated. No one in to days world need  library as large as mine. FYI, It as taken a life time (Started in the 60's), 8tracks to tape, Tapes to CD, CD to Mp3, put it together. begging everyone for there old CD's, buying every used CD I could find for less that a dollar. Oh Goodwill was very happy when my wife made me donated boxes and boxes of CD to them (you don't need them get ride of them), back to the fact that she knows I'm insane.

Please note, the library is not for sale, I only share it with my immediate family, and most of them prefer, use spotify.

But, I've done everything I can thing of (FYI, I was one of the one of the database in the 60's) to that IT from flat file to indexed files to random, to real data bases, worked on IMS, DB2, CinCo, IBDM, Orcale and several more)

My PC, is a 12Th gen Intel, has 128GB, SSD internal drives, 3.2 WD Gold drives. running on 1000g comcast internet.

Issue seems to be the amount of I/O to Media Center's Index and maybe the was I'm storing Album Art (putting the on the physical tracks with the music)

Again, No-one needs and I don't know anyone that would want to go to the expense of having one except me), if you would, don't ask, because the answer is, no.

But if anyone, can help reduce me twice yearly reload to under 5 to 6 day it would be very appreciated.

Musicguy

[blgentry]

But if anyone, can help reduce me twice yearly reload to under 5 to 6 day it would be very appreciated.

Twice a year reload?  You should not need to re-import.  Restoring an MC database backup should work for most circumstances.  A database backup should load relatively quickly.  I can't imagine it taking more than about 15 minutes or so.

Brian.

[This2ShallPass]

MusicGuy, you are my hero! I've got about 300K tracks in my collection, and I consider myself both a Collector and a Listener, and it's the hobby that keeps me off the streets. I too go pillaging at libraries, at friends' houses, Goodwill, record stores, etc. I have great pride in my scavenging abilities!

While only a 10th of the size of your collection, my PC loads the library (in MC 31 client) in 2-4 seconds from a NAS and the MC Id server a floor away via CAT6, but if I understand MC correctly, it's probably loading a local library (XML) file from my PC, which is 4 years old. Extrapolating that for you would mean that you shouldn't have to wait much more than a minute all told.

Cheers brother-from-another-mother!

[eve]

When I used JRiver for music, I got up to about ~2 mil tracks and somehow it worked. Was about 30s-1m to pull the library from the library server IIRC.
That's completely reasonable IMO


Honestly MusicGuy, you've worked on databases, it's hard to expect the performance of a standalone db from 'software' like JRiver. The jriver db itself stored on an NVMe drive makes a bit of a difference in speed for MC but not a huge amount (I assume it's just loading it into memory). At the scale you're working with, having some kind of dedicated database you interact with for queries may be a nice addition to MC for you. In my case I'm actually going through this process right now, for my music.