Topic: Managing a huge library (Read 5304 times)

linden_arden · « **on:** January 17, 2005, 09:23:01 am »

I have had a huge music collection for some time and I just could never figure out the best way to manage it. I have been playing around with Media Center for some time now and I think I have found what I need.

My question is about the best way to setup the library. Perhaps other folks have had similar experiences and can help me out before I set out to organize my 30,000 songs once and for all. I think I have the correct solution, but I want to make sure there isn't another method I might be over looking.

The best way to explain what I am looking for is to use an example: Consider the band Uncle Tupelo. If I take a song from an Uncle Tupelo album, it is not sufficient to just have th 'Artist' as Uncle Tupelo. Uncle Tupelo is a band made up of Jay Farrar and Jeff Tweedy. My music collection has tons of Jeff Tweedy: Uncle Tupelo work, Wilco work, Golden Smog, solo concerts, etc. Sometimes I feel like listening to Jeff Tweedy and I'd like to be able to just click 'Jeff Tweedy' and get his stuff rather than search through for all the bands he has been a part of. Moreover, there are several ways to describe Uncle Tupelo's music: country, alt-country, country rock, americana, and so on. It wouldn't make sense to just use a one genre label.

I think MC handles this all in stride. I setup two new library categories: "Musician" and "Stlye" and set them up to handle strings of data. So for an Uncle Tupelo song, the "Musician" field reads as "Uncle Tupelo;Jay Farrar;Jeff Tweedy" and the "Style" field reads as "country;alt-country;country rock;americana".

This seems to work. Does this seem like the correct solution for my situation?

Also, if you have a huge, very detailed library, and you have any tips/advice/warnings for me before I start this massive project, please let me know. It seems like there is no real limit on how detailed your library can be.

JONCAT · « **Reply #1 on:** January 17, 2005, 04:11:37 pm »

In a rush, but how about:

using the genre field or other field in the tag as "Jeff Tweedy" or creating some playlists of just his work.

Playlists would be another idea, although I find they can be a pain.

If it helps, you can define locations on the drive to add to your media lists. You can probably further define multiple variables of ordering by tag fields.

Keep exploring, MC is powerful; you'll find a way to lay out it in a good way.

Jon

BlueGlow · « **Reply #2 on:** January 17, 2005, 04:32:34 pm »

Hi LA,
You're in the right place. I have around 40,000 songs as well and Media Center is the only sensible choice for that many songs. I've been using it for several years now and I still check out other apps to compare but nothing else comes close on features and speed.

I try to keep my songs well organized but so far my collection is not as well tagged as you are describing. MC can handle it, I just haven't had the time to add all that information. That said, I do have a couple of tips based on my experience.

1. Get JRiver Media Center 11. It's the best tool for this job.
MC 10 is much more stable, but 11 is much faster for large libraries.
2. Don't upgrade to every new version of MC 11. Start with the current version
and then upgrade only if you need it to fix a bug. It can be frustrating
when you're fine tuning your setup and normal alpha version bugs and
enhancements interfere with your plans.
2. Get a second hard drive (or two) to back up your music. 300GB should do it
if you have around 40,000 high-quality MP3's.
3. Backup your music before you start on this project.
4. Backup your MC library data often, File/Library/Backup Library.
I've only needed my library backup once for smartlists that got erased
but better safe than sorry.
5. After importing, use smartlists to identify duplicates and then
eliminate them.
6. Get the plugins from KingSparta and @l@n/Alan. (more below)
They're in the official listings, but I usually check the plugin forum
for the lastest updates.
7. Be prepared for a lot of work. Organizing this much music is still a
relatively new thing in computer software. The problems and user needs
are not well understood yet. Very few people have this much music so
all the music library software is biased towards smaller collections
of singles. For example, I don't know of any serious music library that
has 'album' as a unique type which can have attributes of its own.
'Album' is just an attribute of a song so you end up tagging all songs
with all album data.

MC is the best but you'll still be doing lot's of workarounds to get your
library where you want it. Understand that, make the commitment to MC
and you'll almost always be able to make it do what you want.

Hope that helps,
BG

Some more tips:
---------------
Genres:
I try to use very basic genres in the genre field, rock, classical,
blues, jazz, folk, speech, podcast,etc. Then I add songs to playlists to
classify them as vocal, instrumental, fast, children's, holiday, indie,
electronic, workout, punk, african, etc.
This works very well for me. I can create arbitrary song attributes like
'Thanksgiving' and not clutter my genre list. I think others may be using
custom fields and view schemesto accomplish similar things.

Artist/Band Info:
Find atagger in the plugins forum, embrace it, love it.
It is your friend. The developer, @l@n/Alan, is friendly and
remarkably helpful.

Cover Art and much more:
KingSparta is true royalty, can't say enough about him.
Find his plugins and give them a try. He's the most prolific plugin
developer and his plugins supercharge Media Center.

KingSparta · « **Reply #3 on:** January 17, 2005, 06:42:20 pm »

Quote

30,000

is not huge

Larger than most maybe

I have three 200 gig drives, and four 300 gig fire wire drives, they hold alot of mp3's

many people do it dif ways, it depends on the person

i have mine in folders by dates

my music\1940's\1941\
my music\1940's\1942\

etc....

Some Prefer thet standard

My music\Artist\album\artist - songname.mp3

BlueGlow · « **Reply #4 on:** January 17, 2005, 08:32:36 pm »

Quote from: KingSparta on January 17, 2005, 06:42:20 pm

30,000 is not huge
Larger than most maybe

This begs the question, what is a huge library? (new offtopic topic)

On topic, KingSparta, what method do you use to keep organize so many files?
Genre?
Custom fields?
Playlists?

Mastiff · « **Reply #5 on:** January 18, 2005, 03:04:59 am »

You shouldn't ask King how many files he's got...you'll only be shocked!

I have mine sorted by genres (I use pretty broad genres, Country covers bluegrass, roots and so on, rock covers anything of rock that isn't alternative, metal or progressive, pop-disco-dance covers most of the rest. I tend to think in "WAF" - stuff my wife can listen to will be in rock, pop-disco-dance and country. The rest is split up after my preferences. Really, trying to be to precise with genres will drive you crazy. I started that way, but merged a bunch of genres (like disco, pop, dance, adult pop and a few others turned into pop-disco-dance, and hard rock, metal, death metal, speed metal, black metal, doom metal and on and so on got merged into metal).

I have a main library on two 200 gig hard disks, and one full backup on mirrored drives that I use as a carputer. Nice to have a CD changer with more than 3600 CDs in the car!

And of course this is all controlled by NetRemote and my own playlist selecter working with Girder.

himini · « **Reply #6 on:** January 18, 2005, 03:07:50 am »

Quote from: KingSparta on January 17, 2005, 06:42:20 pm

I have three 200 gig drives, and four 300 gig fire wire drives, they hold alot of mp3's

My God, man. You are insane.

Mastiff · « **Reply #7 on:** January 18, 2005, 03:19:00 am »

No. He's King.

Jaqqe · « **Reply #8 on:** January 18, 2005, 03:25:38 am »

Quote from: himini on January 18, 2005, 03:07:50 am

My God, man. You are insane.

Who said they're full ? ;-)

KingSparta · « **Reply #9 on:** January 18, 2005, 05:53:06 am »

Quote from: Jaqqe on January 18, 2005, 03:25:38 am

Who said they're full ? ;-)

they are almost, out of all of them there is aprox 200 gigs free

but two of them are backups of the other two

one of them is a 200gig raid 1 drive so thats not backed up

Mastiff · « **Reply #10 on:** January 18, 2005, 06:03:43 am »

King, I've been meaning to ask you: What happened to your caps? Carpal tunnel syndrome, or something?

linden_arden · « **Reply #11 on:** January 18, 2005, 08:19:57 am »

Quote from: BlueGlow on January 17, 2005, 04:32:34 pm

Hi LA,
You're in the right place. I have around 40,000 songs as well and Media Center is the only sensible choice for that many songs. I've been using it for several years now and I still check out other apps to compare but nothing else comes close on features and speed.

Thanks for the tips. I'm sure I will be utilizing many of your tips. I think I've got the method worked out in my head, I'm just not looking forward to implementing it.

I know all about backups. I have 'lost' my collection a couple of times, but have always managed to get the files back. I am thinking of starting my own hard drive recovery company! I think I have 3 copies of my library floating around right now. Ultimately, it is going to be much more formal: 1) media sever, 2) back-up, and a 3) synchronized copy in my car. I am trying to do three major projects at once (organize library, whole house audio, and carputer) so it is all pretty daunting, but I feel like it is worth it.

Mastiff · « **Reply #12 on:** January 18, 2005, 09:01:05 am »

It's not daunting, just stupid.

I suggest you take one thing at a time, the requirements are so different that you shouldn't do them all at once.

GHammer · « **Reply #13 on:** January 18, 2005, 09:11:31 am »

Quote from: Mastiff on January 18, 2005, 09:01:05 am

It's not daunting, just stupid. I suggest you take one thing at a time, the requirements are so different that you shouldn't do them all at once.

Unless you like redoing them because one project exposes flaws in another. Ha! I do that too often!

KingSparta · « **Reply #14 on:** January 18, 2005, 10:26:07 am »

Quote from: Mastiff on January 18, 2005, 06:03:43 am

King, I've been meaning to ask you: What happened to your caps? Carpal tunnel syndrome, or something?

I'm On Medication

runemail · « **Reply #15 on:** January 18, 2005, 10:36:16 am »

If you write good reviews of all your albums, or import from Amg, and include in your tags. A normal search in mc should bring up all relevant artists and albums, then there is no need for custom fields. (at least not for getting relevant hits).

To do custom fields manual for each track or album on a "huge library", is a never ending task.

KingSparta will probably write a plugin that can do this long before you are finnished anyway.

linden_arden · « **Reply #16 on:** January 18, 2005, 10:50:15 am »

This is an interesting idea. Eventually I want my tags to be incredibly detailed. One wish is to attach lyrics and possibly even guitar tabliture to each song if it is availible. Because I have a lot of live shows I'd like to add the recording information and notes about the show to the tags. Who knows if I will have the patience to do all of this.

I know it is going to be a ton of work, but I guess I just need a really detailed database to reflect how I listen to music. I want a very organic listening experience which basically means I lot of background work ahead of time.

I will be researching these formus a bunch more and checking out these plugins people have been referring to.

BlueGlow · « **Reply #17 on:** January 18, 2005, 12:41:36 pm »

Quote from: linden_arden on January 18, 2005, 08:19:57 am

Ultimately, it is going to be much more formal: 1) media sever, 2) back-up, and a 3) synchronized copy in my car. I am trying to do three major projects at once (organize library, whole house audio, and carputer) so it is all pretty daunting, but I feel like it is worth it.

Good luck with those projects. I think I'll get a few of the UPnP media devices for whole house audio this year and leave it at that. What's the plan for the carputer? Media center on wheels?

Mastiff · « **Reply #18 on:** January 18, 2005, 02:22:05 pm »

Is there another viable solution than MC on wheels?

linden_arden · « **Reply #19 on:** January 18, 2005, 02:50:10 pm »

Quote from: BlueGlow on January 18, 2005, 12:41:36 pm

Good luck with those projects. I think I'll get a few of the UPnP media devices for whole house audio this year and leave it at that. What's the plan for the carputer? Media center on wheels?

www.mp3car.org is the spot for carputer stuff. Some people do some amazing stuff with their's, but mine is just going to be a rolling jukebox.

I've got a VIA EPIA 5000 mini-itx form computer thats small enough to fit under my seat. The computer will just have a hard drive and a wi-fi card. I am going to run a stripped down version of WinXP and, of course, Media Center. It's going to have a 'smart' power supply that interacts with the ignition, which should revent all the major power problems associated with carputers. Audio will go from the computer into my headunit. I plan on controlling it with my Dell Axim X30 Pocket PC via ad hoc wireless connection using Netremote.

I have done some 'dry land' testing and the results are promising. I don't think a lot of carputer users use MC, but I think in combo with Netremote it is going to serve me perfectly.

cfgreen · « **Reply #20 on:** January 19, 2005, 12:58:51 am »

I think the biggest problem I ran into was deciding what I was going to listen to...

So I set up Chart Hits and Picks

AMG Album picks

Styles, Moods (still havent a good clue on how I'll use moods)

Album Themes, Then I shot for Album and Single Grammys

The very Best thing was adding a Artist ;Delimited field

hit_ny · « **Reply #21 on:** January 19, 2005, 02:18:09 am »

Quote from: linden_arden on January 18, 2005, 02:50:10 pm

www.mp3car.org is the spot for carputer stuff. Some people do some amazing stuff with their's, but mine is just going to be a rolling jukebox.

I've got a VIA EPIA 5000 mini-itx form computer thats small enough to fit under my seat. The computer will just have a hard drive and a wi-fi card. I am going to run a stripped down version of WinXP and, of course, Media Center.

I have a query about this when i was looking into getting mp3s in the car. I read in places that vibration and heat in the car would kill the HDs. I guess Mastiff will have some more info on this.

So i opted for a cd player that played mp3s via a cassette adapter. Mind, this was the best solution about 3 yrs back, have not bothered changing since it did the job for me.

As long as you have your music fairly organised, its easy to burn a few cd-rws for a trip. Of course if you're a trucker spending days on end on the road then this might not be the best.

Mastiff · « **Reply #22 on:** January 19, 2005, 07:07:08 am »

That's just bull. Maybe with the old harddisks the hardcore mp3car.org guys use - they pride themselves in using small footprint systems, preferably with anorectic versions of Win98 and WinAmp, or even Linux. The user friendliness of their systems is rather low in my opinion. I have had MC in my carputer for a few years, and no harddisks has been harmed...except for in a very abrupt turn (some people really don't know that they should stop for traffic inside a roundabout...), when the carputer fell down. Killed one drive, but that was still under warranty, so I'm OK...

hit_ny · « **Reply #23 on:** January 19, 2005, 07:47:02 am »

Hmm..thats interesting, you're saying that general vibrations dont cause much effect either. The odd speed-bump/pothole etc. In the sense it has not caused any problems for you.

Its possible you have your shocks on medium to absorb what must be vibrations that a desktop HD spinning at 7200 RPMs or more isn't really supposed to handle. Or do you have more rugged HDs/comp mounted in your car.

I guess hot weather isn't much of an issue in Norway

linden_arden · « **Reply #24 on:** January 19, 2005, 09:17:43 am »

Quote from: hit_ny on January 19, 2005, 02:18:09 am

I have a query about this when i was looking into getting mp3s in the car. I read in places that vibration and heat in the car would kill the HDs. I guess Mastiff will have some more info on this.

So i opted for a cd player that played mp3s via a cassette adapter. Mind, this was the best solution about 3 yrs back, have not bothered changing since it did the job for me.

As long as you have your music fairly organised, its easy to burn a few cd-rws for a trip. Of course if you're a trucker spending days on end on the road then this might not be the best.

If you are interested in carputers at all you really shoud check out www.mp3car.org. It is amazing the stuff that they are doing over there. The general consensus is that most HDs can survive almost anything you'll experience on a normal drive. I think some people have taken them off-road as well. My thinking is I'm as likely to kill a hard drive in my car as in my house, so as long as I keep a good back up of my collection, I'll be in good shape. This isn't just speculation - there are tons of folks that have logged tons of miles with carputers without harddrive issues. Some pepole use laptop drives because they are supposed to be more 'rugged', but emprical evidence suggests that desktop harddrives are actually more rugged. If you check out the link and search the forum for hard drives you'll see loads of discussion about the issue.

I currently have a head unit which plays mp3 CDs. It has been a very good solution, but it never fails that I expand my music collection and my mp3 CDs just can't keep up with my tastes. I never thought having 100 songs on a CD would be inadequate. The navigation is just not what I am looking for either. I listen to music in a very fluid way and I am constantly skipping to distant spectrums of my collection, so compilations just don't cut it.

Again, I don't want to hijack this board to talk about carputers, so if you are curious at all about them check out the site I linked. It has been a really cool project for me to get into and I keep learning more and more about computers, even though I was pretty savy to begin with.

I will warn you that this can be a pretty serious project, time and resource-wise. You can do it on a budget, but it is tough. The awesome part is that it is a very free-form project. There are so many different paths you can take and so many 'problems' to solve that you can really be creative. The forums I linked have a decent FAQ as well as a 'newbie' board and people are always willing to help.

hit_ny · « **Reply #25 on:** January 19, 2005, 10:04:00 am »

Thanks for the informative post, i must admit i'm amazed. Will check out that link.

One of the resons i opted for the ugly/low-tech solution is that kitting up a car with audio adds nothing to the resale value of the car. Unless you find a buyer that shares the same interests which i did not want to bother with. I needed something that was easy to add and remove and of course hide, (not wanting the extra attention when the car was parked etc). I listen to a lot of mix music so i dont tend to skip tracks that often and on the road, the less distractions the better.

Getting back to managing music. I'm running at 200Gb+ of music. I rate each track and average the track ratings into an album score.

I have 3 bands 2.50-3.00 is bronze, 3.00-3.50 is sliver and gold is 3.50 and above. Another way of looking at the 3 bands is to think of them as 3,4, or 5 star albums.

I did this as i wanted to know how much better a album was vs another album by the same artist or another in a series. My ratings are highly subjective based on how much i like a track, has nothing to do with sales or chart positions etc. The idea being to customise MC to what i like.

This is the hard part, it takes forever to rate stuff and be in a decent enough mood not to affect ratings which invariably happens. I tend to review albums now and then and see if they still do the trick , sometimes i promote/demote tracks etc.

I thought i could isolate the albums that scored less than 2.5 (avg) and get rid of the offending tracks. The space savings looked tempting, but as i go into ellimination mode i find its getting harder to delete tracks as they are not so bad for some, its just that they were not as good when compared to a random sample out of my library. FOr osme i deleted tracks for others i kept the albums intact. Its easier to do this if the tracks are not mixed in, the mixed ones are left as is.

All is on local HDs, that i mirror on a regular basis.

Mastiff · « **Reply #26 on:** January 19, 2005, 05:59:58 pm »

Hit_ny, just to finish off your question, this is in a Chevy Suburban, so I have never really experienced a sudden and violent bump. Thos shocks takes the pot holes out of any road... And I have my full carputer inside a metal suitcase, so I can take it out in a minute or so. Ain't nobody gonna get that one if they buy my car!

rhom · « **Reply #27 on:** January 21, 2005, 04:11:20 pm »

i use the Artists field (a user field i created), it's a list, semi-colon delimited, standard field

for bands like lisa lisa & cult jam eg;

Artist = Lisa Lisa & Cult Jam
Artists = Lisa Lisa;Cult Jam

this way you just modify the view scheme (or create a new one) to show the Artists field (grouped if required) instead of the Artist field and it breaks it down into each person listed in the Artists field quite nicely.

you can't modify the Artist field to accept multiple values which is why i had to create a new field, othwise i would have just used the standard Album Artist and Artist fields.

Iridescence · « **Reply #28 on:** January 21, 2005, 04:40:12 pm »

Quote from: BlueGlow on January 17, 2005, 04:32:34 pm

.

.
5. After importing, use smartlists to identify duplicates and then
eliminate them.

Newbie question here. I have a large library and have a lot of duplicate files. What is a "smartlist" will these remove the songs from the HD and not just from the library?

Thanks

INTERACT FORUM

Author Topic: Managing a huge library (Read 5304 times)

JONCAT