INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Praise - 380,000 files  (Read 3425 times)

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Praise - 380,000 files
« on: October 27, 2006, 04:06:27 pm »

MC 12 is now managing 380,000 files in my library.   3TB of files!
It is running FAIRLY well, and I have to give praise.  There isn't a single app that I have used that can deal with a library this large, handle all the tagging, thumbnails, renaming, rules, playlists, etc that I use. 

The only problem I have is with the IsMissing command ... it looks for EVERY FILE which can take hours.   So I avoid that one like the plague!

Performs well on networked drives, slower performance with large files (ie VIDEO) than others, but very acceptable!

Now, to SHARE that library across all machines in my home .... still working on that one!  (I'm mapping the library and the thumbnail directories to network drives and making the database use its own network drive mappings to import the files.  So far it works, just no updates to the files like # Plays because the file is always open somewhere else ... but I can live with that)

Again, a big hoorah for being able to manage my crap, and kudos on a very well-done interface!

Now, what song was I looking for again?   I know it's here somewhere ;)

darichman

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1362
Re: Praise - and a holy crap!
« Reply #1 on: October 27, 2006, 08:34:20 pm »

3TB? Wow. I have over 100000 files in my library and it's only 300GB! Guess I must have a lot of piccies :)

Quote
Now, to SHARE that library across all machines in my home .... still working on that one!  (I'm mapping the library and the thumbnail directories to network drives and making the database use its own network drive mappings to import the files.  So far it works, just no updates to the files like # Plays because the file is always open somewhere else ... but I can live with that)

I'd really like to do this as well. Any chance someone could guide me through it? I've never mapped network drives before.
Logged

JONCAT

  • Guest
Re: Praise - and a holy crap!
« Reply #2 on: October 27, 2006, 09:52:38 pm »

Sounds like you got some lossless in there bro.

MC12 is impressing me as well.

I am thinking about trying the library share again. I just setup an Apple Airport Express and the laptop is using MC Library Server captured by Airfoil to send to the Airport>Klipsch 2.1 in the living room.
I like that seeking has been added to library server but I still get dropouts which I was not getting when trying my mapped drives. I am not converting audio on the laptop either so who knows...

one thing about sharing the library directory/thumbnails: does all the main PCs storage need to be added as network drives and NOT mapped. If I map the drives on my main PC, my laptop is not going to find them once the original database is loaded. Network shares and mapped drives do not provide the same syntax for file locations.
So I'm guessing we have to add all the drives on the host PC through the network itself so the database points at a location all the PCs on the network can use. I would kind of like to be able to edit files on the laptop if I need to, although it sucks we can't have write access, well unless we reboot I guess.

Correct?

v/r
DC

Logged

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: Praise - 380,000 files
« Reply #3 on: October 28, 2006, 12:20:51 am »

Where did you source all those 300K files?   Are they all from CD?  If they're lossless then I'd guess they are, right?  I'm at only 10K files and that's 250GB.  They're all from CD, though.  MC handles it perfectly on my fast computer.  Searching is instant when the thumbnails are fully cached otherwise that slows it way down.

Addition: 25K files total including pictures and video.
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: Praise - 380,000 files
« Reply #4 on: October 28, 2006, 06:25:04 am »

The only problem I have is with the IsMissing command ... it looks for EVERY FILE which can take hours.   So I avoid that one like the plague!
Very useful command.

Hours does not sound right.

Given the amount you have i'd think 10+ minutes would be enough. (unless you find it takes longer only with MC 12).
Logged

JONCAT

  • Guest
Re: Praise - 380,000 files
« Reply #5 on: October 28, 2006, 09:04:36 am »

I tried adding all my local drives through the network, and I get doubles without trying any filtering.

Hoping someone can chime in regarding my earlier post.

How do we share a library directory so that other PCs on the net can have correct access to the files.

drosoph: how are you doing this?

DC
Logged

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: Praise - 380,000 files
« Reply #6 on: October 28, 2006, 09:17:57 am »

I have "IsMissing" as the leftmost pane of my default audio view.  You say it's taking "hours"?  It shouldn't even take "minutes".  I've got ~20,000 files and when MC is waking up "IsMissing" will take 8 secs max, and when switching views it's more like 2 secs.  And this is on a single-proc laptop with half the library on firewire. 
Logged

4BYE

  • Regular Member
  • Galactic Citizen
  • ****
  • Posts: 416
Re: Praise - 380,000 files
« Reply #7 on: October 30, 2006, 04:59:48 am »

Now, to SHARE that library across all machines in my home .... still working on that one!  (I'm mapping the library and the thumbnail directories to network drives and making the database use its own network drive mappings to import the files.  So far it works, just no updates to the files like # Plays because the file is always open somewhere else ... but I can live with that)


I'm dealing too with a problem of sharing through the house with several machines. Audio works great and images are shown too. My big issue is my DVD/Video files. I cannot use library server with it and have never got an answer on several questions how to sove this, or even if this can be solved. It would be too much work to have each machine have it's own library, also because or the reasons you give. Now if have to go to windows explorer and find the file I want to watch and then have it played with Power DVD. This is not what I expected ofcourse. I added a lot of RAW images from my Canon d60 and 20d and it has a lot of problems to show thumbs through the library server. For the time being I'll use BreezeBrowser Pro to have some slideshows because it's really fast.

Regards,


Theo
Logged

Mr ChriZ

  • Citizen of the Universe
  • *****
  • Posts: 4375
  • :-D
Re: Praise - 380,000 files
« Reply #8 on: October 30, 2006, 07:17:48 am »

Having similar problems with photos over Library Server.
MC downloads the entire image instead of the thumbnails, meaning
my machine is brought to a stand still serving them out, and the clients
take for ever to see the images...

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Praise - 380,000 files
« Reply #9 on: October 30, 2006, 07:44:11 am »

Library Server does not serve thumbnails. Thumbnails are handled like a cache memory in a local database. A client builds its own thumbnails and often this means that the complete file must be "streamed" before the client can create the thumbnail.

I would like to have a fast "thumbnails totally off" mode as an option. Only the actual cover art or image file would then show up on MC's display during audio track or image playback. All browsing would happen in "text only" mode.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Mr ChriZ

  • Citizen of the Universe
  • *****
  • Posts: 4375
  • :-D
Re: Praise - 380,000 files
« Reply #10 on: October 30, 2006, 08:09:05 am »

I guess it just looked that way because the cover arts are comparatively
small files, hence they go really fast.

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: Praise - 380,000 files
« Reply #11 on: October 30, 2006, 09:32:21 am »

Since it is a MC to MC connection, the client could be aware of receiving thumbnails only.  Then, MC Server could send a very small thumbnail with the required thumbnail, noting that the sent data is merely a thumbnail and not the complete file.  It would be like converting music before sending but it would convert images before sending.  If I recall, my thumbnail file for medium sized thumbnails is around 40MB, much less than 1000MB for the full jpg's.
Logged

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Re: Praise - 380,000 files
« Reply #12 on: November 01, 2006, 10:20:48 am »

I have "IsMissing" as the leftmost pane of my default audio view.  You say it's taking "hours"?  It shouldn't even take "minutes".  I've got ~20,000 files and when MC is waking up "IsMissing" will take 8 secs max, and when switching views it's more like 2 secs.  And this is on a single-proc laptop with half the library on firewire. 
What is your syntax exactly?    I use [=IsMissing([Filename])]=1

Am I doing this correct, and you have to remember, it is looking for missing files on network drives as well!

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Re: Praise - 380,000 files
« Reply #13 on: November 01, 2006, 10:23:24 am »

Where did you source all those 300K files?   Are they all from CD?  If they're lossless then I'd guess they are, right?  I'm at only 10K files and that's 250GB.  They're all from CD, though.  MC handles it perfectly on my fast computer.  Searching is instant when the thumbnails are fully cached otherwise that slows it way down.

Addition: 25K files total including pictures and video.
My source is private.   Many are from CDs, others "not".  I do not run lossless, I try to keep them at 160kbs ABR.    I do have a lot of video as well (movies, tv, etc).   I am also a photographer with over 71,000 7MP Photos in my library.  Hope that explains the size.

drosoph

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 661
  • TiVo-aholic
Re: Praise - 380,000 files
« Reply #14 on: November 01, 2006, 10:27:21 am »

I tried adding all my local drives through the network, and I get doubles without trying any filtering.
Hoping someone can chime in regarding my earlier post.
How do we share a library directory so that other PCs on the net can have correct access to the files.
drosoph: how are you doing this?
Hey Doc,

This is simple, and without all the fuss you're thinking about.    Just create a network MAPPED Directory to the files that you want (I recommend creating accounts for users on other machines, or using the same user/login on all machines in your home).    Once you've created this share, it is VERY easy!

Original Files are on D:\Audio\, Create a map to this directory called \\MachineName\Audio
Do a Find & Replace on FILENAME to replace all "D:\Audio\" with "\\MachineName\Audio\"
That should be it.

Now, map your LIBRARY directory (wherever you have stored it -- docs and settings, or wherever) and create a NEW library on the remote machine and point it to that library (\\MachineName\MCLibrary\) .... No Media Server involved ... it opens instantly ...

Let me know if you have any problems!

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: Praise - 380,000 files
« Reply #15 on: November 01, 2006, 01:38:40 pm »

drosoph--

I can't write an expression to save my miserable life--I have trouble spelling "IBM" reliably.  This was written for me by marko, who thankfully takes pity on me:

if(IsMissing([Filename]),no,yes)


And yes, I did forget that you were looking for these files across a network.  I wonder if that should be radically different from a firewire drive.  Aren't gigabit and firewire roughly equivilent in throughput?

Logged
Pages: [1]   Go Up