INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: How to compact the library database?  (Read 4791 times)

hsc

  • Regular Member
  • Recent member
  • *
  • Posts: 31
  • Change this by choosing profile
How to compact the library database?
« on: December 26, 2007, 06:42:03 am »

Hi,

I'm looking for a method to compact (shrink) the library DB. Is there an easy way to do this?
  Some background: I splitted my main library into 2 separate libs, going from one 120k files lib to one with about 90k (music) and one with 30k (audio books), to speed up library download and performance for my 800Mhz P3 client. I did the splitt by duplicating the original lib and removing the not wanted library entries (90k music files on the Audio Book lib, and 30k audio book files on the music lib). But in the end the resulting libraries were not that much smaller. The original lib had about 13mb (compressed, backup), the new ones had about 12mb each. I also tried to remove the references to the removed files from the database, until the total number of entries reported matched to what I would expect. Still the lib sizes do not shrink.
  In the end I did a reimport of all the audio book files into a new/empty library and the resulting size was about 3mb, but of course this is no solution, as I would lose some of the metadata not included in the file tags.
  BTW: I do have MC 12.0400 installed.
Thanks for any help on this topic,

Horst
Logged

pillshovel

  • Junior Woodchuck
  • **
  • Posts: 78
Re: How to compact the library database?
« Reply #1 on: November 30, 2008, 09:54:00 am »

Sorry to bring up such an old topic, but I have been looking for a solution to this issue as well.  I stream my music at work and as the library database grows, it takes longer and longer to load over DSL.  I can delete the entire library and start fresh and get it back to a reasonable size, but I don't like doing this as I have to go back and edit certain settings every time.  Is there any way to compact the library through MC itself?

I've done some searching and found a lot of info about  database view schemes to see the deleted files database, but isn't there some way to just do a simple purge?  I don't have the exclude files that were deleted checked and I have the fix broken links option checked.  It seems to me that with these two options set as I have there would be no reason for a deleted file database anyway.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42376
  • Shoes gone again!
Re: How to compact the library database?
« Reply #2 on: November 30, 2008, 10:57:08 am »

There's no database compact function, although the percentage of wasted space should always stay relatively low.

You can see all the audio records by creating a smartlist like this:
[Media Type]=[Audio] ~d=a ~sort=[Filename]
Logged
Matt Ashland, JRiver Media Center

pillshovel

  • Junior Woodchuck
  • **
  • Posts: 78
Re: How to compact the library database?
« Reply #3 on: November 30, 2008, 11:55:53 am »

I've noticed that the database doubles in size when I rename all the music in my Auto-Import folder externally.  I assume this is because it is keeping track of the old filenames too.  Is there a simple way of preventing at least that circumstance?
Logged

Lasse_Lus

  • Citizen of the Universe
  • *****
  • Posts: 999
Re: How to compact the library database?
« Reply #4 on: November 30, 2008, 12:46:42 pm »

my vote for some database tools  :P
Logged
MT5FR

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42376
  • Shoes gone again!
Re: How to compact the library database?
« Reply #5 on: November 30, 2008, 06:30:56 pm »

I've noticed that the database doubles in size when I rename all the music in my Auto-Import folder externally.  I assume this is because it is keeping track of the old filenames too.  Is there a simple way of preventing at least that circumstance?

Media Center has powerful internal renaming tools, so you might just do the renaming inside the program.

And since duplicate data is pooled (i.e. an artist, name, etc. is never in memory or on disk twice), the database should not be increasing in size much in this situation.
Logged
Matt Ashland, JRiver Media Center

Lasse_Lus

  • Citizen of the Universe
  • *****
  • Posts: 999
Re: How to compact the library database?
« Reply #6 on: December 12, 2008, 03:52:35 am »

..although the percentage of wasted space should always stay relatively low.

but matt, when i clean (run manually autoimport) my big "text library" i have still have a lot of waste after. MC removes the files
that i have removed but not the waste/data, like mediafiles.jmd is cleaned out nicley, but the text field is like 33 mb

the only way to keep the library cleen and neat (as i have found out) is to keep a clone of a fresh "untouched" library and replace these fields with the ones you want to clean...i dont know if there is a difference with audio and text..but i sure would appreciate a compacting tool  :)

also when MC runs the autoimport i receive the message "4384: Import: RemoveBrokenLinks: Too many broken links -- probably an incorrect library, etc." in the log so it stops..but its nothing wrong with my library i do have removed a lot of files but its not an incorrect library..

when i run the autoimport manually there is not such a message !

can this be changed ? or could you give me actual number when the autoimport fails

you may ask why i have these "problems" the answer is that i use MC not only for music but also for some textfiles projects  :)

regards Lars

Logged
MT5FR

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: How to compact the library database?
« Reply #7 on: December 12, 2008, 04:37:26 am »

You could check my posts about "deleted files database". The Auto-import feature is supposed to be able to clear the deleted files from the hidden library section if all maintaining options are enabled and if the files do not physically exist on the disk anymore, but with my instructions you can actually check how well the the automatic cleaning works for you and do it manually if needed.

As long as the deleted files are included in the library all old field data is preserved in the field specific .jmd files. After you have got rid of the deleted files you can start cleaning the individual database fields. MC does not rewrite or save changes to the field specific .jmd files before you actually change or add a field value. In order to compact the "field (text).jmd" file you would need to change the "Text" field value in one of the files. For instance, you could import a new short text file.

EDIT

Here is a link to my post, which contains further links: http://yabb.jriver.com/interact/index.php?topic=46324.msg319087#msg319087
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Lasse_Lus

  • Citizen of the Universe
  • *****
  • Posts: 999
Re: How to compact the library database?
« Reply #8 on: December 12, 2008, 06:07:37 am »

thanks a lot Alex..i did see this one time but i could not reproduce it..but as always you give a perfect explanation  :) i think i missed it because i use the "silent import command" and the "you would need to change the "Text" field value in one of the files. does not seem to affect the database size when using "silent"..
anyway you helped me a lot and gave me some new inputs that i hopefully can use  :)

big thanks

Lars
Logged
MT5FR

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: How to compact the library database?
« Reply #9 on: December 15, 2008, 10:54:18 am »

the only way to keep the library cleen and neat (as i have found out) is to keep a clone of a fresh "untouched" library and replace these fields with the ones you want to clean...i dont know if there is a difference with audio and text..but i sure would appreciate a compacting tool  :)

Did Alex's instructions obsolete your above workaround  ?

Its a matter of 2 things :
- not enabling the 'ignore files previously removed form library' option in the import dialog.

- and using the 'DB delete' (prefer it be named DB Deleted) smartlist described by Alex here

had made these smartlists when he posted back in 2005 (circa 11.0, was it) but had completely forgotten about them, turns out i only had 400 odd 'deleted' files to remove since  :)
Logged

Lasse_Lus

  • Citizen of the Universe
  • *****
  • Posts: 999
Re: How to compact the library database?
« Reply #10 on: December 15, 2008, 12:08:06 pm »

Did Alex's instructions obsolete your above workaround  ?

in theory yes i have everything automated on my server so i have to do same changes (when i have the time)


Did Alex's instructions obsolete your above workaround  ?

Its a matter of 2 things :
- not enabling the 'ignore files previously removed form library' option in the import dialog.

i dont know about that one when using "silent import", i import everything from a created mpl file.

- and using the 'DB delete' (prefer it be named DB Deleted) smartlist described by Alex here

had made these smartlists when he posted back in 2005 (circa 11.0, was it) but had completely forgotten about them, turns out i only had 400 odd 'deleted' files to remove since  :)

i have a view for "garbage" that i have used..but i was looking for something that could be automated..i mean the ultimate solution what i think is that you could send a silent MC12.exe/ compact DB command   

Logged
MT5FR
Pages: [1]   Go Up