Topic: 'Saving Database' excessively, resource hog (Read 10373 times)

LisaRCT · « **on:** November 22, 2003, 11:01:55 pm »

I have taken to using a small library, so as to prevent the excessively frequent 'Saving Database' message windows I get with my 20,000 file library, which slows/stalls/freezes my system (depending on what is running) until the save is finished.
Unfortunately while this makes a 'band-aid' fix it leaves my library mostly inaccessable without changing libraries and again putting up with the frequent system stalling saves.

SO, is there a way to STOP the database from saving? Can I turn OFF the database statistics and other minor saves, leaving the ability to Manually save when needed? Perhaps only turning on the save when tagging or importing (manually or otherwise)?

LisaRCT · « **Reply #1 on:** November 23, 2003, 02:01:06 pm »

OK, I just imported about 200 mp3 files and I want to do some tagging before I finish my import routine.
I sort the files so I can do some mass-tagging . . . I select the group I wish to tag/edit the Artist, right-click . . FREEZE!
up pops "WAITING - Saving Database" so I sit.
Finally that finishes and my right-click menu appears, I edit my Artist and move over a column to edit Genre and FREEZE!
"Please WAIT - Saving Tag Changes"

Now why do I need to save the tag changes when I am not yet Finished Tagging??

So, I right click on the Genre - click on 'Edit Selected Item' - FREEZE!! "Please Wait - Saving Database"

I don't need the database saved yet, I am still making changes and can save them all when I am finished.

Why does it choose to save the database while I am in the already 2 layers deep in the right-click menu (thus freezing the menu)?

Why does it lose focus? If I try to click on a menu choice AS the "Wait-Saving" dialog freezes my screen, when it asks me to confirm the save I must first click on the Windows toolbar for the confirm window to become in focus so I can click it.

This all makes tagging and such real bothersome. I prefer to finish my tags/edits/etc. then have it save when I am done so I can walk away and do something else while it saves.

LonWar · « **Reply #2 on:** November 23, 2003, 05:33:53 pm »

Ya, I remeber a post a while back talking about this. They said that this was the way it will work.

I am still hoping that we can be allowed to removing the auto saving...

JimH · « **Reply #3 on:** November 23, 2003, 05:59:18 pm »

Lisa,
I think we determined that it was all the lyrics you had.

Jim

LisaRCT · « **Reply #4 on:** November 23, 2003, 06:27:40 pm »

ummm, Jim . . . .
I had removed ALL my lyrics, ALL my artist Bio's, etc. before the database save timing or whatever was changed. The change that Matt made got rid of my lag problem but replaced it with this new problem.

KingSparta · « **Reply #5 on:** November 23, 2003, 06:54:26 pm »

>> SO, is there a way to STOP the database from saving?
a save db on exit would be fine with me for large db's over x in size, no switch\option or check box needed

how about you?

LisaRCT · « **Reply #6 on:** November 23, 2003, 07:04:20 pm »

Heck I'll try anything (almost)

Besides, I liked having the lyrics so my daughter can play sing-along, or for my g/f learning to play the songs on her guitar.
And the bio's were handy for finding names of a band's members or often other band's an artist has been in before.

jleerigby · « **Reply #7 on:** November 24, 2003, 02:36:35 am »

Quote from: KingSparta on November 23, 2003, 06:54:26 pm

>> SO, is there a way to STOP the database from saving?
a save db on exit would be fine with me for large db's over x in size, no switch\option or check box needed

how about you?

This would be a little dangerous. If you had a media core crash who knows how much data you would lose. Several hours worth probably.

Lisa - I'd be interested to see your system info. My library is about the same size with 75% lyrics saved and I never notice MC saving the database.

If your PC is on the slow side then it sounds like you now have a good reason to upgrade.

LisaRCT · « **Reply #8 on:** November 24, 2003, 08:31:33 am »

Quote from: JLee on November 24, 2003, 02:36:35 am

If your PC is on the slow side then it sounds like you now have a good reason to upgrade.

I don't think so.

Media Center Registered 9.1.308 -- C:\Program Files\J River\Media Center\

Microsoft Windows 2000 Workstation 5.0 Service Pack 4 (Build 2195)
AMD Athlon 1734 MHz MMX / Memory: Total - 1048 MB, Free - 524 MB

Internet Explorer: 6.0.2800.1106 / ComCtl32.dll: 5.81 / Shlwapi.dll: 6.00.2800.1276 / Shell32.dll: 5.00.3700.6705 / wnaspi32.dll: 4.57 (1008) , ASPI for Win32 (95/NT) DLL, Copyright © 1989-1997 Adaptec, Inc. / Aspi32.sys: 4.57 (1008)

Ripping / Drive D: Copy mode:ModeSecure CD Type:Auto Read speed:Max
Drive E: Copy mode:ModeSecure CD Type:Auto Read speed:Max
Digital playback: Yes / Use YADB: Yes / Get cover art: No / Calc replay gain: Yes / Copy volume: 32767
Eject after ripping: No / Play sound after ripping: No

Burning / Drive D: PLEXTOR CD-R PX-W4012A Addr: 0:1:0 Speed:40 MaxSpeed:40 BurnProof:Yes
Test mode: No / Eject after writing: Yes / Direct decoding: Yes / Write CD-Text: No
Use playback settings: No / Normalization: None

LisaRCT · « **Reply #9 on:** November 24, 2003, 08:38:23 am »

Quote from: JLee on November 24, 2003, 02:36:35 am

Quote from: KingSparta on November 23, 2003, 06:54:26 pm
>> SO, is there a way to STOP the database from saving?
a save db on exit would be fine with me for large db's over x in size, no switch\option or check box needed how about you?
This would be a little dangerous. If you had a media core crash who knows how much data you would lose. Several hours worth probably.

Which is why I suggested a Manual save Option . . . . which I'd use immediately after I finished tagging or editing each batch of files. Thus I would choose how much work to risk loosing just like I do when writting in my word processor. That would not risk 'hours of work', and if it did it had been my choice to take the risk not to do a save.

LisaRCT · « **Reply #10 on:** November 24, 2003, 09:48:42 am »

There is an already option which says "Update Tags when file info Changes"

However, if I uncheck this, I cannot even save tags manually . . . . which seems to be contrary.

I do not want to have MC save the changes each time they occur, yet I want to be able to save them manually.

I expect this would help my problem some, at least with the Tag updates if not with the database saves.

nila · « **Reply #11 on:** November 24, 2003, 10:04:10 am »

I know JRiver REALLY doesn't want an 'Apply' button but a button that could be toggled on and off for tagging would REALLY work great.

We want to change a bunch of files - we click it - make ALL the changes we want, as many times as we want - then press it again and it updates all the files we changed with ALL the tag changes we made.

How about this? - You make it work like it does not as default - add an extra button that 'disables updates' - make the button hidden as default so it doesn't negatively effect anyone - for those of us that want it it's there - we press it and it disables the tag updates - we can then do ALL our updates - when we finish we press it again and MC goes back to normal mode and updates all our tags with all the changes that were made while it was switched off??

That'd work great - wouldn't effect ANYONE negatively - but would be a GREAT way for the more power users that wanted it (or for those with big libraries!)

Would that be a workable compromise?

LisaRCT · « **Reply #12 on:** November 24, 2003, 10:54:48 am »

As far as I am concerned that sounds like a fairly good solution Nila. I am open and willing to try anything.
I really feel there is a solution somewhere (aside from constantly buying the latest cutting-edge system every 6-12 months), and I know I cannot continue to use my media as I like with things as they are now (on my system anyway).

KingSparta · « **Reply #13 on:** November 24, 2003, 10:58:14 am »

and or some programs have a time you can save changes so after 15 mins all changes that need saved can be saved.

I would prefer a save on exit with a manual save button.

nila · « **Reply #14 on:** November 24, 2003, 12:06:17 pm »

Save on exit just has two downfalls thou - one - ALOT of work can be lost easily - especially if people shutdown windows with MC still running - I'd imagine that'd cause it all to be lost.

Also - it really slows down the shutdown time which is a drag - MC already takes awhile to close!

LisaRCT · « **Reply #15 on:** November 24, 2003, 12:28:28 pm »

I don't particularly like 'save on shutdown' either for the same reasons that Nila lists.

I want to be able to import sereral hundred songs, edit my tag info, then have it save when I am done so I can walk away while it works.

I also do not want the database updating / saving every few minutes stalling the interface while I am trying to select songs to play. That too can be saved manually and at exit without too much risk in my case.

LisaRCT · « **Reply #16 on:** November 25, 2003, 10:33:03 am »

shAf · « **Reply #17 on:** November 25, 2003, 12:43:23 pm »

Lisa ... it might be interesting to know the file size of your database. I have near as many tracks, pretty much simply tagged, and I've never noticed MC saving my db ... as if it happened in the blink of an eye. Any chance your relatively large file is fragmented over your relatively full drive?

fex · « **Reply #18 on:** November 25, 2003, 03:03:57 pm »

Quote from: shAf on November 25, 2003, 12:43:23 pm

Lisa ... it might be interesting to know the file size of your database. I have near as many tracks, pretty much simply tagged, and I've never noticed MC saving my db ... as if it happened in the blink of an eye...

Same here with about 19'000 tracks and around 1'000 images.

Quote

Media Center Registered 9.1.307 -- C:\Programme\Media Center\

Microsoft Windows XP Workstation 5.1 Service Pack 1 (Build 2600)
Intel Pentium 4 1983 MHz MMX / Memory: Total - 523 MB, Free - 219 MB

LisaRCT · « **Reply #19 on:** November 25, 2003, 03:58:37 pm »

I have about 20,000 music files, about 60% mp3 and the rest ape files.
The drive has been defragged and the files occupy about 140 GB out of a 200 GB drive. This is an ATA133 drive with 8mb cache.
I had all my files fixed up with coverart, lyrics, artist bio's, , but even eliminating these did not help.

xen-uno · « **Reply #20 on:** November 25, 2003, 04:28:17 pm »

Could it be related to the Indexing Service?

From Win Help...
------
Using Indexing Service
Indexing Service creates indexes of the contents and properties of documents on your local hard drive and on shared network drives. You can also control what information is included in the indexes. Indexing Service is designed to run continuously and requires little, if any, maintenance.

Open Control Panel>Administrative Tools>Computer Management (Local).
In the console tree, click Indexing Service (under Services & Applications).
-----

See if your media drive/directory is in the catalog. If it is then change the "Include in Catalog" yes/no value to the opposite of what it is now and retest with MC.

10^-27

fex · « **Reply #21 on:** November 25, 2003, 04:32:46 pm »

Quote from: LisaRCT on November 25, 2003, 03:58:37 pm

I have about 20,000 music files, about 60% mp3 and the rest ape files.
The drive has been defragged and the files occupy about 140 GB out of a 200 GB drive. This is an ATA133 drive with 8mb cache.
I had all my files fixed up with coverart, lyrics, artist bio's, , but even eliminating these did not help.

All my 19'000 music files are mp3 (various bitrate), no ape files. About 90 GB without lyrics or artist bio's, but all of them with internal cover art. Drive is an external 200 GB drive (8 MB cache, firewire). You use about 33% more GB. Could this be the reason? King Sparta, any comment?

Update: No indexing.

JimH · « **Reply #22 on:** November 25, 2003, 04:43:47 pm »

I think the problem is related to the size of the library files (jmd files) and not the size of the music collection. Disk speed would also be a factor.

fex · « **Reply #23 on:** November 25, 2003, 04:56:20 pm »

mediafiles.jmd = 5.72 MB
Disk speed is 7200 RPM

KingSparta · « **Reply #24 on:** November 25, 2003, 04:56:26 pm »

Quote

King Sparta, any comment?

What JimH Said.

NoCodeUK · « **Reply #25 on:** November 25, 2003, 05:27:47 pm »

Another idea...how about if it did a background save like MS Word does say every 15 minutes by default but user definable at which time any changes made to the library since the last save get updated to the tags...

Adam

LisaRCT · « **Reply #26 on:** November 25, 2003, 07:32:25 pm »

Quote from: JimH on November 25, 2003, 04:43:47 pm

I think the problem is related to the size of the library files (jmd files) and not the size of the music collection. Disk speed would also be a factor.

200GB 7200RPM ATA133 - so it is not that.

What are the factors affecting Database size (other than number of files)??

JimH · « **Reply #27 on:** November 25, 2003, 07:35:36 pm »

How large are the jmd files you have in the MC program directory? (If that's where they are stored.)

LisaRCT · « **Reply #28 on:** November 25, 2003, 07:55:19 pm »

only 107 MB

KingSparta · « **Reply #29 on:** November 25, 2003, 07:55:32 pm »

mediafiles.jmd 150,785 megs

LisaRCT · « **Reply #30 on:** November 25, 2003, 08:31:17 pm »

It is not just the size of the database and how long it takes to save . . . .
It is also the Frequency of the saves!!!

What can be trimmed out of the database? What cannot?

I could care less about when a song was imported or played last, but I expect there are Smartlists and probably other functions which rely on this info.

Is there a way that some things can be rearrainged?
For instance; If I have 12 songs from an album, there is No Need to save the artist bio 12 times, like the cover-art, only save a pointer to the SINGLE INSTANCE of the bio . . . .
Do Not save the Artist-Bio 'IN' the file tags . . . . place a pointer in the tags and store the bio in a seperate folder (like cover-art is done). Same with the Lyrics . . . point to them and store them seperately.
That would possibly allow the lyrics, bio's, and cover-art to be 'turned-off' thus speeding things up when these features are not currently being used.

Also, what determines WHEN a database save is necessary?

nila · « **Reply #31 on:** November 26, 2003, 03:50:56 am »

56.2 Mb.

Would definitely love some more control over the saves.

This effected me a LOT when I started trying to tag my pictures. The constant updates were making it take AGES (have since stopped trying to tag the pictures)

LisaRCT · « **Reply #32 on:** November 26, 2003, 08:40:28 am »

I can get my mediafiles.jmd file down to under 5mb IF I strip everything out of the tags other than: Artist, Album, Track #, Name, Genre, and whatever automatic tags which cannot be manually edited/cleared. This gets rid of any noticable delays and saves.

So, I can choose between a small library with great detail, OR I can have a large library with no extra details . . . . $:-\$
Does that choice work for you Nila? How about you King? Me either.

Or like I said, there can be some controls about how/when the database is saved. Also, my suggestions about POINTING to the Bio/lyrics/etc like is now done with CoverArt.

KingSparta · « **Reply #33 on:** November 26, 2003, 10:22:41 am »

well what would be better would be that the DB had direct access to the fields and only that field would be updated and not the whole 150+meg database.

I would like more control, yes, for sure

LisaRCT · « **Reply #34 on:** November 26, 2003, 11:24:43 am »

I took my backup drive, loaded it as my library and stripped all the frills trimming the DB down to 5MB . . . MC9.1 is fast & crisp.

As soon as I switch to my main drive (same library contents, except containing all the lyrics, bio's, etc / but with it's own library files & DB) MC is S -- L -- O -- W even if I am not doing anything in the library. Simply playing an internet radio stream I can see a MAJOR decrease in performance.

Now I understant the database jumped from 5mb to over 100mb,
BUT . . . . why is the database being saved so frequently when I am not making any changes??!??!??!?

KingSparta · « **Reply #35 on:** November 26, 2003, 11:28:37 am »

Quote

why is the database being saved so frequently when I am not making any changes??!??!??!?

I think it is updating the Plays\Played count

LisaRCT · « **Reply #36 on:** November 26, 2003, 11:34:14 am »

Quote from: KingSparta on November 26, 2003, 11:28:37 am

Quote
why is the database being saved so frequently when I am not making any changes??!??!??!?
I think it is updating the Plays\Played count

Which I need like a hole in my head

I'd rather have more power for other uses myself.

KingSparta · « **Reply #37 on:** November 26, 2003, 02:02:48 pm »

Quote

Which I need like a hole in my head

sometimes it is nice to have.

maybe Matt cut this out of the newest build if, so this would be a good thing.

shAf · « **Reply #38 on:** November 27, 2003, 06:52:36 am »

I happen to use the "last played" feature a lot. When I select a genre, the tracks are sorted according to "last played", which allows me to simply play a random selection of what hasn't been played recently.

Even with this feature enabled, and updating the list on the fly and continuously, I do not experience the interference that LisaRCT is describing (my library having a similar number of tracks, albeit with very few APE files).

LisaRCT · « **Reply #39 on:** November 27, 2003, 08:22:08 am »

Hi shAf,
I am not suggesting it be eliminated, I would just prefer to have more control over what is saved and when . . . I am sure you are amoung many who use this feature.

LisaRCT · « **Reply #40 on:** November 29, 2003, 12:18:18 pm »

Can't the processor load (focus / thread priority) on the Database save function be made just a bit less so it is more background?
As it is everything within MC is locked-up during these saves and plays havoc with application focus elsewhere on my system. ?

JimH · « **Reply #41 on:** November 29, 2003, 01:27:45 pm »

Lisa,
Are you running the latest build? It was that one or the one just before in which Matt backed off the saves for extremely large libraries.

LisaRCT · « **Reply #42 on:** November 29, 2003, 03:42:21 pm »

Hi Jim,
Yes I am . . . . unless I am mistaken he backed down the save interval (Thanks!)

But when it saves it still steals so much focus I am in a holding pattern until it is finished.

cfgreen · « **Reply #43 on:** November 29, 2003, 05:28:02 pm »

On a large DB 45k Files, MJ Fix Cleared up your problem for me.
Thank the powers that be.

However I've been still having Meda Core crashes on refreshing the drive tree on my music drives (2). I did have 500+ artists in the root of each, Draggin a new album to the music drive Would then need a refresh, Smuck Done.

Anyway Thinking that there were just to many artists per root, I split the each drive to about 30-40 gig Directorys (0-B. C-F & G-J etc. Seems to be helping Less Dumps but still doing it.
Meanwhile I heading to do some info on ntfs ( max limits )
Thanks Charles

JimH · « **Reply #44 on:** November 29, 2003, 05:41:52 pm »

Could be just a disk error. Is either drive a Firewire or USB drive?

cfgreen · « **Reply #45 on:** November 29, 2003, 05:53:26 pm »

Nope

Firewire is only used for backup now (had my heart in a flutter a few times using 2 dual drive boxes) Xp can and will do some weird things here. Such as droping the firewire on top of a fixed data drive.

No the system is as follows now
c: 120g Sys
d: 45G Workdrive (incoming album music)
E: and F: are 160Gs W/ the music (190G) split across them

Media Center Registered 9.1.313 -- C:\Program Files\J River\Media Center\

Microsoft Windows XP Workstation 5.1 Service Pack 1 (Build 2600)
Intel Pentium III 998 MHz MMX / Memory: Total - 523 MB, Free - 267 MB

Internet Explorer: 6.0.2800.1106 / ComCtl32.dll: 5.82 (xpsp1.020828-1920) / Shlwapi.dll: 6.00.2800.1276 / Shell32.dll: 6.00.2800.1233 (xpsp2.030604-1804) / wnaspi32.dll: 4.71 (0002) , ASPI for Win32 (95/NT) DLL, Copyright © 1989-2002 Adaptec, Inc. / Aspi32.sys: 4.71 (0002)

Ripping / Drive G: Copy mode:ModeSecure CD Type:Auto Read speed:Max
Digital playback: Yes / Use YADB: Yes / Get cover art: No / Calc replay gain: Yes / Copy volume: 32767
Eject after ripping: Yes / Play sound after ripping: Yes Soundfile: chord.wav

Burning / Drive G: TDK CDRW401240B Addr: 0:0:0 Speed:40 MaxSpeed:40 BurnProof:Yes
Test mode: No / Eject after writing: Yes / Direct decoding: Yes / Write CD-Text: Yes
Use playback settings: No / Normalization: None

DMC-A1 · « **Reply #46 on:** November 29, 2003, 06:03:06 pm »

Definitely would like more control over saves.

I like the idea of having a manual save button (like MS Word, Excel, et al) when the TOOLS/OPTIONS/GENERAL/"Update Tags When File Info Changes" checkbox is deselected. "Manual Save" button would allow me to make batches of tag changes without actually changing the file until I am ready. This would also allow me to check differences in a file's original ID3V1 and ID3V2 tag info before MJ writes over them with sync'd info. Would even be happy if the existing "Update (Selected) Tags (from Library)" menu item was active when the "Update Tags...Changes" button is deselected in OPTIONS.

User comments re "Losing File Info" are valid, but are an every day fact of life for computer users. Draft a document without saving to a name, you lose it on system crash, etc. Maybe if this is a concern JRiver could provide a notice or somekind of warning to better serve less computer savy operators.

RhinoBanga · « **Reply #47 on:** November 30, 2003, 02:19:11 am »

You could easily narrow down the window of opportunity for loosing any changes by logging the details of the change to a permanent file instead of holding them in memory.

Then when you are ready you can apply the changes.

Inside the logfile you would hold the last modified date of each track and prior to actually performing the changes you can check the tracks to see if it was modified externally and give the user the choice of continuing or cancelling the batch.

This type of mechanism also gives you the ability to modify over MC sessions.

In fact if you think about it they already have a similar type of facility as you can undo/redo changes.

Can't see why it's such a big deal to implement and think of the cool things you could do. Make changes during the day, batching them up, then apply them overnight. A major productivity gain which isn't what computers are supposed to be for?

nila · « **Reply #48 on:** December 01, 2003, 03:37:58 am »

Sounds good to me Rhino, that would REALLY speed things up for tagging my 10,000 images.

I could make ALL to all 10,000 images, over multiple fields, get them all fully tagged and sorted at lightning speed and then leave the slow process of updating the files to happen overnight.
It would also mean the files would have to be modified once for all the field changes to be written in one big go instead of being modified 6 times for each different field.

It would GREAT increase the speed and power of MC as a tagging tool.

It could work like MS Word - if it was shut down without changes being applied or it crashed next time it opened it just prompted: Your changes were not applied last time you used MC, Below is a list of the changes you made
[list changes here]
Do you wish to apply them now?
Yes/No

For me personally this change would probably be the biggest incentive to make me get around to tagging those images, right now it's something I just keep putting off and off and off

gpvillamil · « **Reply #49 on:** December 01, 2003, 06:20:04 am »

Quote from: RhinoBanga on November 30, 2003, 02:19:11 am

You could easily narrow down the window of opportunity for loosing any changes by logging the details of the change to a permanent file instead of holding them in memory.

Then when you are ready you can apply the changes.

Inside the logfile you would hold the last modified date of each track and prior to actually performing the changes you can check the tracks to see if it was modified externally and give the user the choice of continuing or cancelling the batch.

This type of mechanism also gives you the ability to modify over MC sessions.

In fact if you think about it they already have a similar type of facility as you can undo/redo changes.

Can't see why it's such a big deal to implement and think of the cool things you could do. Make changes during the day, batching them up, then apply them overnight. A major productivity gain which isn't what computers are supposed to be for?

This kind of journalling mechanism would also pave the way to having the same library open on multiple machines. With a bit of cleverness, any changes would be written to a local changelog on each machine, and the actual tag updates could be managed by temporarily locking the library for writes from all but one system at a time.

INTERACT FORUM

Author Topic: 'Saving Database' excessively, resource hog (Read 10373 times)

LisaRCT

LisaRCT

LisaRCT

LisaRCT

jleerigby

LisaRCT

LisaRCT

LisaRCT

nila

LisaRCT

nila

LisaRCT

LisaRCT

fex

LisaRCT

fex

fex

LisaRCT

LisaRCT

LisaRCT

nila

LisaRCT

LisaRCT

LisaRCT

LisaRCT

LisaRCT

LisaRCT

nila