Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rolf_eigenheer]

For speeding up MediaCenter I deleted 3/4 of database entries.
But the database size (zipped) only reduces from 27MB to 18.5MB. Database operations are still slow.
How to pack the database to remove deleted items and defrag ?

[JimH]

The database isn't usually the limiting factor.

Are your files on a network drive or server?  Or on an external USB drive?

Complex custom views can also slow down display.

[rolf_eigenheer]

Yes there are some 'complex' views.

I have some favourite artists which are stored in a Folder called 'INTERPRETEN'. In a different folder 'SAMPLER' there are compilation albums.
My main ARTIST view filters all artists from 'INTERPRETEN' and expands artist, so that also the corresponding tracks from the samplers are shown.
So it is possible that far more than 30 albums contain tracks for a given artist. To get a better overview I use calculated field 'Album (auto)'
 If the 'AlbumType' is 'MultipleArtist', 'Album (auto)' returns an empty string
 else if the user defined field 'AlbumSet' is not empty, 'Album (auto)' returns  'AlbumSet'
 else it returns 'Album'

Yes. there are some operations to be done before grouping. But with only 70'000 audio files in the lib, no query can be too complicated.

[rolf_eigenheer]

I made some more investigations on that issue.
My artist view filters 37'500 tracks (in the path INTERPRETEN' out of 70'000.
Expand 'artist' increases the dataset by 3200 tracks.
Therefore the dataset contains 40'700 tracks.
In the first level of theater view all 518 artists are shown.
In the second level all the albums of the selected artist are shown (1 to 50)
In the third level the individual tracks are shown.

- Clicking on an artist in the first level takes 6 seconds until the albums are shown.
- Clicking on an album in the second level takes 6 seconds until the tracks are shown.  This delay is the same if even there is only one track on the album.
- Going back one evel takes another 6 seonds.

All thumbnails are generated. Defender is well configured. The computer performs well. Proof: Selecting 'Play' on an album instantly sends the tracks to playing now and starts to play. Seleting and acessing the track works fast. But changing to the next level takes incredible long. It seems as the initial expansion is redone every time.

[zybex]

6 seconds is definitely too much, 70.000 tracks is not that many. It should take way less than 1 second.
It's very unlikely that it has anything to do with the DB disk size. The DB is pretty much entirely loaded in memory - if not by MC, then by OS caching as different sectors are accessed. Deleting tags to make it smaller has no effect there - all it matters is the total number of records, Caching takes care of keeping relevant data in memory.

If you want to completely rule out the DB size, you can create a RAMDisk and put the DB there, to check if performance increases. But I think you already saw with ProcMon that there are no more disk accesses when browsing the library? Or at least, after loading the view for the first time I would expect no more DB access.

Can you post ALL the expressions involved in your Views and relevant calculated fields? there must be something there that is causing the slowness. I also remember a recent thread where the Parent/Top-level view was affecting the underlying views due to inheritance.

[rolf_eigenheer]

Can you post ALL the expressions involved in your Views and relevant calculated fields? there must be something there that is causing the slowness. I also remember a recent thread where the Parent/Top-level view was affecting the underlying views due to inheritance.

The field Album (auto) is a calculated field:

if( math( isequal( Left(albumtype(), 6),Single,1)),
  if(isempty([albumset]),
  [album],
  [albumset]),

)

'Albumset' is a user defined string field which is stored inside the file tags.

[rolf_eigenheer]

It seems that all processing is repeated when entering a selection.

We would expect, that the dateset is reduced when we select one artist. It should be further reduced when selecting one album.
And displaying the tracks of a selected album should take no time.

Adding the tracks to Playing now works with no delay. And displaying 1 to 6 tracks on screen shouldn't take additional 5 seonds.

[rolf_eigenheer]

Since there is no pack/defrag functionality, I want to rebuild the whole database.  I'm lucky that all tags are stored inside the media files.

I want to preserve all database fields including calculated fields. And I want to preserve all my custom views and any settings.
But all media shall be deleted from the database. And the database should have almost zero size.

Can this be done from within MC or do I have to delete manually some files from the library folder?

[JimH]

Backup first.

Then add a new Library (under the File Menu).

Reimport.

Before you do that, try removing any complex views you've set up.

[zybex]

Is it also slow without that "Expand" modifier?

The expression you pasted is fine - there's an extra Math() there is not needed, but I don't think that would cause your problem:

Code: [Select]

if( isequal( Left(albumtype(), 6),Single,1), if(isempty([albumset]), [album], [albumset]), )
And, let me repeat myself: it doesn't matter if each File record takes 1 KB or 100 KB, as long as it fits in memory. Getting to a given record/field is based on offsets, pointers and indexes, and it doesn't matter if they're more or less 'packed' together.

[rolf_eigenheer]

Backup first.

Then add a new Library (under the File Menu).

Reimport.

Before you do that, try removing any complex views you've set up.

When doing that, an exact copy of the db is loaded.
When I do Clear Library, all Views are reset too.

[rolf_eigenheer]

Is it also slow without that "Expand" modifier?

The expression you pasted is fine - there's an extra Math() there is not needed, but I don't think that would cause your problem:

Code: [Select]

if( isequal( Left(albumtype(), 6),Single,1), if(isempty([albumset]), [album], [albumset]), )
And, let me repeat myself: it doesn't matter if each File record takes 1 KB or 100 KB, as long as it fits in memory. Getting to a given record/field is based on offsets, pointers and indexes, and it doesn't matter if they're more or less 'packed' together.

If everything is indexed, then the size of db does not influence the speed. Is really everything indexed ?

Removing 'Expand' reduces the dataset by 20% but increases speed by factor 3 or more. It seems that expansion is done more often than required.
Expand artist is applied to the initial dataset. Therefore there is no need to do it after entering subviews. It is NOT applied to the selected files. If it were so, then if a subview filters for a multiartist album, all tracks from these artists on any albums would have to be shown. This is not the case.

[zybex]

Sounds like a problem with Expand, not the db size  ¯\_(ツ)_/¯

If you just want to drill down over AlbumTypes->Artist->Album->tracks, I think that can be setup without Expand. I'm not the best guy to help there as I don't use MC for audio though. But I've seen people here with half a million tracks, and they don't report this kind of slowness.

You mention Expand causes a 3x slowdown... If it's taking 2 seconds per click now, that's better but I think it's still not normal. Anyone else with large collections can confirm, or report on their speeds please? It would be useful to know what's normal.

If everything is indexed, then the size of db does not influence the speed. Is really everything indexed ?

Regardless of indexing (ordering), if each record has 100 fields it doesn't matter if they're empty or not. Each record needs to contain a length/offset+data for each field. Having empty fields just means that the int32 representing the length of those fields is zero, but it still exists. When acessing a field's data, like [Album], MC still needs to get the Offset of that data within the record and the length of the contents. For performance, it doesn't matter at all if [Album] is at offset X or Y, or if there are other fields there with data or not. It's random access memory, not sequential. Once a record is loaded into memory (MC's memory or Windows I/O buffers), record size no longer matters.

To be clear: if you don't access a given field, there's no performance hit by just having it in the database. If you access it, then Indexing can of course help, specially if your expression asks for sorting or grouping by that field.

[rolf_eigenheer]

Thanks zybex!
The machine is a atom driven windows tablet. I thought I could run at least a stripped down audio only collection with 70'000 titles.
But you're right. The main problem is the implementation of the expand function. Expand is executed when it shouldn't.