Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[jumper24]

Hi

We are on a chase for a media player that can do instant search.
 - Its for use on a radio show - for qucik search, making playlists.. (dalet is out of the question)

Right now there is 770000 items in the lib.

JRiver is a spiltsecond faster than musicBee (but still very slow - several seconds)
 - the windows media player buildin win 8.1 is almost instant.

We have state of the art 12core PCīs, lots of ram - so its not a performance problem.

The question is.. is there any plan from your side to update your search technology ? Maybe with some SOLR tech.

We have tried ALLOT of players.. We are so far out that we are in research to build our own music search engine :-)


Kepp up your good work !!

Best regards Tommi

 

 

[JimH]

Welcome to the forum.   Have you imported all your files?  Search shouldn't take several seconds.

Antivirus software should not be set to look at each media file each time it's opened.

[jumper24]

Thank you jim - no antivirus running.
Yes all media is imported into JRiver. (been a jriver user for over a year - also with osx licens)

Also started from scratch with a new library (you never know - takes only 2 days to import everything again:-)

Everything is running from the fastest Samsung pro SSDīs.
 - At one point we was running JRiver + the library from a RAM disk (did not improve anything)

Running on 3 different systems with same results.

Also notice, we are talking soon close to 1mill items with lots of metadata. (not sure anyone did that before ?)

[Matt]

700,000 items in the library is a lot, but you should do better with JRiver than any other program:
http://jriver.com/speed.html

There are some things you can do on your side to help performance.  For example:

Use a simple view (no list grouping, no panes, etc.).

Search in specific fields (ie. "[Artist]=dylan" is orders of magnitude faster than "dylan"). Keeping saved Smartlists could help with this.

Turn off default search fields (Options > Library) that you don't need searched.

[jumper24]

Already did that 
 - Only use file list view, no grouping.. And i only search in album, artist, composer, genre, name.

[Matt]

Already did that  
 - Only use file list view, no grouping.. And i only search in album, artist, composer, genre, name.

If you make a playlist and search there, is it faster?

If you paste the final search into the box, is it faster?

What if you search using brackets, so [Bob Dylan] vs Bob Dylan?  Brackets speed up the search since it only has to probe the beginning of the values.

Could you post more details about the timing and search used, maybe using a stop watch?

[JimH]

Is everything on a local drive?  If not, please describe your setup.

[Matt]

Is everything on a local drive?  If not, please describe your setup.

Everything is running from the fastest Samsung pro SSDīs.
 - At one point we was running JRiver + the library from a RAM disk (did not improve anything)

[jumper24]

Is everything on a local drive?  If not, please describe your setup.

Windows, JRiver and the lib is on SSD..
All the media files itself is right now in 3 places (all complete backups)
1. on internals disks on the main computer.
2. on a external sata raid enclosure.. extremely FAST
3. on a Synology DS1512+
 - But dosent matter for the search speed where the files are in my tests, be it over network or local sata disks.

[JimH]

It might be useful to test each drive separately.  They may not be as fast as you expect them to be.

The Synology NAS will be limited by the network speed, which can also be slower than you expect.

[jumper24]

If you make a playlist and search there, is it faster?

If you paste the final search into the box, is it faster?

What if you search using brackets, so [Bob Dylan] vs Bob Dylan?  Brackets speed up the search since it only has to probe the beginning of the values.

Could you post more details about the timing and search used, maybe using a stop watch?

I will make some more scientific tests tomorrow.. :-)
But.. on a feeling.. [Bob Dylan] is a little faster than Bob Dylan (pasted in)

If i can find a screencast program i can record it.

[jumper24]

It might be useful to test each drive separately.  They may not be as fast as you expect them to be.

The Synology NAS will be limited by the network speed, which can also be slower than you expect.

notice the media collection is not spread across these 3 destinations. All 3 places have the complete collection media files.

On the main computer i have toggelt between the internal drives and the external exclosure (5x4TB in raid0 = VERY fast).
On the slave computers i connect to the Synology.. via 1Gb network, around 100Mb read pr sec.
But dosent matter for the search speed in my tests how the collection is attached to JRiver.

[Matt]

For search speed, it will not matter where the media itself is located.

Search is purely a database operation, and there should be no interaction with the physical media files.

[Matt]

If you'd be willing, please provide a library backup to matt at jriver (dot) com.

This way, I could test the exact same search on my side.  That's the best way for us to learn about performance.

Thanks for your help.

[jumper24]

If you'd be willing, please provide a library backup to matt at jriver (dot) com.

This way, I could test the exact same search on my side.  That's the best way for us to learn about performance.

Thanks for your help.

Have just dropped you a mail with a lib to test with.

[Zootsuit]

Matt, your suggestion on using brackets is very helpful. Typing the search terms into the search field is very slow. Any addition suggestions on how to speed that up?

[Matt]

jumper24, thanks for the library backup.

Most of the lag is from evaluating the search suggestions (which can block the file list from refreshing due to how database thread safety is managed).

I need a little time to think on this one about how to improve it.

[Matt]

Next build will improve the speed when searching your library.

Some of the changes coming that help performance:
Faster: Typing in the search box with large libraries is more responsive.
Faster: The media type field no longer changes an empty media type to 'Unknown' at display time (it imparted a small performance penalty for no real-world benefit).
Faster: Improved search performance in the month field (which was one of the more intensive default search fields so the overall performance gain is appreciable).
Faster: Gets on the filename field are around 20% faster.
Changed: Album Artist is no longer included when offering search suggestions (since it often duplicates artist, and even when it doesn't normally doesn't contain a useful search value).

All of these should add up to make a noticeable difference.

Since you have 800k files, it may still take a second or so to update after a generic search.  You could use a field prefix like ar= if you want even more speed.  Doing that narrows the search and also disables suggestions.

[Matt]

One thing that might help for a use-case like yours would be if the search didn't auto-execute as quickly, but would execute immediately if you hit enter.

This way, on your way to typing "bob dylan" it would be less likely to pause while updating the view for a partial search like "bob d".

Currently it waits for no text input for 350ms then updates the view.  Depending on how quickly you type, the behavior will vary.

If we could make it adapt based on view update speed, it would be better for all users.  For a more modest library (say 20k files) it could update instantly instead of waiting 350ms.  For a large library like yours, it could back off to a second or so (but you could hit enter to force it to run).

[jumper24]

Thank you so much !!

Cant wait to try it...

[jumper24]

VERY Nice update..

I like the speed when using ar=*something* (same for name and album)

But, What is the difference between:

1. Remove alle Fields except for Artist from the default search in the "manege Library fields"
(still somekind of slow if you know how fast ar= are - should the speed not be the same ?, did i misunderstood this feature)
and
2. use the ar=*something*

My dream is the speed of ar=*something*, but across artist, album, composer also.
Is this possible ? Or would that require JRiver to be multithreaded or something ?

But THANKS !!!

[Matt]

I like the speed when using ar=*something* (same for name and album)

But, What is the difference between:

1. Remove alle Fields except for Artist from the default search in the "manege Library fields"
(still somekind of slow if you know how fast ar= are - should the speed not be the same ?, did i misunderstood this feature)
and
2. use the ar=*something*

The only difference is that ar= will disable search suggestions, because they turn off when they sense syntax like [, ], =, etc.

The suggestions shouldn't really get in the way of view updating or typing since they're in a low priority background thread.  This gets a little less clear when you have 800k files, so it's possibly relevant in your case.

You have many fields marked as default search fields.  Reducing the list more will help performance.

Remember to hit enter after you type so you're not waiting for the program to decide you're done typing (it'll now wait longer on a huge library like yours).

[jumper24]

The only difference is that ar= will disable search suggestions, because they turn off when they sense syntax like [, ], =, etc.

The suggestions shouldn't really get in the way of view updating or typing since they're in a low priority background thread.  This gets a little less clear when you have 800k files, so it's possibly relevant in your case.

You have many fields marked as default search fields.  Reducing the list more will help performance.

Remember to hit enter after you type so you're not waiting for the program to decide you're done typing (it'll now wait longer on a huge library like yours).

Okay

A quick test (and with ENTER as soon as i can)

ar=oasis
takes around 2-300ms

Default search reduced to only Artist
around 1.2sec. (give and take)

Default search, back to stock, around 27 fields.
Around 1.2sec

Seems like the search engine is ignoring "default fields to search" and just search it all.. (see attached image)

Try it out if you have 1min :-)


 

[Matt]

Seems like the search engine is ignoring "default fields to search" and just search it all.. (see attached image)

You're right.

It's actually by design but I had forgotten (I thought the change only applied to suggestions, not the search itself):
17.0.4 (9/16/2011)
Changed: Stock fields don't use the 'Default search field' option, but instead rely on rules in the source code.

Let me think on it a while.

[MrC]

If possible, it would be nice for the Default search field to be reverted to actually have a consistent meaning throughout MC.  Set default values as they are today, but allow uses to deselect the default.

There is a description of how it works in the Properties wiki:

Fields in Media Center have several defined characteristics. These are shown in the annotated example Manage Library Fields dialog.
File:Library_Fields.png

Each field:
...
11. May be a Default search field when performing search queries (see Note: Default Search Field)

...

Note: Default Search Field. Beginning with MC17, the Default search field setting is not used for stock fields; instead searching fields relies on rules built into the program. Nearly all stock fields are now searched by default. User-defined fields will honor this setting.

[Matt]

If possible, it would be nice for the Default search field to be reverted to actually have a consistent meaning throughout MC.  Set default values as they are today, but allow uses to deselect the default.

Currently the option is only relevant for user fields.

It might make sense to allow three-states for stock fields: auto, on, off.  However, I'm not sure if new work here would benefit a very large audience.

[MrC]

Currently the option is only relevant for user fields.

It might make sense to allow three-states for stock fields: auto, on, off.  However, I'm not sure if new work here would benefit a very large audience.

Yeah, I get it.  But its one of those crazy things: MC knows which fields are stock or user fields, yet shows the impotent option  for stock fields irregardless.  It should be greyed out if it is going to not be usable.

I don't get the auto setting in this case.  From a user's perspective, they either want to search this field, or they don't.  Auto is like MC saying to the user "Maybe I will search this field, but you'll have to go figure out if that's the case, cuz' I'm not telling you!".

[Matt]

Yeah, I get it.  But its one of those crazy things: MC knows which fields are stock or user fields, yet shows the impotent option  for stock fields irregardless.

I agree it shouldn't show the option if it doesn't do anything.

The issue with it being a two state thing (basically what we had in the past) is that I want the rules for picking fields to search to be smart and changeable with new builds.  For example, the library got a lot faster so we could search more in v17.  Then we added a system to skip empty fields or fields that don't apply to the current media types.  There's some more smarts in there as well.

Another possibility is to offer an option at the search box itself where you could pick between a normal search and a shallow (but faster) search.  It could save the selection between runs.

[MrC]

I see.  May "auto" could then be "Default (on)" or "Default (off)", as per MC's internal configuration for the field.

I considered the Deep Search idea, but was almost certain it would be rejected.  :-)  So I'm glad you thought of it!

And off topic, btw, I'm LOVING the ability to tag mass fields via MCWS, and the clipboard being in RFC 4180 format!  Being able to send an MCC command to copy the clipboard means I don't  have to force users to Export to MPL, or manual copy.

[6233638]

It does seem odd to have those options when they don't do anything.
 
Looking at the fields being searched by default, there are a lot there which I would not have considered relevant when searching, that I would disable for some extra speed.
 
I would be more likely to use "Album Artist" for a search than "Artist" for example, as that will only bring up albums by that artist rather than showing their tracks on compilation albums.

[jumper24]

@MrC or @Matt

Have you thought more about search speed optimization ?
 - The issue: MC searches all fields without respecting the "Default search field" option - so VERY slow search if you have a big library .

[MrC]

I don't work for JRiver, so have nothing to offer here.  Matt has been out due to a bonk on the head.

[jumper24]

Matt has been out due to a bonk on the head.

Opps i'm very sorry to hear that ..