INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Faster search in database?  (Read 2077 times)

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Faster search in database?
« on: July 21, 2009, 09:06:26 am »

I know little about programming, so I am sorry if this is a stupid suggestion or misguided.

However, i seems like the database search is a CPU-limited operation, when i search in a database of around 200 000 files the search is reasonably fast, but far from instant, however it seems like the search only uses one core (i have a quad-core), and it also seems like it seldom uses that core 100%, does this mean that the program is not throwing enough CPU-power at the problem, and that it could be faster? If so that would be nice. (although i suspect making the search faster is in fact pretty difficult, if not it probably would have been done already...)
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42002
  • Shoes gone again!
Re: Faster search in database?
« Reply #1 on: July 21, 2009, 03:44:26 pm »

The answer to this sort of depends on the type of database search or view you're looking at.

Building a view has several components -- doing a database search, sorting, possibly grouping, adding to a list, drawing, filling panes if any are showing, etc.  Normally the database search is not the bottle-neck.

Some of the steps listed above are thread parallelized, and some aren't.  Thumbnails draw several at a time, and sorting runs in several threads.  Searches are run in a single thread, due to how database locking works.  Panes have to fill one at a time because each one feeds the next.

Posting a log of a view update after search would point to what takes the time in your case.

It might also be as simple as the fact that the program waits a little (depending on how many files you have) before executing a search as you type.  Is it possible this wait time is too long in your case?

We've put a lot of work into making Media Center fast, but it's always fun to try to make it faster :)

Thanks.
Logged
Matt Ashland, JRiver Media Center

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Re: Faster search in database?
« Reply #2 on: July 21, 2009, 05:44:00 pm »

Ok, i was talking about the speed of just typing into the the the search field, and getting results (although updating thumbnails could also be faster, it seems like the whole program can stop a little second if the source of the thumbnails (network) is slow, and a lot needs to be updated at the same time, but thats another issue).

You mention posting a log of a view update after search, how can i create this log?
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71524
  • Where did I put my teeth?
Re: Faster search in database?
« Reply #3 on: July 21, 2009, 05:48:55 pm »

You mention posting a log of a view update after search, how can i create this log?
MC's Help/Logging
Logged

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Re: Faster search in database?
« Reply #4 on: July 21, 2009, 06:20:52 pm »

Ok, i tried now, i first i searched for a term, the view updated to show the searchresults, i then search for a new term ("verve"), this is the log file from the event. I don't know if it's helpful or not:

Media Center; Version: 13.0.171; Types: 16383
223931750: 2660: General: Starting logging: Date: 22.07.2009 01:16
223931750: 2660: General: Log Reset: Logging reset
0002844: 2660: General: CMCResourceHelper::GetIsModalPopupShowing: Main window disabled
0005219: 2660: General: CPanesWnd::UpdatePanes: Start
0005266: 2660: Database: CMediaFileIOLoad::Load: Loading: C:\Users\Anders\AppData\Roaming\J River\Media Center 13\Library\Cache\File List Sort\Cache (1).dat
0005266: 2660: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Audio] ~sort=[Rating],[Artist],[Filename],[Name]-d,[Album Artist (auto)],[Album],[Disc #],[Track #]; Elapsed ms: 56,165
0005938: 2660: General: CPanesWnd::UpdatePanes: Finish (719 ms)
0005938: 2660: General: CMJFileListCtrl::StartFileInsertion: Start
0005938: 2660: Database: CMediaDatabase::RemoveFileAbsolute: Filename:  Mode: 0  Track: 0
0005953: 2660: General: CMJFileListCtrl::StartFileInsertion: Finish (15 ms)
0005953: 2660: General: CMJFileListCtrl::FinishFileInsertion: Start
0006891: 2660: Database: CMediaDatabase::RemoveFileAbsolute: Filename:  Mode: 0  Track: 0
0007016: 2660: General: CMJFileListCtrl::FinishFileInsertion: Finish (1063 ms)
0007297: 2660: Reader: CLocalReader::OpenInternal: Opening: C:\Program Files (x86)\J River\Media Center 13\Data\Default Art\NoThumbAudio.png
0007297: 2660: Reader: CLocalReader::Close: Closing: C:\Program Files (x86)\J River\Media Center 13\Data\Default Art\NoThumbAudio.png
0009203: 2660: General: CPanesWnd::UpdatePanes: Start
0009766: 2660: Database: CMediaInfoArraySort::Sort: Files: 69; Elapsed ms: 0,543
0009766: 2660: Database: CMJSearchHelper::GetResults: Search: verve [Media Type]=[Audio] ~sort=[Rating],[Artist],[Filename],[Name]-d,[Album Artist (auto)],[Album],[Disc #],[Track #]; Elapsed ms: 560,909
0009766: 2660: General: CPanesWnd::UpdatePanes: Finish (563 ms)
0009766: 2660: General: CMJFileListCtrl::StartFileInsertion: Start
0009781: 2660: Database: CMediaDatabase::RemoveFileAbsolute: Filename:  Mode: 0  Track: 0
0009781: 2660: General: CMJFileListCtrl::StartFileInsertion: Finish (15 ms)
0009781: 2660: General: CMJFileListCtrl::FinishFileInsertion: Start
0009781: 2660: Database: CMediaDatabase::RemoveFileAbsolute: Filename:  Mode: 0  Track: 0
0009797: 2660: General: CMJFileListCtrl::FinishFileInsertion: Finish (16 ms)
0009828: 2660: Reader: CLocalReader::OpenInternal: Opening: C:\Program Files (x86)\J River\Media Center 13\Data\Default Art\NoThumbAudio.png
0009828: 2660: Reader: CLocalReader::Close: Closing: C:\Program Files (x86)\J River\Media Center 13\Data\Default Art\NoThumbAudio.png
0012297: 2660: General: CMCResourceHelper::GetIsModalPopupShowing: Menu showing
0015375: 2660: General: CMCResourceHelper::GetIsModalPopupShowing: Menu showing
0017922: 2660: General: CMCResourceHelper::GetIsModalPopupShowing: Main window disabled
0017922: 2660: General: RunProgram: Start
0017922: 2660: General: RunProgram: Filename: C:\Users\Anders\AppData\Roaming\J River\Media Center 13\Log.txt / Parameters:
0017922: 2660: General: RunProgram: Performing ShellExecute...
0017969: 2660: General: RunProgram: Running process...
0017969: 2660: General: RunProgram: Finish (47 ms)
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42002
  • Shoes gone again!
Re: Faster search in database?
« Reply #5 on: July 22, 2009, 08:44:38 am »

The interesting thing here is that the search + sort for "verve" took 0.5 seconds.

Populating panes and the list took another 0.1 seconds.

So it's actually reasonably fast.  It took less than one second to search and display results for 200,000 files when searching multiple fields.

I wonder if the problem is just that it waits too long before starting the search after you stop typing?
Logged
Matt Ashland, JRiver Media Center

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Re: Faster search in database?
« Reply #6 on: July 22, 2009, 10:07:06 am »

The interesting thing here is that the search + sort for "verve" took 0.5 seconds.

Populating panes and the list took another 0.1 seconds.

So it's actually reasonably fast.  It took less than one second to search and display results for 200,000 files when searching multiple fields.

I wonder if the problem is just that it waits too long before starting the search after you stop typing?

I can give my subjective evaluation of the case. First i do a search for something else then "verve", so this is the search result displayed. I mark the term in the search field, and write verve. It takes a little while before the characters show up in the search field (0,5 seconds maybe), about the same time it shows my whole collection in the windows, instead of the old search-result. Then it goes maybe another half a second before it displays the search result for my search term (verve). I hope this was more or less understandable :) Less than a second is impressive, but it still not "instant" ( i don't know how fast it would have to be for it to feel subjectivly instant). I guess i am just asking abit too much :), as i mentioned the reason for the question was that the CPU doesn't seem to run at maximum capacity during the search.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42002
  • Shoes gone again!
Re: Faster search in database?
« Reply #7 on: July 22, 2009, 10:15:27 am »

Thanks for the details.

I see one problem right away.  If you type a search, then type another search over the top, it immediately shows the view with no search.  Then it lets you type and filters down.

If it would just wait for you to stop typing, it would feel a lot better in this case.

It also waits two seconds before even responding to your changes, which is too long I think.  It adjusts based on the number of files in your library.  It's 0.25 seconds for most users.
Logged
Matt Ashland, JRiver Media Center

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Re: Faster search in database?
« Reply #8 on: July 22, 2009, 10:31:04 am »

Ok, thanks for interesting info, i guess the updating of the views can't be done "independent" of the search? With that i mean that it could start processing the search for "verve" at the same time as it updates the view to show all files? And also maybe the waiting time could somehow be ajusted automatically (or evnetually user configured) also depending on CPU-speed? Maybe two seconds was a much better match with my old slow computer, I know how a computer that is more powerful than i need (you can't always be rational when it comes to a hobby :)), and search is much faster.
Logged
Pages: [1]   Go Up