INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: New Thumbnail Storage (EXPERIMENTAL)  (Read 5039 times)

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
New Thumbnail Storage (EXPERIMENTAL)
« on: August 28, 2006, 06:45:05 pm »

We're playing with a different spin on thumbnail storage.  It allows better looking thumbs in most views and requires less decoding time.

The trade-off is that it needs to do slightly more disk seeking. (but not I/O)

It looks like a win on our machines.  We'd like to hear what you find.

To test:

  • Scroll fast in some thumbnail views with different size thumbs and look at the quality (so you can compare later)
  • Shut down MC12
  • Overwrite your existing Media Center 12.exe with the experimental copy here:

ftp://ftp.jriver.com/pub/downloads/music/tmp/ThumbTest.zip
  • Manually erase all thumbnails in Options > Tree & View
  • Let it build all thumbs
  • Finally, compare the speed and thumb quality  (it won't be fair to compare speed while thumbs are building)


Let us know what you find.  Be sure to note how much better it looks at big thumbnail sizes.

Thanks!
Logged
Matt Ashland, JRiver Media Center

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #1 on: August 28, 2006, 07:20:30 pm »

It's a win, no doubt.  The thumb-building process was lightning quick, and these thumbs are VASTLY better looking.  The scrolling is fast, but not overwhelmingly faster.  I don't notice any issues having to to with disk access--switching views seems fine.  And the thumbs look GREAT!!


EDIT:  I thought the image thumbs might be a tad soft, but i take that back.  They just look great.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #2 on: August 28, 2006, 07:24:39 pm »

After shrinking to make a thumb, a little sharpening is needed or else things look too soft.  We still need to tune how much sharpening the new system needs if it gets the green light.
Logged
Matt Ashland, JRiver Media Center

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #3 on: August 28, 2006, 07:44:25 pm »

I think the sharpness is so good as it is, it's not worth messing with.  My biggest gripe now would be thumbnail max size--let's make them even BIGGER!  Super Size!  (Oh, and vertical split views would also help.)
Logged

datdude

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2222
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #4 on: August 28, 2006, 09:20:25 pm »

Just replaced MC12.exe and it trys to start up, I can see the interface, but it just closes down.

Rats, it just reverted to an older verion of my library after I replaced it with the standard MC12.exe.
Logged
"You are not a beautiful or unique snowflake." -  Just a very big snowball

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #5 on: August 28, 2006, 09:35:41 pm »

You need to replace "Media Center 12.exe" in your program files folder. (not MC12.exe in the system folder)

Make sure you install build 62 first.
Logged
Matt Ashland, JRiver Media Center

datdude

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2222
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #6 on: August 28, 2006, 09:43:09 pm »

Allready did both, and restarted after replacing.

Guess I will have to wait, hopefully I will get to see it.
Logged
"You are not a beautiful or unique snowflake." -  Just a very big snowball

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #7 on: August 28, 2006, 10:07:43 pm »

Just completed the test as described.  Went back to original version of the exe and re-generated again to confirm my opinions.

I would say the new "tester" version does have slightly higher quality thumbs.  It isn't night and day at all, but they do seem sharper and "crisper".  Scrolling also does seem to be a bit "smoother" to the eye (though the effect is certainly subtle).  I'm having trouble reproducing because the "test" MC is crashing now when I try to switch back to it again (as soon as I click Images in the tree).  I have something else going on with it though... I'll mention in a separate post.

I would say I saw no signifigant difference in thumbnail generation speed at all (if anything it was slower with the test version, but only slightly).
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

GHammer

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1930
  • Stereotypes are a real timesaver!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #8 on: August 28, 2006, 10:31:13 pm »

Working well here, I do not notice any lag even when I jump from top to bottom of lists with 1000s of entries.
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9119
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #9 on: August 29, 2006, 02:59:47 am »

1st impressions are good, especially for the larger thumb sizes. I personally didn't notice any massive improvement in scrolling, with it still hitting the odd speed bump on its way from top to bottom.

Echo the request for a larger 'default largest' thumbnail size.

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #10 on: August 29, 2006, 09:31:37 am »

1st impressions are good, especially for the larger thumb sizes. I personally didn't notice any massive improvement in scrolling, with it still hitting the odd speed bump on its way from top to bottom.

Same here.

Also...

I'm having a ton of problems with recent builds and Thumbnailing though.  I have not been able to complete a manually started Build All Missing Thumbnails without either: a) crashing MC, or b) crashing the whole computer BSOD-style (which happened last night).

I do have the vast majority of my files all on network drives.  I would guess, from the way MC is acting, that the crashing is when thumbnailing some of my Video files... But I'm not able to determine what ones at all (or even if it's the same ones each time).

When MC12 is building Thumbnails "on the fly" (especially when viewing a large video view scheme or a "mixed" scheme) in "details" mode, it becomes extremely non-responsive to the point of being completely unusable.  Generally it will consume nearly 100% CPU time for 15-20 minutes at a time, only repainting the UI very occasionally.  And, about 1/2 the time when it gets "stuck" like this, it eventually crashes.  I have no similar problems with MC11.1 and the exact same Library.

Last night, to try to alleviate this problem, I tried to "Build All Missing Thumbnails" three separate times.  The first two times it crashed after going through about 1200 of 30k files.  Then I suspected it could be some strange DirectShow filters I had installed on that machine (that one still had the Morgan Stream Switcher installed for example), so I went on a cleaning frenzy.  I uninstalled all of my codecs and filters except for my bare necessities:

ffdshow-20051129.exe (which I reinstalled from scratch)
XviD 1.1 (XviD-1.1.0-30122005.exe from Koepi)
Huffyuv 0.2.2.1
DirectVobSub 2.23
RealAlternative (current build)

The third time I ran the "Build All Missing Thumbnails" routine, it made it into the 2500k range and I went to bed.  When I woke up this morning, I was greeted with a BSOD.

I have logs (with all the options enabled).  Is there any information I can provide to help track this down?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

Doof

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5908
  • Farm Animal Stupid
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #11 on: August 29, 2006, 10:14:45 am »

I'm on a small library here at work, so my numbers may be weird...
 
Thumbnail Creation

The first time I tried this, the experimental version took 1:05 to create 644 thumbnails. I then figured I should time the old system to compare. So I went back, deleted the thumbnails and told it to recreate missing thumbnails. It took 19 seconds. I tried again. This time it only took 12 seconds. So I fired up the new version one more time, erased them and recreated them and it took 20 seconds. So all in all, not much different (although I don't know why it took over a minute the first time). I'll give this another go when I get home to see how it works on my primary library (~15,000 files).

Quality

Very noticeable difference. There were a few thumbnails in the original setup that were very noticeably poor quality. With the new system, they are much cleaner and nicer looking.
 
Scrolling

I don't notice a difference, but then, I wasn't having any scrolling issues here to begin with. Maybe my home library will be more noticeable.
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #12 on: August 29, 2006, 01:52:54 pm »

After working about 50 minutes this experimental version crashed on thumbnail # 39684 of 48782.

The thumbnail file size was about 410 MB at that stage.

I restarted MC and it started to build thumbnails again from scratch. After a some browsing the file size is now 40 MB.

The previous MC12.0.62 complete "large" and "small" files for the same library were 224 and 32 MB.

MC11.1 thumbail file for the same library is only about 150 MB, which I already considered very big. Probably MC11.1 has not build a thumbnail for each audio file yet since the old icon display doesn't force it to do so.

In general, MC really should use the same thumbnail for all album files if a linked image file is used. MC wants to build about 39000 thumbnails (I have about 9000 audio files without art and also document files) even this library has only about 9400 image source files (2090 external cover art files, 7004 image files including the 2090 imported cover art files and 318 video files).

If I don't count the videos the image files use about 1.4 GB of HD space. This experimental version is going to make an about 500 MB thumbnail file for those images (if it ever reaches the end without crashing).

EDIT: fixed the file amounts
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #13 on: August 29, 2006, 02:29:30 pm »

In general, MC really should use the same thumbnail for all album files if a linked image file is used.

This was a major addition to the MC 12 thumbnail system.  The same thumb won't get stored on disk or in memory twice, even if a file uses internal cover art.

This applies to the current and this experimental system.
Logged
Matt Ashland, JRiver Media Center

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #14 on: August 29, 2006, 03:22:47 pm »

Hmm...

Shouldn't the MC12 thumbnail files then be smaller even if you have reduced the compression or increased the resolution somewhat?
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #15 on: August 29, 2006, 03:49:16 pm »

I browsed through the audio and image categories. Then I displayed my video folders one by one. This time MC managed to build the thumbnails without crashing. The file size is now 425 MB.

I can start testing it now.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #16 on: August 29, 2006, 04:59:44 pm »

As a test I let MC11.1 build all missing thumbnails. The resulting file is huge, 618 MB. So the MC 12 files are indeed smaller.

However, my old 150 MB MC11.1 thumbnail file already contained all album covers, image files and video files, just not every audio file.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #17 on: August 29, 2006, 05:10:51 pm »

Since I copy the library files and the thumbnails to my other PCs this is not a small issue to me. It is already slow to copy the 150 MB MC11.1 file over LAN.

A library backup is only about 8.5 MB, but a restore erases old thumbnails.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #18 on: August 29, 2006, 05:54:11 pm »

Just a heads up that we won't be going with this system.  The disk seeking is too much for some systems.

However, we're still considering the high-quality big thumbnails as an option.  As Alex has pointed out, they do increase the size of the thumbnail database quite a bit so we're not sure if it's worth it.

The alternative is to show a low-res version first and then fill in the thumbnail from the original image in a background thread.  This isn't good for stacks, videos, LAN files, RAW images, etc. so it's not our first choice.
Logged
Matt Ashland, JRiver Media Center

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #19 on: August 29, 2006, 06:03:41 pm »

That is a shame, IMO.  The image quality of the large thumbnails was truly compelling.
Logged

RobOK

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 615
  • long time MC user for PC, iPod and Tivo
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #20 on: August 29, 2006, 06:32:29 pm »


The alternative is to show a low-res version first and then fill in the thumbnail from the original image in a background thread.  This isn't good for stacks, videos, LAN files, RAW images, etc. so it's not our first choice.

I think Picassa does this, it works quite well.  I can't comment on your last statement about videos and LAN files though.

Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #21 on: August 29, 2006, 06:39:49 pm »

The alternative is to show a low-res version first and then fill in the thumbnail from the original image in a background thread.  This isn't good for stacks, videos, LAN files, RAW images, etc. so it's not our first choice.

This is a big issue.  It's the way iPhoto works (of course) and it's performance is absolutely dismal when the files live on a network share (or even a removable drive).  Dismal to the point of being unusable with even remotely large libraries.

HTPC and Media-centric computing are a major reason to invest in NAT and external storage (especially once you start using it for video).  The disk storage needs are just so high, and the drive to be able to access your media from throughout the house (and on the move), that it pushes people towards these solutions.  Slower speed media definitely needs to be considered seriously with any MC disk access scheme.

I didn't notice any disk seeking issues with the new scheme (I assume it was all local disk access), but my local drives are all very robust.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #22 on: August 29, 2006, 06:57:06 pm »

I compared MC12 experimental vs. MC11.1 since it was possible to try them side by side.

MC12e displays the biggest possible thumbnails sharper and has less compression artifacts. (MC12e on the left.):



Click to enlarge. (650 kB, lossless 24-bit png)


Scrolling speed of 7004 image files at the smallest possible size was much better with MC12 (about 2.5x faster). At the biggest size I didn't notice a practical difference in scrolling speed. (2.8 GHz P4, 1 GB RAM, fast local HD)

Just a heads up that we won't be going with this system. The disk seeking is too much for some systems.

However, we're still considering the high-quality big thumbnails as an option. As Alex has pointed out, they do increase the size of the thumbnail database quite a bit so we're not sure if it's worth it.

The alternative is to show a low-res version first and then fill in the thumbnail from the original image in a background thread. This isn't good for stacks, videos, LAN files, RAW images, etc. so it's not our first choice.

Actually, I agree with jgreen. The display quality is very nice. This reminds me about my request of a quick hi-res display in thumbnails view. How about adding a button that would temporally increase the thumbnail quality on currently displayed screen? MC would then draw temporary high quality thumbnails that would stay until a next screen redraw.

I really would like to have more options. For example, when I access the media files over LAN I would like to be able to configure the thumbnail generation as follows:

- audio: fast low quality thumbnails, missing thumbnails automatically built as a low priority background task after each startup
- images: slow high quality thumbnails, only on demand (like the current default)
- video: no thumbnails or perhaps: right-click > build thumbnails for selected files
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72367
  • Where did I put my teeth?
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #23 on: August 29, 2006, 07:16:22 pm »

That's New York.  Just below Central Park, I think.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72367
  • Where did I put my teeth?
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #24 on: August 29, 2006, 07:17:52 pm »

That is a shame, IMO.  The image quality of the large thumbnails was truly compelling.
I don't think we're going to lose any quality on thumbnails.  We're just trying to speed up the scrolling and display.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #25 on: August 29, 2006, 08:32:20 pm »

Please try the build that just went up everyone.

It keeps the big, sharp thumbs.

It keeps the speed of the small thumbs of old. (doesn't require the disk seeking like the experiment)

It adds a medium sized thumb that is used for common thumbs. (this about triples decoding and disk I/O time for the sizes 80x80 to 180x180)

It improves the look (sharpness, sizing quality, etc.) of all thumbs.

Thanks for any feedback.
Logged
Matt Ashland, JRiver Media Center

jgreen

  • Citizen of the Universe
  • *****
  • Posts: 2419
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #26 on: August 29, 2006, 09:02:08 pm »

Just for the halibut, after deleting/rebuilding/inspecting thumbs, I reset thumb size to 480xauto and deleted and rebuilt them again.  Here are some observations:

--If the original image size is close to the 480xauto size, the thumb quality is virtually perfect.

--If the original size is large, say 2000 px on a landscape orientation, the thumb quality is very good, although crisp compared to the previous.

--If the original size is medium, say ~800 px on a portrait orientation, the 480xauto thumb is absurdly soft. 

It seems that MC is making a judgement re/how to cache the thumbs.  The bicubic downrez of the medium images just should not be that soft.  I think that for those images in particular, MC is caching a small pic and then uprezzing it considerably.  maybe the reason the 2000 px landscape images don't look as soft is that the thumb is so much smaller when compared to portraits (480 on widest dimension vs 480 on the narrowest).
 
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: New Thumbnail Storage (EXPERIMENTAL)
« Reply #27 on: August 30, 2006, 03:42:02 pm »

That's New York.  Just below Central Park, I think.

Unfortunately I didn't take these pictures. (I would love to visit New York again. The last and only time was many years ago). I suppose that the pictures are more or less official promo material for the latest Bon Jovi album. I downloaded them from this web site:

http://fever-of-fate.com/index.php/caps/view/bon_jovi_have_a_nice_day/

I have all 138 images in a subfolder under the album folder. They make a nice slideshow when I listen to the album.

Edit: I just realized that the images are screens captures from the "Have A Nice Day" music video. The same site has images from some other music videos too.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755
Pages: [1]   Go Up