INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Moving the library, and storage of additional files (Thumbnails)  (Read 5717 times)

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588

A small request really- Is there any chance of storing thumbnails along with the library with which they are associated if this library is moved to a custom location? This nearly caused a massive panic; My procedure was as follows-
My main library has been stored in F:\MCLib, pretty much from its creation.
My server is set to backup this whole folder nightly to avoid any data loss or accidental trashing :)
When I decided to reinstall Windows, I reloaded the library into MC, expecting everything associated with the library to be present, but the thumbnails (A lot of coverart added to films for example) were missing.

Fortunately, I've also set the server to backup the whole of my user folder, so no damage has been done, but I'd appreciate it if you would consider changing this, as it seems a little illogical at the moment :)

Cheers

-Leezer-
Logged

xsivfun1

  • Regular Member
  • Recent member
  • *
  • Posts: 40
  • Change this by choosing profile
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #1 on: June 03, 2008, 09:07:16 pm »

I support this suggestion 100%.  Can you?
Logged
Thx,

1 xsivfun1

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #2 on: June 04, 2008, 04:29:39 am »

You would need to backup the following locations:

1. The library folder (this is no brainer)

2. The cover art folder (If you use a separate location for the cover art files -- i.e. the folder that contains the actual source image files on the disk.)

3. Optionally the thumbnail cache folder. MC creates automatically a fast thumbnail cache from the linked or embedded cover art images, video content and image files. Losing the thumbnail cache should not be a problem because MC can recreate it, but if there is a big amount of cached thumbnail data including thumbnails that are created from video content the recreation process may take considerable amount of time. It is possible to make a backup copy of the thumbnail cache, but it must be perfectly synced with the library data -- i.e. the backup of the thumbnail cache and the backup of the library folder must be from the same "snapshot".

The default thumbnail cache location on XP (it is somewhat different on Vista) is: C:\Documents and Settings\[your user name]\Application Data\J River\Media Center 12\Thumbnails

The thumbnail cache location can be changed by editing the Windows registry, but it should always be on a fast local HD for obvious reasons.

-- If the thumbnail cache is not backed up MC's internal backup tool is fine for backing up the library data. If you have several libraries you need to run the backup tool on each library separately.

-- When the thumbnail cache is backed up MC must be closed and the library folder (or folders if you have more than one library) must be backed up at the same time.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #3 on: June 04, 2008, 04:59:18 am »

Quote
When I decided to reinstall Windows, I reloaded the library into MC, expecting everything associated with the library to be present, but the thumbnails (A lot of coverart added to films for example) were missing.

Are you speaking about the cached thumbnails or the actual cover art files?

How did you link the cover art files with your video files?

I have my cover art files in the same folders with the video files and they are automatically included in my backup archives. I don't let MC to rename and copy the video cover art images to the default cover art location.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #4 on: June 04, 2008, 07:20:36 am »

I've set MC to store coverart in a folder along with the library and file tags wherever possible, but any added coverart, whether it was a screenshot or an image originally vanished along with the thumbnail cache. (The source files are still there, but with random names; FWIW, the cover art image location is showing as F:\MCLib\Covers\unknown.jpg for everything, although the right image is showing)
Two facets to this really-
1. If there are two libraries, the same thumbnail cache/ coverart folder is used thoughout. I can't see any way to change this :) If I've chosen to move the library elsewhere, then IMHO logically I'd expect any associated files to be moved with it. Internal preferences and constants throughout two libraries can be stored in the same folder, but things like thumbnails should be kept with thier associated library :)
2. Constructing thumbnails for 20,000 odd video files is time-consuming to say the least. In this day and age, storage shouldn't be a problem; While I agree that thumbnails shouldn't be added to the default backup, it should at least be an option.

Cheers

-Leezer-
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #5 on: June 04, 2008, 06:56:39 pm »

I've set MC to store coverart in a folder along with the library and file tags wherever possible,

MC has three cover art naming and location options. Which one you have selected?

MC does not add physical tags to video files so the "in the file's tag" option is not applicable for video. (WMV is an exception because MC can use the same tagging system that is used with WMA audio files, but I have not tested if cover art inside file tag would work with WMV files.)

In any case none of the naming options is good for video files. They have been created for audio files long before using cover art with video files became common.

My advice (as I said) is to link the cover art files manually without letting MC to copy or rename the source image files.

Quote
but any added coverart, whether it was
 a screenshot or an image originally vanished along with the thumbnail cache. (The source files are still there, but with random names; FWIW, the cover art image location is showing as F:\MCLib\Covers\unknown.jpg for everything, although the right image is showing)

Something is not correct. I don't think MC has any code that would rename the files as "unknown.jpg".

If you mean by "screenshot" the "Use screengrab as thumbnail" feature I would recommend using the "Screegrab to file" option instead. Save the file to your preferred location and give it a descriptive name. After that link the image file manually with the video file. Use either the Right-click > Cover Art > Add From File command or place MC and Windows Explorer side by side and drag the image to Action Window > Tag > Image (expand the small thumbnail placefolder by clicking on it) when the video file is selected in MC. (I prefer the latter.)

Quote
Two facets to this really-
1. If there are two libraries, the same thumbnail cache/ coverart folder is used thoughout. I can't see any way to change this :) If I've chosen to move the library elsewhere, then IMHO logically I'd expect any associated files to be moved with it. Internal preferences and constants throughout two libraries can be stored in the same folder, but things like thumbnails should be kept with thier associated library :)

You can keep the cover art files in your preferred location. The thumbnail cache system is separate and even if they are sometimes erased they will be recreated automatically if the libary contains links to the original cover art files.

Quote
2. Constructing thumbnails for 20,000 odd video files is time-consuming to say the least. In this day and age, storage shouldn't be a problem; While I agree that thumbnails shouldn't be added to the default backup, it should at least be an option.

Constructing thumbnails from external cover art files is actually quite fast. Constructing thumbnails for 20000 video files by reading the video content would be very time-consuming.

I have only a few hundred video files, but I prefer to use externally linked cover art because thumbnail creation is much faster. If I want to use a screengrab I save it as a properly named file and link it manually so that it becomes an externally linked cover art file and will be included in my next backup run.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #6 on: June 05, 2008, 05:00:51 pm »

The option for cover art is "In a specified folder" (F:\MCLib\Covers); IMHO it's a lot more logical to store the coverart in a single central folder which is regularly backed up- It's easier to re-rip/ download any media files which are missing than it is to re-locate/ scan & edit my customised coverart if a drive fails.
I've also checked "Also store image in file's tag"

This screenshot shows what I mean about Unknown.jpg; All my video files look like this-



Final point, storing thumbnails/ covers along with the media is all very well, but IMHO the whole point of the database is to give me a single, centralised metadata store, which I can easily backup. If I loose a drive, the media files are easily replaceable but the metadata isn't. Storing the metadata along with the files makes everything a massive degree harder to backup.


Cheers

-Leezer-
Logged

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #7 on: June 16, 2008, 06:33:29 pm »

Minor bump- I've discovered the root cause of the 'Unknown.jpg'
This is in part a problem with my film tagging scheme- If the artist field is not filled (I've been using the simple title in some places, not filling in the title as it messes up other bits of my viewscheme for films), then the coverart file is stored as 'Unknown.jpg' in the specified coverart folder.
However, this poses a problem in that it overwrites any previous 'Unknown.jpg'
If the thumbnail cache is then regenerated, MC uses the newest 'Unknown.jpg' for all files with this specified coverart.

IMHO this small bit of behaviour should be classified as a bug. I'd also appreciate more views on my initial points :)

Cheers

-Leezer-
Logged

stefann63

  • Member
  • *
  • Posts: 3
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #8 on: July 07, 2008, 02:22:27 pm »

I would like more flexible library data base and thumbnail cache folder location.
We have separate accounts on a computer (me, wife and kids) and i want to share the library and same thumbnails cache to other users.
If I use same library database folder there will be more that one thumbnail cache folder – each per user.
When one user makes a change to the library, all other thumbnail cache folders must be recreated – I have 1000+  home HD video clips, 14000+ photos and this can take a day to build the thumbnails.

I would like to have the thumbnail cache stored in a library subfolder.
   D:\MC12\Library\FamilyLib\              - path to the library folder chosen in Library manager
   D:\MC12\Library\FamilyLib\Thumbnails       - thumbnails cache for that library
This will allow me to share and backup/sync them together.

Thanks.... :)
Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #9 on: July 07, 2008, 03:10:51 pm »

I would like to have the thumbnail cache stored in a library subfolder.
   D:\MC12\Library\FamilyLib\              - path to the library folder chosen in Library manager
   D:\MC12\Library\FamilyLib\Thumbnails       - thumbnails cache for that library

You can change the thumbnail cache location by tweaking Windows registry. The key is
HKEY_CURRENT_USER\Software\JRiver\Media Center 12\Properties\Thumbnails - Base Path. You would need change the value on each user accout separately.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

stefann63

  • Member
  • *
  • Posts: 3
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #10 on: July 07, 2008, 04:15:51 pm »

Thanks Alex B for Your fast response.
Modifying this registry entry will move the whole thumbnails cache folder for all libraries used by this user?

Logged

Alex B

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 10121
  • The Cosmic Bird
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #11 on: July 07, 2008, 04:59:53 pm »

Modifying this registry entry will move the whole thumbnails cache folder for all libraries used by this user?

It will not move an existing folder, but you can move the files manually outside MC:
Close MC, move the base folder's contents to the new location and edit the registry value.
MC will use the new thumbnail location when it is started again. The new location should (practically must) be on a fast local hard drive. Each library has a code named subfolder under the same basefolder so the thumbnails for all libraries will be moved. Personally I use a separate folder outside the library locations, currently C:\MC12tn\.

The registry setting is user specific (when a user logs into Windows, the user-specific registry settings become available in the "current user" registry branch). After you have moved the thumbnails and fixed the registry value for one user you need to separately fix each other user's registry before starting MC on any of the user accounts.
Logged
The Cosmic Bird - a triple merger of galaxies: http://eso.org/public/news/eso0755

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #12 on: July 08, 2008, 01:28:27 am »

Final point, storing thumbnails/ covers along with the media is all very well, but IMHO the whole point of the database is to give me a single, centralised metadata store, which I can easily backup.

The database is just a 'directory' of links to files, which is what you get with MC's library backup. It's very quick and does its job well. It's about as small as you can get it to be which helps when done frequently.

Cached thumbnails are another 'single centralised' store as well as the one main coverart folder (if you choose to do it this way).

These are the three areas you need to seperately reproduce if you change PC's. You could opt to just transfer your main coverart folder and then get MC to rebuild the thumbnail cache.


If I loose a drive, the media files are easily replaceable but the metadata isn't. Storing the metadata along with the files makes everything a massive degree harder to backup.

Depends which drive :)

if it's a media drive.

- Replace drive & config to right parititon letter.
- Sync back to the new drive from the backup.
above 2 steps done with MC closed for obvious reasons

You are good to go, MC is not even aware you lost a drive and believes nothing has changed.
Logged

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #13 on: July 08, 2008, 07:25:00 am »

I appreciate what you're saying, but a large part of my point is that there is absolutely no option to move the thumbnails from where they are stored by default within MC's options.
If I choose to move a library, IMHO it's a logical step to that anything which is intricately associated with it (Thumbnails etc.) would be moved along with the library- That's simply common sense :)

I don't know about your database, but I have over twenty custom fields, and large amounts of custom data added to each of these. There is no way this could be stored in any of my files tags; Some of this could be stored in a MP3's ID3 tags, but there's no way to store tag data in many of the odder video formats, and even when there is, this doesn't correspond to what I want to store.

Final point, there is no way on earth I could reliably back up over 4tb of media (About 270,000 files, probably a few more after the 30,000 I added at the weekend)  ;) All my drives are standard 500gb or larger desktop drives; There's some reduancy with the newer drives & the most important stuff (My photos & custom created stuff), but I simply cannot afford the cost of backing up and giving redunancy for this sort of data.I appreciate what you're saying, but a large part of my point is that there is absolutely no option to move the thumbnails from where they are stored by default within MC's options.
If I choose to move a library, IMHO it's a logical step to that anything which is intricately associated with it (Thumbnails etc.) would be moved along with the library- That's simply common sense :)

Cheers

-Leezer-
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42387
  • Shoes gone again!
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #14 on: July 08, 2008, 09:26:07 am »

I appreciate what you're saying, but a large part of my point is that there is absolutely no option to move the thumbnails from where they are stored by default within MC's options.
If I choose to move a library, IMHO it's a logical step to that anything which is intricately associated with it (Thumbnails etc.) would be moved along with the library- That's simply common sense :)

Thumbnails do a lot of disk access, so it's critical that they be on a fast drive.  Putting them on the Windows drive ensures this.  Since they can be rebuilt, they're not critical enough to be included in a backup (considering their large size).

We used to put thumbnails under the library, and it resulted in lots of angry customer threads about slowness of the program.
Logged
Matt Ashland, JRiver Media Center

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #15 on: July 08, 2008, 09:37:41 am »

Reading your posts from the top, the main issue from what i can tell is that you lost cover-art associations after reinstalling windows.

Correct me if i'm wrong but with a ton of them called  'Unknown.jpg', even if MC did backup the coverart as you requested, how will it be possible to relink them again ?

Don't they need to be uniquely named if they all live on one massive directory.

Quote
I don't know about your database, but I have over twenty custom fields, and large amounts of custom data added to each of these. There is no way this could be stored in any of my files tags; Some of this could be stored in a MP3's ID3 tags, but there's no way to store tag data in many of the odder video formats, and even when there is, this doesn't correspond to what I want to store.

Not sure why you mentioned this wrt to your request as its to do with the library.

But we have something in common here :)

I don't update tags on my files at all (for reasons mentioned in earlier posts),  relying entirely on the library to hold all relevant metadata. So no different to image or video files that don't support tagging, cept it applies to everything in my library. Done this for a cpl of yrs now and even tho i've been through countless crashes the data has always been good.

I have this theory that if MC crashes while its writing to file tags then the library can be left in an uncertain state as i 'think' it only updates in the library once the filiewrite completes.  So if it gets interupted during a file tag write, then that entire field can go blank !! dunno if this is currently the case but it happened with older versions

But with no file tag updates, library updates are very quick, so if there is a crash, chances are it happens after the library has already been updated, ergo less corruption or only loss from the last library save, which kicks off fairly regularly. I mean the main library here.

That's why i like the fact the library backup is as small as it is.

..and of course i cannot rely on update library from tags.

So i save to mpl the album's details for every album in its own folder as an extra precaution.
Logged

leezer3

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1588
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #16 on: July 08, 2008, 11:05:52 am »

Thumbnails do a lot of disk access, so it's critical that they be on a fast drive.  Putting them on the Windows drive ensures this.  Since they can be rebuilt, they're not critical enough to be included in a backup (considering their large size).

We used to put thumbnails under the library, and it resulted in lots of angry customer threads about slowness of the program.

I'm not (At least I hope!) suggesting that they be included in the main library backup; My thumbnails are well over 300mb, and this is rather large for a standard weekly job :)
As I'd very deliberately moved the main library, it took me by surprise that there were things missing; The whole point of moving the library was to avoid this.

It's also good to see a developer point of view. I take your point that they need to be on a fast (IE. Local probably drive), but the current situation clearly isn't perfect.
With that in mind, I'd probably implement a compromise solution of some degree-
How about a library backup wizard? At the moment, the library backup is a rather esotoric tool, with no real documentation. All it needs is a little secondary GUI, with the ability to select which elements of the library are backed up and the amount of space this backup will take up. I'd also implement an option somewhere to change the thumbnail storage path, with an appropriate warning as to the location.

Quote from: hit_ny
Reading your posts from the top, the main issue from what i can tell is that you lost cover-art associations after reinstalling windows.

Correct me if i'm wrong but with a ton of them called  'Unknown.jpg', even if MC did backup the coverart as you requested, how will it be possible to relink them again ?

Don't they need to be uniquely named if they all live on one massive directory.

I think this is a bug- See my second post on 'Unknown.jpg' :)
http://yabb.jriver.com/interact/index.php?topic=46808.msg322319#msg322319

Quote from: hit_ny
Quote from: leezer3
I don't know about your database, but I have over twenty custom fields, and large amounts of custom data added to each of these. There is no way this could be stored in any of my files tags; Some of this could be stored in a MP3's ID3 tags, but there's no way to store tag data in many of the odder video formats, and even when there is, this doesn't correspond to what I want to store.

Not sure why you mentioned this wrt to your request as its to do with the library.

But we have something in common here Smiley

I don't update tags on my files at all (for reasons mentioned in earlier posts),  relying entirely on the library to hold all relevant metadata. So no different to image or video files that don't support tagging, cept it applies to everything in my library. Done this for a cpl of yrs now and even tho i've been through countless crashes the data has always been good.

I put this in because people keep on suggesting I store data either with the files or in thier tags. This simply isn't feasible!


Cheers

-Leezer-
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: Moving the library, and storage of additional files (Thumbnails)
« Reply #17 on: July 08, 2008, 11:43:07 am »

I think this is a bug- See my second post on 'Unknown.jpg' :)
http://yabb.jriver.com/interact/index.php?topic=46808.msg322319#msg322319

yes, much clearer now.

So if  MC gives a unique name to ever jpg that has empty fields.

'Unknown(1).jpg', 'Unknown(2).jpg' etc

or some date-time combo which would be quicker if the cover folder was very big.
Logged
Pages: [1]   Go Up