Topic: Sidecar files (Read 9008 times)

park · « **on:** June 03, 2009, 12:11:44 am »

Are the new sidecar files .xmp files? I'm not really sure what other software out there uses external metadata files, so maybe xmp is unecessary, but if Adobe Bridge could read the keywords etc. from .xmp files then it would make for a very smooth MC/Bridge workflow.

Is there any plan to extend this to other media types? ie. camera raw files, .wav audio files etc.
This would be really good for .wav files (most of my stock audio is in .wav format and they get moved around a lot. I've stopped organizing them in MC since they get disconnected from the library so often, and the library data always gets toasted).
It would be good for camera raw too. Especially if MC could see the image adjustment data inside the lightroom/bridge generated .xmp files and use it to generate it's own thumbnails.

How much of a unit are the source and sidecar files?

If I rename the containing folder (outside of MC) of a file, will MC still just grab all the metadata from the sidecar file as if nothing changed, or is the file path written into the sidecar file too?
If I rename files from properties on my video library, will all the sidecar files be moved/renamed too? Or would they be deleted and regenerated in the new location?
Are the sidecar files inside the MC library? Perhaps autostacked with their parent files?
Is the data still kept inside the MC library too or is it all moved out to the sidecar files? What happens if there is a descrpency between the two?

So many questions. A simple explanation of the new feature and how far you plan to take it will answer most of them though.
I imagine that polishing this feature off is going to be quite an undertaking, but I'm glad you're doing it. I feel safer knowing that the metadata is fixed somewhere other than in MC's potentially corruptable library files. Of course, tagging ability for mkv and mp4 files would make much of the benefit of this feature mute, but I'm sure there'll be files with untaggable containers for quite a while yet.

Alex B · « **Reply #1 on:** June 03, 2009, 05:29:02 am »

Quote from: Matt on June 02, 2009, 05:02:24 pm

7. NEW: Video tagging via sidecar support.

Thanks. This is a great new feature.

Quote from: park on June 03, 2009, 12:11:44 am

Are the new sidecar files .xmp files? I'm not really sure what other software out there uses external metadata files, so maybe xmp is unecessary, but if Adobe Bridge could read the keywords etc. from .xmp files then it would make for a very smooth MC/Bridge workflow.

Last time I checked Adobe had not actually created a standard for video xmp. If and when they do it it would be logical to start using the same format.

Quote

If I rename the containing folder (outside of MC) of a file, will MC still just grab all the metadata from the sidecar file as if nothing changed, or is the file path written into the sidecar file too?
If I rename files from properties on my video library, will all the sidecar files be moved/renamed too? Or would they be deleted and regenerated in the new location?
Are the sidecar files inside the MC library? Perhaps autostacked with their parent files?
Is the data still kept inside the MC library too or is it all moved out to the sidecar files? What happens if there is a descrpency between the two?

Apparently the sidecar files are just additional tag holders and not yet "stacked" anyhow. They are abandoned when the video files are moved. MC creates new sidecar files on the next tag write. Naturally a mechanism that moves also the sidecar files is needed.

Quote

Is there any plan to extend this to other media types?

I too would like to see sidecar support extended to all media types. In addition, I'd like to see separate settings for sidecar files in General > Importing & Tagging, for instance:

Sidecar files
Store tags in external sidecar files if the file type does not support tagging.
[] Video
[] Audio
[] Images
[] Documents

I think the sidecar files should be created independenly of the "Update file tags when file info changes" setting so that the other, "taggable", file types could still be protected from changes whenever preferred.

Yaobing · « **Reply #2 on:** June 03, 2009, 09:53:35 am »

The current sidecar file is xml, in the form of the My Movies xml file format, with additional MC tags added at the end. "_Original_Filename JRSideCar.xml" is the filename.

Currently it is only for video.

MC always looks for the sidecar in the same folder as the video file. Although the full path is written in the sidecar, it is always updated with the current folder.

On import, including "Update Library (from tags)", data from the sidecar is imported.

On "Update Tags (from Library)", the sidecar is created/rewritten.

darichman · « **Reply #3 on:** June 03, 2009, 07:32:47 pm »

I haven't updated to 14.0.7 yet - how does it handle the sidecar when moving/renaming files within MC?

I think this is a really great addition - thanks for the work!

leezer3 · « **Reply #4 on:** June 03, 2009, 07:43:12 pm »

Good addition in principle.
However, I'd put the sidecar files themselves inside a hidden folder, and probably set the hidden attribute for them as well.
Basically adds clutter if you're browsing via Explorer at present.

Cheers

-Leezer-

Alex B · « **Reply #5 on:** June 08, 2009, 05:55:35 pm »

Quote from: Yaobing on June 03, 2009, 09:53:35 am

The current sidecar file is xml, in the form of the My Movies xml file format, with additional MC tags added at the end. "_Original_Filename JRSideCar.xml" is the filename.

Currently it is only for video.

MC always looks for the sidecar in the same folder as the video file. Although the full path is written in the sidecar, it is always updated with the current folder.

On import, including "Update Library (from tags)", data from the sidecar is imported.

On "Update Tags (from Library)", the sidecar is created/rewritten.

All this was already clear to me and I think most of it was already explained in my reply (I actually tested the system in several ways before posting). To make the system usable you really would need to make the sidecar files to follow the video files when they are moved.

I also wonder why you did not include the other untaggable media types / formats before introducing the feature. They were mentioned and discussed in all old threads in which sidecar files were requested.

The option to switch them off is also essential. I am sure that some users are not going to be happy when they see the additional files on their drives.

In general, I think it would be better to introduce only a feature or two at a time and be prepared to continue the development persistently in each subsequent build before releasing the new feature outside the beta board. By this way each new feature would be quite finished before it is released to the public.

Now would be also be the correct time to start writing the MC14 manual in the wiki. I think the manual should be separate from the MC13 manual. If each new feature and major change would be documented when it is introduced here there would be no problems with inaccurate and dated information. The documentation would be relatively easy to write when the new feature is discussed and examined here and the beta testers could also say their comments or edit the new documentation immediately because the memories about the just tested feature are clear.

MrHaugen · « **Reply #6 on:** June 12, 2009, 02:44:19 am »

So, this is some kind of tag files that is stored next to the video files, so the tags are kept even if the library is lost right? Anything I've missed? Is it possible to have them in another directory all by them self, so it don't mess up my movie directory's with a lot of extra files?

flac.rules · « **Reply #7 on:** June 12, 2009, 03:33:16 am »

Quote from: Alex B on June 03, 2009, 05:29:02 am

I too would like to see sidecar support extended to all media types. In addition, I'd like to see separate settings for sidecar files in General > Importing & Tagging, for instance:

Sidecar files
Store tags in external sidecar files if the file type does not support tagging.
[] Video
[] Audio
[] Images
[] Documents

I think the sidecar files should be created independenly of the "Update file tags when file info changes" setting so that the other, "taggable", file types could still be protected from changes whenever preferred.

I agree . They should be created independantly of tag-settings, as i general rule i am much more careful about changing the files themselfs (taggign), than i am of just adding a sidecar-file. The problem witht his forum is that there is always someone who voices my opnion in a precise way before me, so i am reduced to writing posts of the "+1"-style

It is a great feature by the way.

Alex B · « **Reply #8 on:** June 12, 2009, 04:22:21 am »

A bug (I assume it's unintended):

Apparently this replaced native Windows Media Video tagging. MC13 and older have excellent support for reading and writing native Windows Media tags. Currently MC14 cannot even read WM tags that were created with older MC versions. There is no need to use sidecar files with WMV video files.

hit_ny · « **Reply #9 on:** June 12, 2009, 08:40:43 am »

I've been using the mpl version of sidecar files since v10 as i don't update tags for my audio files.

My understanding of the variety described in this thread is that its mostly for use with other apps that support it.

I like the idea of selecting a bunch of files, making changes in the library and having those changes automatically save back to their respective album folder sidecar file.

I'm currently trying to write a script that would do this with mpl..just as soon as i get my head around the automation COM stuff

Daydream · « **Reply #10 on:** June 12, 2009, 02:35:16 pm »

Quote from: hit_ny on June 12, 2009, 08:40:43 am

My understanding of the variety described in this thread is that its mostly for use with other apps that support it.

Err, wait, is that true? 'Cause I'm not that sure; and since I'm not that sure I still try to figure out what's the main use of this feature. You can export metadata for files that can't be tagged. Then what? Somebody has 15 MC databases and wants to (re)import stuff from sidecar files? I thought the main idea is to have a central point of control. Maybe somebody can clarify this for me.

rpalmer68 · « **Reply #11 on:** June 13, 2009, 12:53:10 am »

I'm looking at trying to get meta data from my TV recording software into MC using the sidecar support.

I understand from above MC is using mymovies.xml format for the file, and after googling around for a while I really couldn't come up with a definative list of what's supported in this format.

The best I could find was this example;

Code: [Select]

<Title>
 <LocalTitle>Shrek (2001)</LocalTitle>
 <ProductionYear>2001</ProductionYear>
 <RunningTime>90 min</RunningTime>
 <Description>When a green ogre called Shrek discovers his swamp has been 'swamped' with all sorts of fairytale creatures by the scheming Lord Farquaad, Shrek sets out, with a very loud donkey by his side, to 'persuade' Farquaad to give his swamp back.</Description>
 <DataProvider>IMDB.com</DataProvider>
 <DataProviderId>tt0068646</DataProviderId>
 <Rating>9.1</Rating>
 <Covers>
   <Front>folder.jpg</Front>
 </Covers>
 <Genres>Animated,Comedy,Adventure</Genres>
</Title>

Are these the only fields MC is looking at, or are there more it will read in?

Thanks
Richard

galahad1974 · « **Reply #12 on:** June 13, 2009, 02:00:36 am »

Daydream,
The primary value for me, is that in the event of a loss of the mc database (something that has happened 5 times in the years ive been using it, though never the fault of the software) I still have all of that data. I enter a huge amount of metadata for each file, and they move around alot. music is great, i just reimport it, and theres all my data. but for movies and TV, i enter Director, actor, plot synopsis, info on how i encoded the file, etc.. once you do that for about 100000 files, you really dont want to have to do it again. now, in that terrible event, i dont have to depend solely on having a good backup of my DB (even then if its 5 days old i loose alot of data). worst case it gets loaded from the sidecar. second i can move a vid file to another users system with the file and all my inf goes with it!

darichman · « **Reply #13 on:** June 13, 2009, 02:41:02 am »

I like the sidecar files.

1. Can we make them hidden by default.
2. They really need to be renamed/moved when the file is renamed/moved. I did a search of my drives and I have about a million orphaned JRSideCar files sitting about. This is the sort of thing that could really confuse/annoy/anger users if they don't know what's going on. The "Remove Empty Folders" dialog also never pops up if there's a sidecar file still sitting in there either.
3. Just my preference here... I'd prefer it if they were just named [Filename].xml

Thanks again for this feature, and hope to see it further developed

hit_ny · « **Reply #14 on:** June 13, 2009, 05:25:46 am »

Quote from: Daydream on June 12, 2009, 02:35:16 pm

Err, wait, is that true? 'Cause I'm not that sure; and since I'm not that sure I still try to figure out what's the main use of this feature.

I gathered as much from park's comment right at the top

Quote from: park on June 03, 2009, 12:11:44 am

Are the new sidecar files .xmp files? I'm not really sure what other software out there uses external metadata files, so maybe xmp is unecessary, but if Adobe Bridge could read the keywords etc. from .xmp files then it would make for a very smooth MC/Bridge workflow.

Quote from: galahad1974 on June 13, 2009, 02:00:36 am

Daydream,
The primary value for me, is that in the event of a loss of the mc database (something that has happened 5 times in the years ive been using it, though never the fault of the software) I still have all of that data. I enter a huge amount of metadata for each file, and they move around alot. music is great, i just reimport it, and theres all my data. but for movies and TV, i enter Director, actor, plot synopsis, info on how i encoded the file, etc.. once you do that for about 100000 files, you really dont want to have to do it again. now, in that terrible event, i dont have to depend solely on having a good backup of my DB (even then if its 5 days old i loose alot of data). worst case it gets loaded from the sidecar. second i can move a vid file to another users system with the file and all my inf goes with it!

This is also my main reason for using mpl but with audio files in my library.

The problem with this method tho is its manual and only good at the point of import which is when i save the mpl for the album in its respective folder.

Meaning that subsequent changes will only exist in the library but won't get saved back to the respective mpl unless explicitly told to do so. So currently the only way around this is to keep several library snapshots.

Daydream · « **Reply #15 on:** June 13, 2009, 06:10:58 am »

MPLs can be stored together and backed up. Covers can be stored together and backup up. Database backups are stored in one place and can be (further) backed up.

The reality of having 100000 files floating around everywhere, somewhat-somehow related to my collections scares me far beyond the point of being afraid to lose the current metadata. And as it happens I trust MC's databases, I don't know from where it came the sudden fear (OK, losing it 5 times is plain unlucky but I hope that's not the norm).

I do recognize the value of sidecar files in developing various relationship with third-party apps (if that will ever happen) but as backup, I can't work the math to say there is so high a risk percentage among MC users that a totally different backup solution needs to be deployed. So from my angle I'd appreciate an On/Off switch for the feature. If for you it's a big help - more power to you!

raym · « **Reply #16 on:** June 17, 2009, 07:40:00 pm »

These sidecar files are spectacular and has solved numerous problems for me. Thankyou!!!!

I totally agree with this though:

Quote from: darichman on June 13, 2009, 02:41:02 am

2. They really need to be renamed/moved when the file is renamed/moved.

Also, when deleting the file, the sidecar should be deleted also.

park · « **Reply #17 on:** June 17, 2009, 09:19:07 pm »

I think that the moving/deleting sidecar files is the most urgent feature required. There are going to be a lot of orphan files strewn all over the place if this doesnt get implemented soon.

Are there any plans to extend them to audio files, like m4a, m4p files?

Also, I read that Premiere Pro creates xmp files for avchd footage that it gets from cameras. It would be nice (understatement) if Premiere and MC were reading and writing in the same format.

hit_ny · « **Reply #18 on:** June 18, 2009, 01:29:20 am »

Quote from: park on June 17, 2009, 09:19:07 pm

I think that the moving/deleting sidecar files is the most urgent feature required. There are going to be a lot of orphan files strewn all over the place if this doesnt get implemented soon.

There needs to be way to move directories which take all the files with them.

Currently if MC does not know about a file then it gets left behind.

There are challenges which Matt highlighted earlier but still think they need to find a more powerful solution than what is available currently.

A programmable directory mover would be my request for MC 14.

park · « **Reply #19 on:** July 06, 2009, 06:36:08 am »

Glad to see that moving/deleting has been implemented. That works with the "Rename, move, copy files" tool too right?

I am also wondering if there are any plans to support .xmp sidecar files for raw images. Currently Adobe software makes such files for any raw image it has processed. So any edits to exposure, white balance, and of course the metadata must be stored in there I guess. It would be great if MC could understand all that data, and use it to generate its' thumbnails. And when the sidecar files have been updated outside of MC, it would be good if MC could see that and rebuild the thumbnail.

maid · « **Reply #20 on:** July 14, 2009, 02:05:46 am »

Can we stop these sidecar files?

Cheers

galahad1974 · « **Reply #21 on:** July 14, 2009, 03:33:55 am »

I hope they dont "stop" them. Its a feature a number of us have been requesting for quite a while, and its implementation is getting to the point where its very useable, being able to disable it would be reasonable, if not to much of a programming issue. Thanks for listening Development team!!!

maid · « **Reply #22 on:** July 14, 2009, 04:14:54 am »

Hello Again,

Well it would be a nice option to turn them off.

Not everybody wants them.

Cheers

Yaobing · « **Reply #23 on:** July 14, 2009, 09:42:13 am »

Quote from: maid on July 14, 2009, 04:14:54 am

Hello Again,

Well it would be a nice option to turn them off.

Not everybody wants them.

Cheers

Yes, I am working on making it optional.

Lasse_Lus · « **Reply #24 on:** November 25, 2009, 07:44:01 am »

hi yaobing, i like the sidecars

..but why is is it only for videos ? i myself really need it for documents

i hope you will get some time over for this...thanks

and i'm not alone

Quote from: Alex B on June 03, 2009, 05:29:02 am

I too would like to see sidecar support extended to all media types. In addition, I'd like to see separate settings for sidecar files in General > Importing & Tagging, for instance:

marko · « **Reply #25 on:** November 25, 2009, 07:49:02 am »

If we're not going to get native tagging, .tiff image files could do with sidecar support too

Scar3crow · « **Reply #26 on:** December 03, 2009, 05:27:09 am »

Is there a way to automatically set the sidecar files to be hidden? As much as I like what they do, I don't particularly like seeing them when I browse folders...

Lasse_Lus · « **Reply #27 on:** December 03, 2009, 06:21:48 am »

Quote from: Scar3crow on December 03, 2009, 05:27:09 am

Is there a way to automatically set the sidecar files to be hidden? As much as I like what they do, I don't particularly like seeing them when I browse folders...

if you dont like to see the sidecar files, you can set up a small batfile that runs every now and then

attrib +H yourpath:\nameofyourfolder\*.xml /s

and then set not show hidden files under folder options in explorer

Yaobing · « **Reply #28 on:** December 03, 2009, 07:57:35 am »

Sidecar option for other media types is on the table. It may take a while for me to get to it.

park · « **Reply #29 on:** December 03, 2009, 08:30:35 am »

How about using xmp files for raw images, and reading using xmp data to build thumbnails?

Also, are sidecar files tied to the "Rename, move, copy files..." tool? (do they get renamed with the media files? if not, do orphan files get automatically deleted?)

MrHaugen · « **Reply #30 on:** December 03, 2009, 04:45:51 pm »

I hope there's introduced a way to set the files as hidden, or not to be created at all. It's just really messy to see them in the folders all the time. Especially for those who browse by file structure only. I'd like to keep the files, but I'd much prefer it's own directory. Much like the cover art.

Lasse_Lus · « **Reply #31 on:** December 03, 2009, 05:25:19 pm »

Quote from: MrHaugen on December 03, 2009, 04:45:51 pm

I hope there's introduced a way to set the files as hidden,

mr haugen
you did not read my post or you don't like batch files ?

..it takes about 60 seconds to create, you can run it from within MC with via the scheduler if you want..

Quote from: MrHaugen on December 03, 2009, 04:45:51 pm

or not to be created at all.

you know you can turn the sidecars off ?!! (settings....)

Quote from: MrHaugen on December 03, 2009, 04:45:51 pm

I'd like to keep the files, but I'd much prefer it's own directory. Much like the cover art.

agree...but isn't the sidecars linked to some moviestuff that need's the sidecar beside the file ? but otherwise this would be preferable in some cases..

MrHaugen · « **Reply #32 on:** December 04, 2009, 03:41:36 am »

Quote from: Lasse_Lus on December 03, 2009, 05:25:19 pm

mr haugen
you did not read my post or you don't like batch files ? ..it takes about 60 seconds to create, you can run it from within MC with via the scheduler if you want..

you know you can turn the sidecars off ?!! (settings....)

agree...but isn't the sidecars linked to some moviestuff that need's the sidecar beside the file ? but otherwise this would be preferable in some cases..

Aaaaah. Sorry. Did not find the sidecar option earlier. It's great that we can turn it off. As it goes for the batch files, yes, you can create one and schedule it. I do not like all this things not built in though. And no, I don't like batch files when it's not really needed. Those things take time to set up. Especially when you have several clients. They can easily be forgotten when installing and reinstalling as well. It's not ideal. But sure, we'll survive.

Would love to have the xml files in a separate folder, or folder structure. Hiding the files is not a great option for those that needs to show the hidden files anyway.

morrison · « **Reply #33 on:** January 08, 2010, 10:41:00 am »

~~Ability to turn off sidecar files removed? Bring back please. Or please help me find it in settings and forgive my inattention)~~
I found )

INTERACT FORUM

Author Topic: Sidecar files (Read 9008 times)