INTERACT FORUM
Windows => Third Party Plug-ins, Programs, and Skins => Topic started by: jmone on January 29, 2016, 05:14:11 am
-
Last Updated: 21 Aug 2016
Background: MC has all the power you need to maintain and modify data within it's GUI. A few members however want to be able to import/export their library to tools like excel, some want to the modify and update this information, and others would like to be able to use a spreadsheet to add new items (for existing meta data holdings, dummy files etc)
This utility has 4 Options (see pic)
- Export the Library From MC: Exports your entire Library (or just the items Selected in the MC GUI) to a MPL and optionally converts to a Unicode/Tab Delimited TXT file which you edit in Excel etc
- Convert a MC MPL Playlist File: Converts an existing MC MPL (playlist) File to a Unicode/Tab Delimited TXT file which you can edit in Excel etc
- Update MC Library: Updates existing Items in your MC Library from a Unicode/Tab Delimited TXT file
- Create/Import a MC Playlist File: Creates a MC MPL (playlist) File from a Tab Delimited TXT file (optionally asks it you want import) that lets you add new items to the MC Library
Examples: Read the posts below for examples of what it can do.
- Adding Items to MC from a Spreadsheet (http://yabb.jriver.com/interact/index.php?topic=102906.msg714492#msg714492) - useful if you want to add items with no associated file such as managed a physical disk collection
- Updating Items in MC from a Spreadsheet (http://yabb.jriver.com/interact/index.php?topic=102906.msg714516#msg714516) - useful if you want bulk update vales for existing items.
- add your "off line" media library to MC (http://yabb.jriver.com/interact/index.php?topic=102906.msg714653#msg714653) by 8139david
- How to update a non editable System Field like Date Imported (http://yabb.jriver.com/interact/index.php?topic=102906.msg714728#msg714728)
- others???
Notes, Limitations, Future Stuff:
- It is an AutoHotKey script so it only runs under Windows
- It takes me about 2mins to export my 30K library to a Unicode/Tab Delimited Text file and will use a bunch of Memory in the process
- It will create a spreadsheet of all used Library Fields (eg if you have not populated a field it will not appear), many of these fields are system fields
- It does not handle some extended characters sets well, so keep an eye on your files - It should now work with Unicode Chars
- *BEWARE* it WILL change or add stuff to your Library and there is no "Undo".... so backup your Library before you modify it.
- To increase the compatibility with various TXT editors, Excel etc, any field in MC that has Tabs and Carriage Returns Line Feeds embedded in the data will be shown as {TAB}, {CR}{LF} in the exported Text File. So in an Excel you may have a Description that looks like "This Movie version is:{CR}{LF}{TAB}- The Original Theatrical Release{CR}{LF}{TAB}- Filmed in Sydney Australia" all in one cell, and in MC it will appear as:
This Movie version is:
- The Original Theatrical Release
- Filmed in Sydney Australia
"Key" Vs "File Key": In the MC GUI, the primary key for all entries is shown as "FileKey", yet during the import and export this field is oddly called just "Key". So if you are using the "Update MC Library" the Column Header in your TXT file should be called "Key" (not "FileKey") or nothing will match and be updated
Setting Up Media Server Access: This script uses MC's Web Services (MCWS) to Import, Export, and Update your Library. For this to work, you will need to ensure that:
- Media Server is running
- In "MC --> Tools--> Options--> Media Network" check both "Use Media Network" and "Authentication" with "Username/Password"
- When you first Run Data Fiddler, it will ask you to enter your Connection Details, and allow you to test it works. If you ever need to change these just delete the "MC Data Fiddler.ini" file in the same directory.
Options:
- I've attached both a 32 and 64 bit compiled Version and the original AHK Script. The 64 Bit will handle much larger libraries but will only run on 64Bit versions of Windows
- The first time you run it, it will ask you for your MC Login Details. The second time the menu will be presented
Let me know of any bugs!
Thanks
Nathan
-
Thanks! I'll try it soon, probably tonight :)
-
How does your program differ from the built-in tool "File>Export Playlist (where you export "all files" to a .cvs and select either "visible fields" or "all fields")?
The JRiver export tool seems to provide a nice spreadsheet file with each File listed on a separate row with its tag properties, that can then be opened/edited in Excel.
I think it is the "import process" that is missing.
Thanks for your work!
-
Nothing yet, I was playing with how to import dummy files to MC and then update them, so ended up looking for how to convert MPLs as they contain the data differently to the Native CSV Export from MC. One thing I could not get was an export that included the FileKey data when creating a CSV using "Included Fields = All Fields" (you have to use Only Visible Fields), which makes it harder to then later do an update. MPL export has this info, but I should do a comparison of what other fields different between the two methods.
Anyway, in summary this is actually a MPL Converter (it asks MC for a MPL of everything and parses that)
-
Update: 29JAN16
- Update: Cleaned up the output a bit
- New: Added the ability to a Convert a MPL to a Unicode/Tab Delimited TXT file
- New: Added the ability to Convert a Unicode/Tab Delimited TXT File to a MPL
Adding Items to MC from a Spreadsheet: This is where it now gets a bit more useful. You can now use any editor that can create a "Unicode/Tab Delimited Text File" (just about anything but in particular Excel for most) and convert it to a MPL that when Imported into MC will add the items to the Library.
1. Create Your Spreadsheet (see pic) - Make sure in each cell the First Row (the Header Row) contains a valid "MC Library Field" name that you want to populate (non-MC Library Field names will be ignored by MC). Add you data for each item you want to create in the rows below (note: now all lines needs to have data for all fields, as blank data cells will be ignored when creating the MPL).
2. Save Your Spreadsheet (see pic) as a Unicode/Tab Delimited Txt File
3. Run MC Data Fiddler and select "Create/Import a MC Playlist File (from Tab TXT File)" and when prompted select the file you created in Point 2.
4. MC Data Fiddler will then create a "MC_Import.mpl" file (and optionally offer to Import it for you). I'd recommend that you Open it up in Notepad and have a look that all seems well then manually import this file by Dragging it onto the MC Menu Bar --> Import. Or Right Click on the newly created MPL file --> Media Center --> Import. Either way, your Data should be in the MC Library (see pic)
You will also be able to see what was imported into MC each time by looking at the MC--> Playlists--> Imported Playlists--> "MC Fiddler Import YYYMMDDHHMMSS". You can delete these playlist files without deleting the items that they imported.
*** WARNING *** - Be careful, backup your library first as there is no Undo if something goes wrong. Try a small import first.
-
Thanks! It works.
Also, I just realized how to import by yourself a mpl playlist for several movies within JRiver :)
See this post: http://yabb.jriver.com/interact/index.php?topic=102843.msg714506#msg714506
The key is that Media Server has to be running.
By the way, in Data Fiddler, you have some fields wrong, eg "MediaType" should be "Media Type", and "SubMediaType" should be "Media Sub Type", see attached field for a correct mpl playlist.
-
Glad it works. Where do you see "MediaType" and "SubMediaType" (I only see "Media Type" and "Media Sub Type)?
The only oddity on the naming convention I've found so far is that in MC, the primary key is shown as "File Key" but in the MPL it is marked as "Key"
-
My bad. I must have made the mystake myself in the txt file when I first tried your program.
The fourth option does create a mpl playlist with the same fields as the txt file. So as you write above, at least some of those have to be labelled exactly like in JRiver.
-
Update: 29JAN16b
- New: Added the ability to Update MC from a Tab Delimited TXT File
Updating Items in MC from a Spreadsheet: You can now use any editor that can create a "Tab Delimited Text File" (just about anything but in particular Excel for most) and use it to update a MC Library directly.
1. Create Your Spreadsheet (see pic) - Make sure in each cell the First Row (the Header Row) contains valid "MC Library Field" name that you want to CHANGE (non-MC Library Field names will be ignored by MC). One of the Fields MUST be labeled "Key" and have the "FileKey" reference shown in MC. This is an oddity, MC shows it in it's GUI as "FileKey", but Media Center Web Services (MCWS) use "Key" and hence so does the MPL and TXT files.
Add you data for each item you want to modify in the rows below.
2. Save Your Spreadsheet as a Tab Delimited Txt File
3. Run MC Data Fiddler and select "WriteInfo : Update MC Library from Tab Delimited TXT File" and when prompted select the file you created in Point 2. Your Data should then be updated in the MC Library (see pic)
*** WARNING *** - Be careful, backup your library first as there is no Undo if something goes wrong. Try a small update first.
-
My bad. I must have made the mystake myself in the txt file when I first tried your program.
The fourth option does create a mpl playlist with the same fields as the txt file. So as you write above, at least some of those have to be labelled exactly like in JRiver.
No Probs - you should now be able to mass import your physical Disk Library in MC withe all the data.
Let me know of any oddities!
Thanks
Nathan
-
Apart from any bugs, the only other thing I can think of that may make sense is instead of extracting all items from MC, I can make it a Search String (eg Audio only etc). That said, MC is way more powerful as you can always display what you want in the MC GUI and the export that as a MPL which Data Fiddler can then process into a TXT file for modification and later update.
-
Wow! powerful stuff, thank you Nathan. Hope to have time to try this this weekend!
-
I'm trying to use your program (latest version) to import dummy movies whose information is specified by a tabbed text file.
What keys should I give them?
But there is a simple solution: from the txt file, use your program to create the mpl playlist. Then manually do a "Import playlist" within JRiver. This imports the corresponding movies, in this case dummy files Movie1.vob, etc
-
So you don't actually need any dummy files at all. It will create the entries in MC without anything needing to be populated in the FilePath unless you want some file to be actually played. Does this make sense?
-
Once I have my "dummy" movies imported, I want to do a "Get Movie and TV information" and store the information, including the Cover art.
Don't I need a dummy file for that?
-
Nope, no need for an actual dummy file at all.
...in other words, create a Tab Delimited file (from say Excel) with all the fields (and corresponding data) you want populated, and then use Data Fiddler to create the MPL that you can then check and import.
Once it is in MC, you can then do all the "normal" stuff like Get TV and Movie Info but play the file and it will add the extra Meta Data.
The only think I'm not sure off is how the Coverart will be saved (as we have had a storm and my main Internet is down so hard to test!)
-
That's what I was checking at the same time.
You're right!
No problem with the cover art: it is saved where the options say it should be saved.
In my case: C:\Users\David\AppData\Roaming\J River\Media Center 21\Cover Art\Videos
-
Beyond playing the inexistent movie, the other thing it cannot do, of course, is "update tag from library".
The latter is useful if you want to have a sidecar for each movie.
But then of course you need a dummy movie file.
I feel more secured with a sidecar: easy to reimport if the library gets corrupted.
But I suppose the solution is to do proper library backups?
-
Excellent! On the "backup" you can always just export it as an MPL as it is the same thing.
-
In the description of the program (first post), you should mention this:
- Media Server has to be running
- with Authentification selected ; the username and password asked by Data Fiddler are those specified in the Authentification of the Media Server
-
Excellent! On the "backup" you can always just export it as an MPL as it is the same thing.
You're right! Thanks!
-
Thanks - Updated the OP re: Library Server
Also, can you test (as with my net down I can not). If you populate the "Filename" field in a TXT file with unique data like "C:\MyDummyMovies\Movie1.MKV" (even though it does not exist) and import via a MPL with Data Fiddler, can you then create a side car file in that folder (the folder will need to exist)?
PS - I could also make Data Fiddler to auto import the MPL file (so you don't have to do it manually) but I'm wary without a user eye balling it first. What do you think?
-
Nope, no need for an actual dummy file at all.
...in other words, create a Tab Delimited file (from say Excel) with all the fields (and corresponding data) you want populated, and then use Data Fiddler to create the MPL that you can then check and import.
Once it is in MC, you can then do all the "normal" stuff like Get TV and Movie Info but play the file and it will add the extra Meta Data.
The only think I'm not sure off is how the Coverart will be saved (as we have had a storm and my main Internet is down so hard to test!)
hmm just thinking out loud, as I haven't had the chance to try this, lets say I want to ONLY update certain fields for existing media .... are you saying that if I export a spreadsheet, and only want to bulk overwrite two fields, I'm assuming that I would NOT remove any of the other fields data and just update the two fields correct? If the other fields were removed, I assume it will nuke whats in them? or will it just update what is on the spreadsheet.
I'm asking that because One of the Fields MUST be labeled "Key" and have the "FileKey" reference shown in MC. This is an oddity, MC shows it in it's GUI as "FileKey", but the MPL and TXT files use "Key".
If the filekey is present in the spreadsheet could for example could it be possible to create say a 10 field template to share update/metadata? Is this "filekey" acting as a Unique record ID? Thinking if this could be a tool to sync metadata between non networked libraries/media files
Also, I was thinking of creating non-system Unique IDs by "track/album") -- there are posts on this in the forum, already, but using excel to populate this then reimport could be a pretty neat solution.
oh BTW -- what happens with system fields -- is this a way to overwrite ones that you normally can't (like date imported?)
-
...so it will update only the fields that are present in the spreadsheet and will leave the rest as they are :)
The issue with updating another library is that the file key will need to also exist and match as it will update the data for a file key on whatever library you are connected to. If the filekey on this other library is not the same you will push data into it's fields that will not correspond. The update function will not create an entry if it does not exist, it just updates one that does... Does that make sense?
-
...and just tested on the System Field "Date Imported" and I could update it just fine! No idea on the data format however (I just cut and past from other file... the value I used in the spreadsheet was "139493357" and MC now shows it as 16/03/2014).
-
FYI - the date format of Value 1 = 1/1/1970 so seems to be using Unix/POSIX/Epoch time in MC incremented by seconds
Edit - I notice if you stuff up a field value like "Date Imported" MC then shows in this field "not imported"
-
Thanks - Updated the OP re: Library Server
Also, can you test (as with my net down I can not). If you populate the "Filename" field in a TXT file with unique data like "C:\MyDummyMovies\Movie1.MKV" (even though it does not exist) and import via a MPL with Data Fiddler, can you then create a side car file in that folder (the folder will need to exist)?
PS - I could also make Data Fiddler to auto import the MPL file (so you don't have to do it manually) but I'm wary without a user eye balling it first. What do you think?
I tried. Then I can see only a few tags, I suppose it categorize it as Data by default. So it seems problematic.
Even if I add a Media Type before hand, it just imports a playlist, without accompanying movie.
I tried all the options.
You get a sidecar only by Update Tags from Library, and this works only if you have an existing, non-empty dummy file.
Concerning your PS:
I tend to think like you.
But maybe you could add this as a 5th option?
-
...so it will update only the fields that are present in the spreadsheet and will leave the rest as they are :)
The issue with updating another library is that the file key will need to also exist and match as it will update the data for a file key on whatever library you are connected to. If the filekey on this other library is not the same you will push data into it's fields that will not correspond. The update function will not create an entry if it does not exist, it just updates one that does... Does that make sense?
thats terrific !... of course you'd need the same key, but if you create an external library from a restore originally, this would be the same .. I'l try to test later today
thanks this could be very cool for me
-
Concerning edit first post:
Try it yourself, but I think you must be authenticated for update library from txt file to work.
Also, it would be nice if the name of the mpl file was more varied.
Thus, if toto.txt is taken as input, the output would be toto.mpl.
And if JRLibrary is exported, then it's given a unique name, eg JRLibrary-2016-01-39-13-49 (including date with hours, perhaps even seconds).
-
Thanks for the effort jmone!
-
Update: 31JAN16
- Changed: When Files are Created the FileName will be appended with a DateStamp in the format of YYYYMMDDHHSS (24 hour) as well use the input file name where appropriate.
- Changed: The MPL playlist header is "MC Fiddler Import YYYYMMDDHHSS" so MC will see each as a unique import (else it would ignore the import) and you can now see what was imported each time in the MC --> Playlist --> Imported Playlists.
Also, Updated the OP with David's observations regarding Authentication must be on for the update library feature to work.
-
Many thanks again, jmone!!!
I'll try the new version tomorrow.
-
Update: 31JAN16a
- Changed: Updated the OP Instructions and the two example posts
- Changed: Updated the layout and working of the Main Menu
- Added: Added the option to import the MPL directly into MC once created from a TXT file
- Fixed: Updated the parser to handle blank lines in data sets better (normally at the end of files)
-
Very nice work!
Here are two pics for the function 'Create/Import a MC playlist (from text file)'.
I first ran it with essai.txt (with dummy vob files in the folder), then with essai2.txt (without dummy files).
As discussed, dummy files are not needed, unless one insists on creating later corresponding sidecars.
essai.txt
Name Filename Director Media Type Media Sub Type toto tutu
Movie61 Movie61.vob A B Video Movie a b
Movie71 Movie71.vob C D Video Movie x y
essai2.txt
Name Director Media Type Media Sub Type toto tutu
Movie161 A B Video Movie a b
Movie171 C D Video Movie x y
The MC playlist obtained from essai2 is essai_2_20160131092341.mpl:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<MPL Version="2.0" Title="MC Fiddler Import 20160131092341" PathSeparator="\">
<Item>
<Field Name="Name">Movie161</Field>
<Field Name="Director">A B</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">Movie</Field>
<Field Name="toto">a</Field>
<Field Name="tutu">b</Field>
</Item>
<Item>
<Field Name="Name">Movie171</Field>
<Field Name="Director">C D</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">Movie</Field>
<Field Name="toto">x</Field>
<Field Name="tutu">y</Field>
</Item>
</MPL>The fields "toto" and "totu" are created in the playlist BUT they are not imported in JRiver's library, since they do not correspond to existing fields.
Since Movie 161 and Movie 171 are already tagged as Video and Movie, one can do immediately a "Get Movie & TV Show information".
Otherwise, one would have to specify manually in JRiver first their Media Type "Video", then their Media Sub Type "Movie", before being able to Get Movie Information.
-
Here is some basic information about AutoHotkey, which was used to make the script and the Windows exe program:
https://autohotkey.com/
'AutoHotkey (AHK) is a free, open-source macro-creation and automation software for Windows that allows users to automate repetitive tasks. It is driven by a scripting language that was initially aimed at providing keyboard shortcuts, otherwise known as hotkeys, that over time evolved into a full-fledged scripting language.'
See also https://autohotkey.com/docs/Tutorial.htm and https://en.wikipedia.org/wiki/AutoHotkey
-
Thanks for the testing and I'm glad it works! Given that the topic of how to manage a Off Line Media Collection in MC comes up from time to time, could I trouble you to write up a little Guide in this thread for future users?
Using Dummy Files: Before I ripped all my disks to HDD, I too used dummy files to get entries into MC... but now we can create entries without actual dummy files I've been pondering if it is worth the effort and complication at all, and it has some downsides. I've been just playing with dummy files and one of the issues is that MC will try to parse them to populate data fields (which will all be the same and not reflective of your actual library should you want to use these tags). Important things like Duration as well as technical details like Dimensions, FPS, Compression, Audio Tracks etc. These will all be wrong if you use Dummy Files.
If the only issue of NOT using dummy files is that "Update Tags from Library" will not work, I have a couple of suggestions:
- Instead of using Sidecar files, just select all the entries and use MC to create a MPL for them all
- I could make a MC Menu Item called "Update Tags from Library (with no file)" and it would create a sidecar for each of the selected (or all) items without a file path (or some other logic)
Also please let me know of any oddities or enhancements.
Thanks
Nathan
-
Thanks, I will write this little guide, reusing some of my posts.
The added functionality you suggest would be nice.
Currently, JRiver allows you to export only your entire libraray as a mpl playlist.
So it would be great to export any selection, if desired with only some fields selected, as a mpl playlist.
And why not as a tabbed txt file too.
But maybe this is better integrated directly in JRiver?
Well, if you can partially add the functionality to Data Fiddler, it's great!
So that it can update the original mpl and txt files from the Library.
-
Actually, you can already create a single MPL for just selected Items (but it is all fields minus the Key for some reason). Just highlight the ones you want then File--> Export Playlist then select:
- MPL
- Selection
If you want just the fields in you view (including FileKey), then use File--> Export Playlist then select:
- Text File (CSV)
- Only Visible Fields
- Selection
I'll think about the "Update Tags from Library (with no file)" option from within MC some more.... but it won't be for a couple of weeks as I'll be travelling soon. Also if the aim is to have a backup of the meta data, the better option is probably just to back up the library itself!
-
Here is some basic information about AutoHotkey, which was used to make the script and the Windows exe program:
https://autohotkey.com/
'AutoHotkey (AHK) is a free, open-source macro-creation and automation software for Windows that allows users to automate repetitive tasks. It is driven by a scripting language that was initially aimed at providing keyboard shortcuts, otherwise known as hotkeys, that over time evolved into a full-fledged scripting language.'
See also https://autohotkey.com/docs/Tutorial.htm and https://en.wikipedia.org/wiki/AutoHotkey
...and as with all my Utils, I post both the compiled versions and the script itself. Feel free to edit the script as you please but be warned there is little (if any) notes in the scripts as I'm way to slack (though it makes it hard for me to remember why I'm doing stuff a couple of months later!)
-
You're right. I had missed the option of exporting a mpl playlist for a selection.
I had seen the various options for csv. (Why not offer all of them for mpl export, in JRiver?)
Good travels!
-
Thanks!, also you have a day for any serious bugs for me to look at then I'm off to China and the Harbin Ice Festival (https://en.wikipedia.org/wiki/Harbin_International_Ice_and_Snow_Sculpture_Festival) (among other things)... at -25C it should be "brisk"!
Edit - Opps: "January is the coldest with the lowest temperature dropping to -38 C (-36 F)" very Brisk!
-
Using Dummy Files: Before I ripped all my disks to HDD, I too used dummy files to get entries into MC... but now we can create entries without actual dummy files I've been pondering if it is worth the effort and complication at all, and it has some downsides. I've been just playing with dummy files and one of the issues is that MC will try to parse them to populate data fields (which will all be the same and not reflective of your actual library should you want to use these tags). Important things like Duration as well as technical details like Dimensions, FPS, Compression, Audio Tracks etc. These will all be wrong if you use Dummy Files.
That's true, this information is incorrectly populated with dummy files.
When I have time, maybe I'll use your tool to clean up my library of "dummy" movies:
export mpl from JRiver for this selection
suppress the corresponding movies from the library (after a backup)
convert it to tabbed txt fil using your tool
open in excell and suppress columns containing false information
convert back to mpl and import
The only thing is that currently, the cover art of each movie is in a special folder with its dummy file (ifo or bdmv), eg:
Inception > VIDEO_TS > VIDEO_TS.jpg
I would need to rename the covers and move them around in some way or other (all cover art back to JRiver's default folder for covert art).
-
I found that MC will store the Coverart in it's Coverart Folder if the filename does not exist (or next to the file if it does), so no probs there (plus you can can always get it again with the "Get Info" tool).
The more I think about it the less I'm liking using Dummy Files, it all seems hard, gives wrong info, and it's not used by MC anyway. The only "downside" is not being able to create a sidecar file using "Update Tags from Library", and what does that give you anyway (well just an individual copy of the meta data from the library) and in reality what are you every going to use it for Vs a Library Backup or a full MPL pull?
-
...another thought, the "Export the Library from MC" option dumps the entire library in MPL format to memory that I then use to create the TXT file. I could also write this entire file out in it original form as "MC_Library_YYYYMMDDHHSS.MPL" and then you would have all data for all fields (including the File Key) in MPL format.
-
Update: 31JAN16b
- Changed: The "Export the Library from MC" option will save the full library as "MC_Library_YYYYMMDDHHSS.mpl" and optionally ask if you want to convert it to a Tab Delimited TXT file for easier editing in Excel etc
Note: Unlike the internal MC GUI option to export as an MPL this will also output the "Key" information for all items.
-
Thanks.
Exporting the key would be a nice feature for JRiver itself.
This way you can select some files, and export playlist as you want, while still having the key.
-
I'm trying to recreate my library of dummy movies, like I said earlier.
I have a problem when opening the txt file in Libre Office Calc: some fields are too long.
Indeed, for some reason, in some cases, all the fields are recognized as one field, and several movies are put into that single field.
Cf. pic.
Any idea to solve this problem?
-
I manage all my digital media on JRiver: films and albums.
They are on some hard disc from which they can be played from JRiver.
In that sense, they are "on line" media.
Now, I also have concrete, physical media: dvds, blurays, cds, lps...
Those are not present on any hard disc accessible by JRiver.
In that sense, they are "off line" media.
I would like all my media, whether on line or off line, catalogued by JRiver.
How can I incorporate the off line media?
Answer:
- create a tabbed text file containing the relevant information about each media;
- then use Data Fiddler to update JR's library with the corresponding information.
Cf. the more detailed explanations concerning Data Fiddler given by jmone in this post:
http://yabb.jriver.com/interact/index.php?topic=102906.msg714492#msg714492
For movies in particular
JRiver's "Get Movie & TV Show Information" works really very well, using only the title of the movie.
So suppose I want to catalog my own dvds for the first time.
Then I simply create a tabbed text file like:
My dvds.txt
Name
Star Wars
Startrek
or
My dvds (with Type and Sub Type).txt
Name Media Type Media Sub Type
Star Wars Video Movie
Startrek Video TV Show
Then I use Data Fiddler: "Create/Import a MC Playlist File (from Tab TXT File)".
The MC playlist obtained from My dvds (with Type and Sub Type).txt is this:
My dvds (with Type and Sub Type)_20160131165203.mpl:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<MPL Version="2.0" Title="MC Fiddler Import 20160131165203" PathSeparator="\">
<Item>
<Field Name="Name">Star Wars</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">Movie</Field>
</Item>
<Item>
<Field Name="Name">Startrek</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">TV Show</Field>
</Item>
</MPL>
When imported in JRiver, it looks like the first attached pic of JRiver.
Remark: since Star Wars and Startrek are already tagged as Video and each given an appropriate Media Sub Type (Movie or TV Show), I can immediately do a "Get Movie & TV Show information". Otherwise, I would have to specify manually in JRiver first their Media Type "Video", then their Media Sub Type, before being able to Get Movie & TV Show Information.
Of course, this procedure also works if I have more information in my text file.
Suppose I have already collected a lot of information about my blurays (cf attached pic of excell spreasheet).
Then I have this text file:
blurays.txt
Name Commentaire Genre Media Type Media Sub Type Producer Critic Rating Director Actors IMDb ID
Solaris Vu au cinéma en 1992 Drama; Science Fiction; Adventure; Mystery Video Movie Viacheslav Tarasov Tmdb 7,5 / 10 Andrei Tarkovsky Donatas Banionis; Natalya Bondarchuk Tt0069293
Jules Et Jim Vu au cinéma en 1985 Drama; Romance Video Movie François Truffaut Tmdb 7,3 / 10 François Truffaut Jeanne Moreau; Oskar Werner; Henri Serre; Marie Dubois; Sabine Haudepin; Vanna Urbino; Serge Rezvani; Anny Nelsen; Michel Subor Tt0055032
This yields this mpl file -- see attached pic for imported playlist in JRiver:
blurays.mpl
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<MPL Version="2.0" Title="MC Fiddler Import 20160131171835" PathSeparator="\">
<Item>
<Field Name="Name">Solaris</Field>
<Field Name="Commentaire">Vu au cinéma en 1992</Field>
<Field Name="Genre">Drama; Science Fiction; Adventure; Mystery</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">Movie</Field>
<Field Name="Producer">Viacheslav Tarasov</Field>
<Field Name="Critic Rating">Tmdb 7,5 / 10</Field>
<Field Name="Director">Andrei Tarkovsky</Field>
<Field Name="Actors">Donatas Banionis; Natalya Bondarchuk</Field>
<Field Name="IMDb ID">Tt0069293</Field>
</Item>
<Item>
<Field Name="Name">Jules Et Jim</Field>
<Field Name="Commentaire">Vu au cinéma en 1985</Field>
<Field Name="Genre">Drama; Romance</Field>
<Field Name="Media Type">Video</Field>
<Field Name="Media Sub Type">Movie</Field>
<Field Name="Producer">François Truffaut</Field>
<Field Name="Critic Rating">Tmdb 7,3 / 10</Field>
<Field Name="Director">François Truffaut</Field>
<Field Name="Actors">Jeanne Moreau; Oskar Werner; Henri Serre; Marie Dubois; Sabine Haudepin; Vanna Urbino; Serge Rezvani; Anny Nelsen; Michel Subor</Field>
<Field Name="IMDb ID">Tt0055032</Field>
</Item>
</MPL>
The field "Commentaire" has been ignored since it does not correspond to a Library field of JRiver.
(If I wanted to avoid that, I should either have renamed that field "Comment" in the text file, or created a new field "Commentaire" in JRiver.)
And for albums?
I haven't tried it myself. But clearly more information is needed, at least: Artist, Album, and Name for each song or piece.
PS: If the files you're cataloging are offline, then JRiver will give them some unchangeable attributes, such as zero (or empty) Duration, Dimensions, File Size. This cannot be avoided, since these attributes are read directly from available files --and in the case of off line media, there is no real file with the proper properties that JRiver can access.
-
Thanks David. I had a quick look at the TXT and MPL files you sent. The TXT file opens fine in Excel so it may be just Libre Office Calc...
When I look at the files it is this section where Libre seems to be thrown:
MPL
<Field Name="Description">"The Three Ages," Buster Keaton's First Feature-Length Film After A Number Of Comedy Shorts, Is His Parody Of Griffith's "Intolerance." Keaton Tells Three Parallel Stories About The Perils Of Romance, One Set In The Stone Age, One During The Roman Empire, And One During The 20th Century.</Field>
TXT
"The Three Ages," Buster Keaton's First FeatureLength Film After A Number Of Comedy Shorts, Is His Parody Of Griffith's "Intolerance." Keaton Tells Three Parallel Stories About The Perils Of Romance, One Set In The Stone Age, One During The Roman Empire, And One During The 20th Century.
I'm wondering if it is the double Quote Marks (") around "The Three Ages," as these don't appear for me when imported into Excel (maybe Libre is seeing them as a Delimiter or an Escape Char then reading everything from there on into one cell?). Try deleting the Quotes one at a time from the start of "The Three Ages" in the txt file and importing into Libre to see what causes it?
I'm hoping to look at the multi line parsing this arvo with some luck....
-
Update: 01FEB16
- Changed: Updated how the Import/Export code works to work in reading and writing multiple lines with a data field and handle embedded Tab, and Carriage Returns/LineFeeds. This should increase the compatibility with various TXT editors, Excel etc, as any field that has Tabs and Carriage Returns Line Feeds embedded in the data will be shown as {TAB}, {CR}{LF} in the exported Text File. So in Excel you may have a Description that looks like "This Movie version is:{CR}{LF}{TAB}- The Original Theatrical Release{CR}{LF}{TAB}- Filmed in Sydney Australia" all in one cell, and in MC it will appear as:
This Movie version is:
- The Original Theatrical Release
- Filmed in Sydney Australia
If editing the TXT file, you can added/remove {CR}{LF}{TAB} as needed
Let me know of any other "weird" characters that cause issues.
-
Example: Updating a System Field (Date Imported). It's hard to update a System Field in MC such as "Date Imported". Here is an example of how to do it. In Excel, create a spreadsheet with just three columns as follows and enter the following data and formulae.
Use Format Cells--> Date on Col C so you can enter a Human Readable date format. Col B should remain as a Format Cells--> General but just past in the formulae "=(C2-25569)*86400" which will the convert the Excel Date Format to Epoc Date and it will be "1454284800" in this case.
A B C
Key Date Imported New Date
12345 =(C2-25569)*86400 1/02/2016
etc
etc
Save as a Unicode/Tab Delimited TXT File and use Data Fiddler to "Update MC Library" and the new date(s) will be set for those (file)Key. There is no need to delete Col C as it will be ignored given it does not match any existing MC Library Field
-
If editing the TXT file, you can added/remove {CR}{LF}{TAB} as needed
Let me know of any other "weird" characters that cause issues.
Ugg the joy of different progs and formats... FYI - Unlike a plain text editor, it seems Excel likes to add extra " marks around a value if you try to use " in the value (so you end up with multiple " in the TXT file that then updates MC.). It also can drop a ' at the beginning of a value
-
@8139david
careful with messing with system fields Date Imported is one that can get tricky. There is a new expression tha will aloow you to port over the coded system field to a more useable date string
You seem to be just getting started with JRiver, so not meaning to sound condescending, but I have had some problems messing with system fields in the past .. best to leave them alone and use custom fields IMHO. This is getting into some pretty advanced stuff here
here's an example of what I'm referring to for date imported http://yabb.jriver.com/interact/index.php?topic=86834.msg713807#msg713807 (http://yabb.jriver.com/interact/index.php?topic=86834.msg713807#msg713807)
PS why not rip at least the audio cds of yours?? seems to defeat the purpose a bit, no?
-
@Arindelle
The system fields is entirely jmone's idea: so he should be warned, not me :)
Some don't have the time to rip all their cds, but they might want to catalog them.
-
@Arindelle
The system fields is entirely jmone's idea: so he should be warned, not me :)
oh I'm sure Jmone knows all about it - he knows more about it than I do ... he's one of JRiver's Yodas
- *BEWARE* it WILL change or add stuff to your Library and there is no "Undo".... so backup your Library before you modify it.
just trying to help. Its a third party plugin .. sort of like jail breaking an iphone :) Powerful stuff, so a heads up to you is all.
Have fun with it.
@Nathan, I've got a cool 15 field template set up ... seems to work well. Thanks for the effort
-
Yup - you could mass update "unintended" data easily (esp with any Non Editable Field in MC's GUI) so always backup your library first. Keep in mind that as it is a published I/F, MC "knows" what you are doing and hence your risk is low on actually corrupting the MC DB itself. The worst is you roll back to your last DB backup to reverse any unintended changes (FYI - I've been in SI for a long time, when modifying data using the API calls is pretty OK... directly modifying the underling DB, now that can be risky!).
On updating the Date Imported there are valid reasons (eg if you imported with a bad system clock). My example was more to show how to covert from a Human Readable date format to that used by MC. The same Method could be used to change any of the Date fields.
Arindelle, I'd be interested to hear what your template does (as others would I'm sure) and if there are any tweaks I need to make.
-
By the way, JRiver assigns a duration of 0 seconds to the "dummy" movies (without dummy files) I import using Data Fiddler.
And Get Movie Information (with overwrite information) doesn't change that.
Indeed, as might have been expected, all the fields which would be read from a real video file are gray and <empty> on the Tag pane.
-
- New: Added the ability to Update MC from a Tab Delimited TXT File
Updating Items in MC from a Spreadsheet: You can now use any editor that can create a "Tab Delimited Text File" (just about anything but in particular Excel for most) and use it to update a MC Library directly.
1. Create Your Spreadsheet (see pic) - Make sure in each cell the First Row (the Header Row) contains valid "MC Library Field" name that you want to CHANGE (non-MC Library Field names will be ignored by MC). One of the Fields MUST be labeled "Key" and have the "FileKey" reference shown in MC. This is an oddity, MC shows it in it's GUI as "FileKey", but Media Center Web Services (MCWS) use "Key" and hence so does the MPL and TXT files.
Add you data for each item you want to modify in the rows below.
2. Save Your Spreadsheet as a Tab Delimited Txt File
3. Run MC Data Fiddler and select "WriteInfo : Update MC Library from Tab Delimited TXT File" and when prompted select the file you created in Point 2. Your Data should then be updated in the MC Library (see pic)
*** WARNING *** - Be careful, backup your library first as there is no Undo if something goes wrong. Try a small update first.
I'm trying to use this function.
I've created a txt file (attached). But when I run DataFiddler, I get the error message on the attached image.
What should I do/try?
-
should look something like this
(http://i.imgur.com/fdTLztt.png)
in a spreadsheet program try saving as a *.CSV text file. Use the semi-colon as the field delimiter. JRiver uses the ";" as its default separator
you just have a text file in columns here. It can't parse anything
-
Hi David, thanks for the data file. I'll have a look and get back to you, it could be the treatment of the "-", "é", "è", "ç" characters but I'll need to test. The rest of it looks OK to my eyes. Try breaking the file into a couple of batches and see how that goes. I should be able to test properly this weekend however.
Thanks
Nathan
-
in a spreadsheet program try saving as a *.CSV text file. Use the semi-colon as the field delimiter. JRiver uses the ";" as its default separator
you just have a text file in columns here. It can't parse anything
Thanks but I think David's format is correct. DataFiddler is using Tab Separated for input files to then make a MCWS call using CSV from that.
-
Thanks to both for your help!
Arindelle's tip made the command apparently go through.
BUT it didn't update any field at all (including the field that was supposed to change, namely, 'First').
-
The text file has to be Tab Delimited for DataFiddler. Changing the Delimiter to another (like ";") will not work. Your best bet is to cut it into some some batches till we find the particular data that is not working (I'm guessing it is one of the extended char sets)
-
I tried all possible variations, on a very small file, nothing worked.
When the fields are separated by tabs, I get the error message above.
When they are separated by ; or , the scripts seems to run through but nothing is updated.
Some examples (I tried many variations):
Key,First
170510,PierreLaurent
170511,PierreLaurent
170512,PierreLaurent
"Key";"First"
"170510";"PierreLaurent"
"170511";"PierreLaurent"
"170512";"PierreLaurent"
-
Thinking the keys I indicated where perhaps not good, I tried similar things with my first 3 keys (as recovered by exporting again my library and reading the keyfile in the txt file generated on demande), same problem: no change is made.
To be exact, I just used this text file (and variations, including tabs, with no success):
Key,First
1,tata
2,toto
3,tutu
PS: I also tried using a normal field, eg:
Key,Comment
1,tata
2,toto
-
I'm having a look now and I'm seeing the same failure msg from MC. (note: it's not the delimiter - it HAS to be a TAB)
-
OK - Found the issue. It is because MC is expecting to update more than one Field when I use the call I'm using and fails as this data set only has one field to update.
This works for example:
Key First NotAnything
1 tata a
2 toto a
3 tutu a
This will then update the values in "First" and ignore the "NotAnthing" field as it does not exist.
I've made a new version that now allows single fields in the TXT file when updating MC
-
Update: 20FEB16
- Fixed: Added Workaround where MC Would reject Updates to the Library where the TXT file only had one field
-
Thanks! Now it works!
-
Thanks but I think David's format is correct. DataFiddler is using Tab Separated for input files to then make a MCWS call using CSV from that.
hmm okay sorry for steering you wrong.
@Jmone why then did this work for me ... I'm going to dig up my "template" and see what I did
-
David - glad it works! had me stumped for a bit as well :) I had no idea the call needed more than one field.
Arindelle - no probs. My script breaks the text file into fields by TAB delimited so I'd be surprised if anything else works (as it would be just part of the the data to be imported into MC).
-
@Jmone - thanks, this utility is great fun to work with and helped me to finally enhance my collection of classical music!
Combining the downloaded library file with a data base of composers and the era they were active, I can now browse much my collection by era like Baroque, Renaissance etc. Love it!
I found two quirks with the program:
- Update of fields in MC: In my local CSV, I maintain the composers data as, for example, "Georges Bizet (1838–1875)". When uploaded to MC, this is changed to "Georges Bizet (1838 311875)". This is easily fixed in MC itself (find/replace on the 'composer' fiel), so no big concern. Curious though why that would happen.
- Special characters: the downloaded library text file can not handle special characters, in particular letters that are often used in other languages. In the world of classical music, names that contain such characters are quite commen (e.g., Frédéric Chopin, Antonín Dvořák). I haven't figured out how to deal with these entries during download and upload. Probably by living by the 80/20 rule and be happy with what I have :)
Again - great tool, and I already have many other ideas how I can now enhance my library!
-
Thanks for the feedback. I know...
- It does not handle some extended characters sets well, so keep an eye on your files
...but I've not had the time to go through to find a better way of handling extended character (for example your "-" is probably a long dash or "Unit Separator" symbol not a minus sign). One day... maybe :)
Nathan
-
Update: 14AUG16
Updated the OP with an new version that should handle Unicode chars better. Please let me know of any issues esp if the Unicode chars are now working correctly.
Notes on Changes:
- Will now work with just one Field (needed at least two previously)
- Better support for Unicode Characters
- Better support for handling commas in a field
- If you make a field blank in an Update it will now make that field blank in MC (instead of ignoring it)
- Added a CheckBox in the Menu to only Export items that are Selected in the MC Menu (instead of the Whole Library)
"Export the Library From MC (Creates TXT File)" : This will download the MC Library (in UTF-8) format then ask if you want to create a TXT Files (with a UTF-16 Header so it will open in Excel Correctly). You should be able to just drag this TXT file into Excel for editing and it will be recognised as a "Unicode" File. When you save in Excel you should see a MSG saying "Some features in your workbook might be lost if you save it as Unicode Text, Do you want to keep using that format". The answer is Yes.
Creating a Blank Spreadsheet : If you are creating a Spreadsheet in Excel manually (not from a Library Export) and have Unicode chars then in the Save As box select "Save as Type: Unicode Text (*.txt)"
-
Update: 20AUG16
Notes on Changes:
- Increased Max Memory Limit to handle larger library (can now use up to the Max Available memory for a 32-Bit App if needed, eg 4GB)
- Fixed typo that had broken the creation of the TXT file correctly
-
Update: 20AUG16b
Notes on Changes:
- Another Change to try to avoid Out of Mem error with large librarys being read. It is slower but it prepares one item at a time rather than the whole Library
- Fixed some parsing logic on the creation of MPL files
-
Still issues with Memory usage on large libraries. Here is a little MemTest prog that will
- keep increasing the RAM used till the script stops
- Just move the Error Window out of the way to see the final amount of Mem User (or look in Task Manager)
- Press ESC to close the app and release the memory
FYI
- The most I get on the 32Bit Version is about 1,800 MB
- The 64Bit Version just keeps going and Windows seems to manage it just fine
Looking at the result, I'm thinking for the DataFiddler Script, I'll just post both a 32 and 64-Bit compiled version of the script.
-
Update: 21AUG16b
Notes on Changes:
- Gave up on the Out of Mem issue with Large Libraries with my 32-Bit version of the script and instead now also include a 64-Bit version as well. The 64-Bit Version should have no issues with running out of Memory but will only run on 64Bit versions of Windows
- Reverted to the previous was of reading the MPL into Memory as it is quicker (and the alt method I tried did not help with Out of Memory errors anyway).
- Included a "fix" for "&" becoming "&" in MPL Exports
-
Thanks @Jmone! This works perfect now: special characters are downloaded properly and my large library is exported completely.
The addition to download only highlighted files from MC is most useful: saves time (quicker download, no deleting of unwanted files in excel required) and reduces risk to screw up files you did not even want to touch in the first place.
Off to do some cleansing in Excel now :)
cheers
-
Thanks for all your testing and patience! I'm glad it is working... well until the next bug is found that is :)
-
I'm going to close this thread, as future updates will be part of MC Tool Box (http://yabb.jriver.com/interact/index.php/topic,106802.0.html)