Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[MrC]

I've created an extensible program that uses scriptlets to manipulate, and do work based on, MC fields.  This allows you to select a set of files in MC, run the scriptlet, and have the new calculated values automatically stored back into one or more MC fields.  Think of it like MC's expression language on steroids.  It also allows you to perform additional actions based on values collected from the selected files.

The scriptlets are written in Perl, as is the main program pscriptor.pl.  This means pscriptor.pl is cross-platform - it can be used on Windows, Mac OS X, or Linux.  The Perl script can be written on the command line, or stored for later in a file (especially useful for more complex scripts).

I've included a some example scriptlets:

   - sample scriptlets that explain how to code a scriptlet (simple and complex cases)
   - get a music file's embedded cover art dimensions
   - convert names in the form of Last, First into First Last
   - uppercase the first letter of a name, except for those sub-names in an exception list
   - swap values of the form role:Artist into Artist:role
   - reports on fields with case differences
   - creates m3u playlists from files in a view
   - randomly fills Playback Range with a given range for true random track sampling
   - acoustic fingerprint audio analysis for track detection and musicality information
   - detect and retrieve dates from arbitrary fields
   - copy the topmost field from an arbitrary grouping of files, and push it downward in the group
   - copy and merge all the values in a field, removing duplicates
   - provide a sequence of numbers
   - make a random playlist of N files and start playback to a specified zone
   - obtains a list of each file's full playlist paths
   - test / verify a media file
   - copy files with flexible folder and file names (including sequence #'s) and create M3U8 playlists
   - makes particles and chapters from ripped Blu-ray discs with metadata lookups from ChapterDB.org

I'll add more samples as requested, the idea being that the body of scriptlets will increase over time.  If you know Perl, you're free to experiment, create and share your own.

I've had the idea of doing this for a while, but was prompted by a request in another thread.  The problem in that thread was to globally reverse a list of role:Artist values to Artist:role.  Since MC doesn't have global substitution with regular expressions, pscriptor.pl has been born of necessity to get the job done.

Download the attached zip file, and unzip the items to a folder (right-click > Extract All).  See the README files inside the folder.  Refer to the instructions inside.  The instructions and descriptions that follow this post while perhaps useful will often be inaccurate as new releases or pscriptor are posted.

MC19 is required currently.

Here are a couple of quick screen movies showing it in action:
- - See the Demo movies at the download site in the Instructions

[Arindelle]

Hey Bravo Mr. C! You've done it again! This will have to wait a bit until after Christmas for me to test ... up to my eye-balls in things to do.

For those of you who might have missed this, check out Mr. C's AMG tag tool http://yabb.jriver.com/interact/index.php?topic=84268.0

Hmm a "steak" nominee? Careful Glynor, better start pumping out those brilliant guides of yours.

Or will this years Porterhouse be chicken filets ? 

[JimH]

MrC,
Thanks very much (again).  You could post a notification over on the MC19 board and point people here.  I think a lot of people will find it interesting.

Jim

[MrC]

Version 1.03 posted:
    - Change: field names specified with -f are now case insensitive
    - Fix: ensure fields specified with -f are in the view (and on the
      clipboard).

[MusicBringer]

MrC,
Thanks very much (again)

Hear! Hear!

[Fred1]

Just discovered your new project - sounds very interesting!

Does this work on a Mac, too (MC-wise)?
Is ist possible to send clipboard content to MC-Mac? I tried that, but failed.
None of the apps like Keyboard Maestro or Quicksilver will work for me on MC-Mac.
So i wonder if there is a way to do this.

[MrC]

Does this work on a Mac, too (MC-wise)?
Is ist possible to send clipboard content to MC-Mac? I tried that, but failed.
None of the apps like Keyboard Maestro or Quicksilver will work for me on MC-Mac.
So i wonder if there is a way to do this.

Yes, it works on the Mac.  That's where I do the development.

You need Mac::Pasteboard and a couple more modules installed.  I use MacPorts to install most packages, and that also requires installing XCode.  So the bootstrap process is larger than for windows, where ActivePerl has the pre-compiled binaries.

Ask offline if you want to move forward (send me a PM).

[MrC]

So this is kinda fun.

Over in this thread, which is part of a duplicates detection scriptlet, I realized I can create another scriptlet which does track and metadata lookup entirely by acoustic fingerprinting lookup (via echoprint).

So I started playing with this.  Let's see where it goes.

Returned from the lookup are a number of useful fields, for example;:

 "songs":
      "title": "Utopia",
      "artist_discovery": 0.39565332550298865,
      "artist_name": "Goldfrapp"
      "id": "SOVLXBI13167714CE8"
      "score": 77,
      "artist_id": "AR8K3HD1187B9B9CA9",

"audio_summary":
      "key": 2,
      "energy": 0.68320400000000003
      "liveness": 0.065955
      "tempo": 91.018000000000001
      "speechiness": 0.026283000000000001
      "acousticness": 0.013006
      "mode": 0
      "time_signature": 4
      "duration": 258.02620999999999
      "loudness": -6.4770000000000003
      "audio_md5": "88e2e52a8deaa86f83243898f0675213"
      "valence": 0.17689199999999999
      "danceability": 0.43351400000000001

So this might be a useful way to lookup untagged tracks for some identification, or for other audio-analysis info.

[MrC]

I've updated pscriptor to version 1.04.  See the Changes file for more info.

[connersw]

Maybe I read too much into it, but in the Read Me Step 1 where it says to configure the MCWS_server in pscriptor-config.txt, I assumed this would be IP Address:52199.  This gave me a time out error.  It ran fine just using localhost:52199.

Thanks for this and Random Playback scriptlet.  Will give me more reasons to waist time playing with MC.

[MrC]

An IP Address works fine, so long as it is the correct one.  The hostname "localhost" just gets translated by the operating system DNS libraries into the IP address 127.0.0.1.  Perhaps you used in external (WAN) IP address instead of your LAN private address?

[connersw]

Yes, that is what I was doing.  LAN works.

[MrC]

As mentioned a few posts above, I toyed with some acoustic fingerprinting as a scriptlet.  I have it working on a the Mac, but I had to compile an executable as part of the toolset.  To make this scriptlet available on Windows, I'll have to do the same (and probably make binaries available to you to download).  So I'm wondering, does anyone have any interest in this?  Here is the scriptlet in action - it correctly identifies all the tracks from Moby's Play album:

   https://dl.dropboxusercontent.com/u/87189402/AcousticFingerprint.mp4

The fingerprint software I'm using provides the following 13 values, which I'm pushing back to MC along with the track Name and Artist:

Key/Value: key: 4
Key/Value: energy: 0.325266
Key/Value: liveness: 0.51388699999999998
Key/Value: tempo: 171.143
Key/Value: speechiness: 0.031287000000000002
Key/Value: acousticness: 0.182311
Key/Value: mode: 1
Key/Value: time_signature: 4
Key/Value: duration: 218.57288
Key/Value: loudness: -10.608000000000001
Key/Value: audio_md5: 53afc1cb8d9a5b6984d573eb4a24fefa
Key/Value: valence: 0.037865999999999997
Key/Value: danceability: 0.085136000000000003

No Album value is available, as a track could be present on many albums.

[bunglemebaby]

So I'm wondering, does anyone have any interest in this?  

Yes.
Of course, that also comes with the question of how much effort does it require to build a separate executable for Windows? Don't spend hours on my account and all that....

Is the Acoustic Fingerprint the one generated by MC or do we need to do that separately (using what utility if so)?

-Jon

[MrC]

No, the fingerprinting software is the open source Echoprint:

    http://echoprint.me/

and the particular program required is codegen.  It performs the analysis, and then the scriptlet submits the analysis data to a web server for look-up.

They don't provide binaries for codegen because of the licensing issues with certain software packages.

[MrC]

After completing a bunch of projects, I turned my attention back to the Acoustic Fingerprinting scriptlet.  I now have it compiled for Windows, and just ran some tests; it is working fine. It seems reasonably fast, taking about 13 seconds to identify all 10 tracks in an album.

I need to product-ize the scriptlet a bit so that you can customize how the returned values are mapped into MC fields, and whether or not you want Artist and (track) Name populated.

[alspoll]

Editor's amendment:

Split from a topic over here:

   http://yabb.jriver.com/interact/index.php?topic=76147.msg599135#msg599135

Glynor,

Is there a way for this to be used with Mediainfo to pull back stream count for audio and text (subtitles)?

What I want to do is to identify video files where there are multiple audio and subtitles streams so I can remux them to remove them to save on some harddrive space (I envision adding 2 fields in jriver audio stream count and subtitle stream count and then creating a smartlist afterwards with > 1 to review).

I looked at mediainfo and see it supports command line, but is beyond my understanding on how to use it and your program to populate MC19.

TIA,

AL

While glynor has not yet had a chance to respond...

What types of files are you wondering about?  Most of mine are mkv files, and are single-stream, and I've never bother pulling the subtitles, etc., so I perhaps don't have the necessary data for evaluating what the mediainfo command line results would look like in your case.

I have a mix of file types, (avi, mkv, mp4, etc).

I have a lot of foreign films and anime with dual audio streams that I want to identify and remove the English dub streams.

[MrC]

Provided that the mediainfo output indicates this, if glynor's tool(s) cannot do what you need, my pscriptor tool will be able to; it just needs a simple scriptlet to call mediainfo and gather/process the results.  It can then fill in MC's fields with the desired results.

What I'm not clear on yet is if a single mkv, avi, or mp4 file contains multiple streams (and therefore mediainfo on that file will output the relevant info), or if your multiple streams are contain each in a single file (and then mediainfo would have to process several files at once and that output collected and processed).

If you're interested in going the pscriptor route, I'll rip some multi-track files w/subtitles and see what the results are.

You could also download, install and run the mediainfo command line tool, and evaluate the output against a single media file.  It is easy to run.  Just open a command window, and probably increase the height of the window, type the full path to where you stored the mediainfo executable, and pass it a full path to a media file.  Use the --full option too.   Example:

   c:\mediainfo\mediainfo.exe  --full M:\Movies\somemovie.mkv

[alspoll]

Here is an example...

Karas  - The Prophecy has 2 audio streams (English and Japanese) and 2 subtitle streams (2 English) ...

I attached a screeenshot of media info for this file..

Thanks for your help.

[MrC]

Can you download the command line tool and get the output of that file?

You can redirect the output to a file for easy posting.  Just add:

   > output.txt

to the end of the command line.

[alspoll]

hmm...  file is coming back blank...

i entered this into the command prompt. files is created, but nothing was written. i validated i downloaded the cli version of the tool.

e:\mediainfo\mediainfo.exe  --full M:\Movies\Karas  - The Prophecy\Karas  - The Prophecy.mkv > output.txt

[MrC]

I ripped a multi-channel audio w/subtitles disc to DVD.  The command line output indicates there are two text streams and two audio streams, so it seems easy enough to grab / store this info.

Please indicate if you're interested in pscriptor, and I'll write you the scriptlet to provide the info you need.

[MrC]

You'll need double quotes w/file paths contain whitespace:

  e:\mediainfo\mediainfo.exe  --full "M:\Movies\Karas  - The Prophecy\Karas  - The Prophecy.mkv" > output.txt

[alspoll]

Thank you, that worked.

Here is the output.

General
Count                                    : 284
Count of stream of this kind             : 1
Kind of stream                           : General
Kind of stream                           : General
Stream identifier                        : 0
Unique ID                                : 264338026680190436507004806273882760741
Unique ID                                : 264338026680190436507004806273882760741 (0xC6DDA6F02E2A7F713A0DFDB7C105A625)
Count of video streams                   : 1
Count of audio streams                   : 2
Count of text streams                    : 2
.
.
.
.

[alspoll]

MrC,

I am interested in using pscriptor.

I appreciate your help with the scriptlet.

Thank you.

[MrC]

Ok, will do. I'll have something for you after I get back from getting some lunch...

[MrC]

I've posted version 1.05 of pscriptor.

Changes Version 1.05:
    - Change: updated the README instructions to better explain unzipping the pscriptor
      contents into a folder.
    - New: scriptlet RandomPlaybackRange can obtain its playback duration from the view.
    - Fix: scriptlet RandomPlaybackRange's duration was off-by one.
    - Fix: scriptlet MakePlaylist was not properly encoding the playlist file names in Windows.
    - New: scriptlet GetIDV3Tag can pull a tag (frame) from an IDV3-tagged media file.
    - New: scriptlet GetMediaInfo will call the mediainfo executable and return the number of audio and text streams.
    - New: scriptlets can now return hash of field key/value pairs which will update MC fields.

[MrC]

@alspoll: Version 1.05 just posted provides you with a new scriptlet GetMediaInfo.  Download and install it.  

Here's how you use it:

   perl pscriptor.pl -c pscriptor-config.txt -E GetMediaInfo -f "your field name"

where your field name is the MC field name you want filled (double quoted if it contains spaces).

It will populate the field with a value like:

   audio_streams: 2; text_streams: 3

which you can split into two other expression fields automatically with the expressions:

   Audio Stream Count:
        removeleft(listitem([your field name], 0), 15)
   Video Stream Count:
      removeleft(listitem([your field name], 1), 14)

The script assumes that mediainfo.exe is in your Windows PATH somewhere.  If it is not, do any one of the following:

  - update your Windows PATH to include mediainfo's folder
  - move the mediainfo.exe file somewhere in your PATH
  - edit the Scriptlet/GetMediaInfo.pm file and set the $exe_path_win variable to specify the full path to mediainfo.exe.

[alspoll]

Thank you very much for the addition of the mediainfo scriptlet.

[MrC]

You're welcome.

This was just a quick scriptlet to a) get you the info you needed, b) to also serve as a template for other uses of mediainfo, and c) show in general of how easy and quick it is to write scriptlets.

[nitephlight]

This is fantastic work can't wait to try it out. You're truly a valuable asset to this community, thanks again!

[MrC]

Blush.

[MrC]

I've published version 1.06, which includes the new AcousticFingerprint scriptlet.  This scriptlet will analyze audio tracks, returning track name, artist, and various musicality information.

Changes Version 1.06
    - Fix: disable debug output for scriptlet hash returns (internal change).
    - New: first release of the AcousticFingerprint scriptlet for audio identification and analysis.  You will need two
      executables, one ffmpeg, and the other the acoustic analysis program called 'codegen'.  See the comments at the
      top of the Scriptlets/AcousticFingerprint.pm file.   Codegen and server-based analysis is provided by the open
      source Echo Nest project (http://echonest.com/).  Example usage:

          perl pscriptor.pl -c pscriptor-config.txt -E AcousticFingerprint -f 'Name'

      The field selected with -f doesn't matter, as the scriptlet does not use it; it just has to be one in the view (I'll eliminate this requirement later).

[nitephlight]

Hm downloads in OP missing or tapatalk f#+/+ing with me as usual lol

[MrC]

The download is there.

[glynor]

It would be quite trivial to use a modified version of the scriptlet to update your Date Recorded field from mediainfo's output.  Let me know if you want a scriptlet for this...

I really need to spend some time checking it out.  I'm sure I can figure out the scriptlet design to make my own.

The one thing I was concerned with (I admit, I haven't tried it at all) is from your initial descriptions it wasn't clear if I can easily call pscriptor to act on a particular file by filename.  If so, I can easily automate any scripts I want to do with pscriptor to happen automatically via MCAutoQueue (which, when combined with a Tag On Import rule, will just happen "automatically").

It looked like it was certainly pretty flexible, but I saw a lot of references in your thread to "selecting files" (interactively in MC's UI).  I can already do most of the things I want manually, but I'd love to be able to use pscriptor (instead of writing my own generic tagging tool) in an automated fashion.

Before I dig in, set my mind at ease... Show me a command line that acts on a single file by [Filename]. 

[glynor]

https://dl.dropboxusercontent.com/u/87189402/get%20cover%20art%20size.mp4



 

Solution.

[MrC]

Pscriptor works by grabbing the Filename from the clipboard's CSV table.  It will either use an already copied selection of MC view rows, or will call MCWS to do the copy for you when using MCWS.

It also needs the Fields in the view that are going to be used for calculation, and those that are going to be changed, so it can grab and compare the old and new values (no sense sending back the same value to MCWS).

I imagine I could add a Filename or FileKey option, where you would pass the filename.  But I'd also need to update it to pull values from MCWS, and provide another option or two to specify which fields you want to use.  This might get a little complex, but I haven't spent much time thinking about this.

[MrC]

It played here, in the browser.  Maybe download and play it?

[glynor]

Pscriptor works by grabbing the Filename from the clipboard's CSV table.  It will either use an already copied selection of MC view rows, or will call MCWS to do the copy for you when using MCWS.

I was worried about this, from the demo video of yours I was able to watch, it seemed like this was the case.

I'm going to move this discussion to your pscriptor thread.

[glynor]

Using Dropbox to deliver video on the web isn't reliable.  Their storage system is NOT optimized for that kind of data transfer.

[glynor]

or will call MCWS to do the copy for you when using MCWS.

How does this part work?

My issue is that, while I have a bunch of these already, it is an ongoing concern.  I can go through and select the ones already funky, which would help.  But, to me, any "fix" like this is only valuable if it will be consistently applied.  The only way to ensure that it is consistently applied, is if it happens automatically.

Otherwise, it is better not to even have the view that uses the data.  Because it can't be "trusted" (or it relies on me being "on top of it" for it to be trust-able, and that's the same thing).

So, for pretty much any use-case I can think of, if I need to be at an interactive console, the utility is extremely limited.  That could come in handy if I had some kind of one-off tagging task that I need to Just Get Done (kind of like a Right-Click > Send To > Targeted Fixer), but... Honestly, there isn't much that the built-in expression language can't do already that I need regularly in ways like this.

For an "expression extension" type of utility to be really useful, I need to be able to apply the scripts over and over, automatically, as files come into the Library.  I think I've built a good tool for exactly that (and it can dump out whatever field values you want from MC, even results of expressions), but I don't have a general-purpose and extensible way to then dump "results" back into tags in MC.

I could, of course, build one, but I'd hoped pscriptor could serve that kind of purpose (especially since it is cross platform).

[MrC]

How does this part work?

    mcws_do_mcc_command(\%mcwsparams, 21000);

My issue is that, while I have a bunch of these already, it is an ongoing concern.  I can go through and select the ones already funky, which would help.  But, to me, any "fix" like this is only valuable if it will be consistently applied.  The only way to ensure that it is consistently applied, is if it happens automatically.

So, for pretty much any use-case I can think of, if I need to be at an interactive console, the utility is extremely limited.  That could come in handy if I had some kind of one-off tagging task that I need to Just Get Done (kind of like a Right-Click > Send To > Targeted Fixer), but... Honestly, there isn't much that the built-in expression language can't do already that I need regularly in ways like this.

For an "expression extension" type of utility to be really useful, I need to be able to apply the scripts over and over, automatically, as files come into the Library.  I think I've built a good tool for exactly that (and it can dump out whatever field values you want from MC, even results of expressions), but I don't have a general-purpose and extensible way to then dump "results" back into tags in MC.

I could, of course, build one, but I'd hoped pscriptor could serve that kind of purpose (especially since it is cross platform).

I suppose there are many different usage scenarios, yours primarily being automated processing of incoming files.  For that, you probably have a limited set of fixed tasks, and you just need an external driver/evaluator.  There are other uses where automatic flow doesn't really work, because users need to provide additional data or control.

If you have a Filename, and the fields you want evaluated, could you place this in CSV format on the clipboard?  Then it would just work (I'd add an option to not auto-copy).

[glynor]

    mcws_do_mcc_command(\%mcwsparams, 21000);

Ahh, I see.  So you still have to have them selected in the UI.

I suppose there are many different usage scenarios, yours primarily being automated processing of incoming files.  For that, you probably have a limited set of fixed tasks, and you just need an external driver/evaluator.  There are other uses where automatic flow doesn't really work, because users need to provide additional data or control.

Yep, exactly.  I see how it could be useful as-is for some purposes.  I just generally don't design my "needs" such that I can't accomplish things with the existing toolset.  If I want to change something structural about the Library, it has to be something I can do long-term, you know?

As I mentioned, writing a one-off to do something like this with my existing C# library would be trivial.  It can, obviously, already write to tags (that's how MCAutoQueue sets the [NeedsProcessing] flag to Completed or Failed or whatever).  But that would be a "Use MediaInfo to do X" kind of applet, not a general purpose tool where I could think up new uses later on without having to fire up the compiler.  Something more general purpose is a much bigger task, obviously (as you well know, I expect).

[glynor]

If you have a Filename, and the fields you want evaluated, could you place this in CSV format on the clipboard?  Then it would just work (I'd add an option to not auto-copy).

Dunno.  Possibly.

I might be able to write some kind of pscriptor connector utility...

[MrC]

Or an MPL, or XML or whatever.  The key things are for me to get the data you want operated on, and we want to avoid trying to pass complex params via the Windows (awful) command line.

[MrC]

Alternatively, maybe I can construct the data via MCWS.  I'd just need a list of filenames or filekeys, and a list of fields.

[MrC]

Here's an example of how AcousticFingerprint can be useful to detect duplicate files, even with different file encodings and compressions:

   http://yabb.jriver.com/interact/index.php?topic=87575.0

[MrC]

And another example, with a new script, that outputs which albums have missing discs and/or tracks (holes).

   http://yabb.jriver.com/interact/index.php?topic=84396.msg599818#msg599818

[lepa]

So this thing can fetch the audio and subtitle streams language names and write them to the tags? That would be nice! Been thinking to request this when MC analyses videos...