INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2] 3 4 5 6   Go Down

Author Topic: pscriptor.pl: a general purpose scriptable MC field manipulator + utility  (Read 97061 times)

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: pscriptor.pl: a general purpose scriptable MC field manipulator + utility
« Reply #50 on: February 19, 2014, 03:18:36 pm »

If you are referring to what mediainfo can report, then yes, it can grab that info.

Think of pscriptor this way.  It:

  1) is a per-file iterator program, that...
  2) performs scripted operations on those files and their metadata, and ...
  3) can update the file's metadata or do other things
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Reading Recorded Date from Home Videos Imported into MC
« Reply #51 on: February 25, 2014, 12:57:12 pm »

Dunno.  Possibly.

I might be able to write some kind of pscriptor connector utility...

@glynor - are you still interested in this?  This morning I've been thinking about mechanisms you could use to fire off pscriptor from your tool set.  I'm not keen on passing a filename via Windows cmd shell, because there are Unicode issues involved, if your filenames have any Unicode characters.  However, a FileKey() would be great.  I could add a code path that would accept a FileKey as input, ignore the old vs. new data comparison and just always return the new value via MCWS.  No clipboard or selection required.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I've posted version 1.07 which includes a a few new scriptlets, and includes the ability to define an output field for each input field.

Version 1.07 Changes:
    - New: fields supplied with -f can now include an output field.  Normally any -f field will be
      used as input, and after the script acts on that field value, the value returned from the
      script or scriptlet will be used to update that same field.  However, there are cases when
      it is useful to use one field as input, and another field as output.  To specify both input
      and output fields, use the new form -f "infield::outfield" on the command line.  Unlike the
      input field, the output field does not have to be in the view; however, having it in the view
      provides better performance because pscriptor avoids sending updates for unchanged values.
    - New: scriptlet DateParse attempts to find dates in fields and outputs a uniform date string.
    - Internal: amg.pl, discogs.pl, and pscriptor.pl now share common modules (MCUtils::MCWS, MCUtils::Misc).
    - Fix: typo in AcousticFingerprint MC field name for artist_id.
    - New: scriptlet MissingTracks, which outputs missing discs and/or tracks from albums.

Note: the DateParse scriptlet requires an additional module to be installed.  See the instructions at the top of the scriptlet file.
Note: DateParse outputs a string date, not an MC Date value, so define your field as a String.
Logged
The opinions I express represent my own folly.

stiv32

  • World Citizen
  • ***
  • Posts: 162
Using Fingerprinting to Deduplicate
« Reply #53 on: March 24, 2014, 04:18:56 pm »

I tried Mr C's pscriptor to add fingerprint to my files.

Firstly I created the Expression fields En ..., needed and added them at the view panel of MC.

I have tried of adding fingerprint to about 500 files. Firlstly I tried individual files playing with commands until it worked. The command I used is

perl pscriptor.pl -E AcousticFingerprint -v -c pscriptor-config.txt -f Name

Then I added the command to about 300 files. None of them returned any fingerprint string.

After that I found the album of the song that returned fingerprints before. I tried other songs of the same album and returned fingerprints as well.

My question is,

am I doing something wrong? Does pscriptor return a fingerprint at each song; even if the song is not at the online database? Should I use a different command?
Logged

stiv32

  • World Citizen
  • ***
  • Posts: 162
Re: Using Fingerprinting to Deduplicate
« Reply #54 on: March 24, 2014, 04:29:06 pm »

This might be helpful.

At the batch fingerprinting I get this:

...
...
...
        in(name): 'Garufa'
        old=new(name): <unknown>        field(name) UNCHANGED: null value return
ed from script
File: x:\t\music\classic\Tangata Rea\Tango All╬▒ Baila\Gar╧Λa Tangata Rea 1999.m
p3
Uncaught exception from user code:
        MCWS write failed:
        http://localhost:52199/MCWS/v1/File/SetInfo?Value=need%20codes%20in%20qu
ery%20for%20fingerprint%20matching&Field=&FileType=Filename&File=X%3A%5Ct%5Cmusi
c%5Cclassic%5C(Various%20Artists)%5CUnknown%20Artist%5CUnknown%20Album%5C11-TUDO
%20CABE%20NUM%20BEIJO(FT-cancao)%20Manoel%20Reis%20com%20Dianas%20do%20c%C3%A9uV
ic%2034335-2-1938.mp3
        500 Internal server error
        <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
        <Response Status="Failure" Information="Field not found."/>
        MCUtils::MCWS::send_items('MCUtils::MCWS=HASH(0x11b88f4)', 'REF(0x244a67
c)') called at MCUtils/MCWS.pm line 55
        MCUtils::MCWS::send('MCUtils::MCWS=HASH(0x11b88f4)', 'ARRAY(0x229bf8c)')
 called at pscriptor.pl line 188




When I try to get fingerprint from one file without success I get this:

C:\Users\PCCCCCC\pscriptor>perl pscriptor.pl -v -c pscriptor-config.txt -E Acous
ticFingerprint -f Name
File: X:\t\music\classic\Juan D'Arienzo - Su obra completa en la
RCA Vol 35-48\Juan D'Arienzo - Su obra completa en la RCA Vol 01-1935-1936\11 Or
questa Juan D'Arienzo - Orillas Del Plata-Instrumental(31-12-1935).mp3
        Track lookup - No results found: ...\Juan D'Arienzo - Su obra completa e
n la RCA Vol 35-48\Juan D'Arienzo - Su obra completa en la RCA Vol 01-1935-1936\
11 Orquesta Juan D'Arienzo - Orillas Del Plata-Instrumental(31-12-1935).mp3
        in(name): 'Orillas Del Plata-Instrumental'
        old=new(name): <unknown>        field(name) UNCHANGED: null value return
ed from script

and also

...\2. Epoca de Oro\Alfredo De ┴ngelis\Cantores\Fl
oreal Ruiz\Marioneta(J.J.Guichandut,A.Tagini) - De ┴ngelis, F.Ruiz (23.7.1943).m
p3
Use of uninitialized value $code in join or string at
        Scriptlets/AcousticFingerprint.pm line 90 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

Use of uninitialized value within %fieldmap in hash element at
        Scriptlets/AcousticFingerprint.pm line 129 (#1)





and when I successfully get fingerprint from a file I get the following:

C:\Users\PCCCCCC\pscriptor>perl pscriptor.pl -v -c pscriptor-config.txt -E Acous
ticFingerprint -f Name
File: X:\t\music\classic\Alfredo De Angelis\From Argentina to
 the world\Altar sin luz.mp3
        Track lookup - No results found: ...\Alfredo De Angelis\From Argentina t
o the world\Altar sin luz.mp3
        in(name): 'Altar sin luz'
        old=new(name): <unknown>        field(name) UNCHANGED: null value return
ed from script





Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Using Fingerprinting to Deduplicate
« Reply #55 on: March 24, 2014, 05:31:30 pm »

I've merged the topic into this pscriptor thread.  It really belongs here since it is only relevant to users of pscriptor.

Let's work on one file until you have it working correctly.

Firstly I created the Expression fields En ..., needed and added them at the view panel of MC.

I hope you are using the wrong terminology.  How exactly are you defining the fields?  They should not be Calculated Data fields - they are just user fields.

   - Show one of your field definitions.
   - Show how you have fields defined inside AcousticFingerprint.pm (if you have changed the mappings).

Use -d to get more debug output.


My question is,

am I doing something wrong? Does pscriptor return a fingerprint at each song; even if the song is not at the online database? Should I use a different command?

From your output in the thread above:

            <Response Status="Failure" Information="Field not found."/>

this would indicate a field is not defined, and looking at the query:

        MCWS write failed:
        http://localhost:52199/MCWS/v1/File/SetInfo?Value=need%20codes%20in%20qu
ery%20for%20fingerprint%20matching&Field=&FileType=Filename&File=X%3A%5Ct%5Cmusi
c%5Cclassic%5C(Various%20Artists)%5CUnknown%20Artist%5CUnknown%20Album%5C11-TUDO
%20CABE%20NUM%20BEIJO(FT-cancao)%20Manoel%20Reis%20com%20Dianas%20do%20c%C3%A9uV
ic%2034335-2-1938.mp3

there is no Field name here (it is empty - nothing but an & after the = char).  Now I also see the text "need%20codes%20in%20qu
ery%20for%20fingerprint%20matching". Replacing the %20 with space it reads "need codes in query for fingerprint matching".  So this appears that something is not working correctly.  Have you successfully used this before?  If I recall correctly, that error indicates codegen is not running successfully.

Try running codegen directly on a file, such as:

   codegen C:\path\to\audio\file

What results do you get?

By the way, increase the width of your console so that your copy/paste looks better.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Using Fingerprinting to Deduplicate
« Reply #56 on: March 24, 2014, 05:44:19 pm »

This might be helpful.

At the batch fingerprinting I get this:


There's no "batch" fingerprinting.  pscriptor works one file at a time, calling the scriptlet for each file.  So you're always just working on one file at a time, even though pscriptor will iterate over each selected file.




C:\Users\PCCCCCC\pscriptor>perl pscriptor.pl -v -c pscriptor-config.txt -E Acous
ticFingerprint -f Name
File: X:\t\music\classic\Juan D'Arienzo - Su obra completa en la
RCA Vol 35-48\Juan D'Arienzo - Su obra completa en la RCA Vol 01-1935-1936\11 Or
questa Juan D'Arienzo - Orillas Del Plata-Instrumental(31-12-1935).mp3
        Track lookup - No results found: ...\Juan D'Arienzo - Su obra completa e
n la RCA Vol 35-48\Juan D'Arienzo - Su obra completa en la RCA Vol 01-1935-1936\
11 Orquesta Juan D'Arienzo - Orillas Del Plata-Instrumental(31-12-1935).mp3
        in(name): 'Orillas Del Plata-Instrumental'
        old=new(name): <unknown>        field(name) UNCHANGED: null value return
ed from script


This track is not being detected at echo nest, so nobody had added it yet, or the offsets need to change to match this track.  Let's assume this track is not in the database, given its title.


...\2. Epoca de Oro\Alfredo De ┴ngelis\Cantores\Fl
oreal Ruiz\Marioneta(J.J.Guichandut,A.Tagini) - De ┴ngelis, F.Ruiz (23.7.1943).m
p3
Use of uninitialized value $code in join or string at
        Scriptlets/AcousticFingerprint.pm line 90 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

This seems to indicate that codegen is failing to run, since you don't have the $code value (it is what is returned by the codegen program).  Since I don't know exactly how you've set this up, its hard to further diagnose.  Use -d to help provide more info.


and when I successfully get fingerprint from a file I get the following:

C:\Users\PCCCCCC\pscriptor>perl pscriptor.pl -v -c pscriptor-config.txt -E Acous
ticFingerprint -f Name
File: X:\t\music\classic\Alfredo De Angelis\From Argentina to
 the world\Altar sin luz.mp3
        Track lookup - No results found: ...\Alfredo De Angelis\From Argentina t
o the world\Altar sin luz.mp3
        in(name): 'Altar sin luz'
        old=new(name): <unknown>        field(name) UNCHANGED: null value return
ed from script

Same thing as above - the track is not found.  It is best to use tracks that are likely to be found first, until you get the kinks worked out.

I'm sure there are things for me to clean-up, as I didn't place a lot of error checking for things like incorrect installations, missing fields, etc.  That will improve over time.

I'll PM you my email address so that you can send more info regarding this matter to me directly and we can avoid spamming this thread.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

A quick follow-up, since we've taken the conversation offline.

1. There was some misunderstanding about how the AcousticFingerprint scriptlet works.  The tracks being looked up were not found in Echo Nest's (very large) database, as they were rare, old Argentinian tango songs from 1898 to 1940.

2. There was some confusion about the codes generated by the codegen program, and the back-end analysis and track matching done by the Echo Nest server that the AcousticFingerprint scriptlet calls.  The codes by themselves are not useful, and require the server to translate and look-up track and acoustic information based on those codes.

3. I've create two new scriptlets: CopyFirst and CopyAndMerge.  Each can be used to copy field values within a specified grouping of tracks.  CopyFirst pushes down the topmost value in the group and CopyAndMerge copies all the values within the group for each field and merges those into a semicolon-separated list of values.  stiv32 is testing out the new scriptlets and changes.
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

I have a problem with the scriptlet GetCoverArtSize.

I have set up my new library field as string : "WW_CoverArtSize"

This is my commandline:
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor107>perl pscriptor.pl -v -t -c pscriptor-config.txt -f "WW_CoverArtSize" -E "Scriptlets\GetCoverArtSize"I get the following error:
Code: [Select]
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\01 - Bolero.mp3
Use of uninitialized value in subroutine entry at pscriptor.pl line 217 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    anid the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.

Undefined subroutine &main:: called at pscriptor.pl line 217 (#2)
    (F) The subroutine indicated hasn't been defined, or if it was, it has
    since been undefined.

Uncaught exception from user code:
        Undefined subroutine &main:: called at pscriptor.pl line 217.
        main::process_user_tracks('ARRAY(0x3368450)') called at pscriptor.pl line 186

What am I doing wrong?

Btw: Long time ago I used a similar script from Mr ChriZ:
http://yabb.jriver.com/interact/index.php?topic=51556.0

Is it possible (with pscriptor) to fill the "Image File (width)" and "Image File (height)" as separate integer values?




Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I have a problem with the scriptlet GetCoverArtSize.

I have set up my new library field as string : "WW_CoverArtSize"

This is my commandline:
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor107>perl pscriptor.pl -v -t -c pscriptor-config.txt -f "WW_CoverArtSize" -E "Scriptlets\GetCoverArtSize"
What am I doing wrong?

This:

   -E "Scriptlets\GetCoverArtSize"

should be the simpler:

   -E GetCoverArtSize

The actual name after -E must match the package name inside the scriptlet.  In your case, it won't (its a name, not a path).

Is it possible (with pscriptor) to fill the "Image File (width)" and "Image File (height)" as separate integer values?

Sure.  The scriptlet could be modified to return those values separately, or you could do it with another simple pscriptor call, or use MC expressions to split the values.  Which would you prefer?



Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

Wow, thank you, that was quick.
Now it works.

I would prefer one pscriptor call  which writes into seperate fields "WW_ImageFileWidth" and "WW_ImageFileHeight" (setup as Integer).

Thank you again for your help and your tool!
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

You're welcome.  I'll have it for you after lunch.  Gonna get me some BBQ!
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I've updated pscriptor to version 1.08.  This version has many changes and goodies:

Version 1.08
    - New: option --args|-a allows passing scriptlets an argument string.
    - New: added pre-processing callback support to allow scriptlets a change to view all the data
      in the view before the per-file work is done (per-file is the only place where metadata updates
      are done).
    - New: scriptlet CopyFirst will copy the first value in a grouping, and push that value to
      other members of the grouping.  The -f option defines the fields for copying, and the new
      option -a specifies the column label that defines the grouping (e.g. Album).
    - New: scriptlet CopyAndMerge will copy all of the values in each -a defined grouping, for the
      field(s) specified with -f.  The per-grouping values will then be merged into a semicolon-
      delimited list, and saved to the output field defined by -f.
    - New: scriptlet GetCoverArtSize can now return WxH values as two separate fields.  See the comments
      in the scriptlet file.
    - Change: all scriptlets have been updated to accomodate the changed per-file callback argument
      list.
    - Fix: several improvements to scriptlet DateParse: reject date candidates beyond "today";
      disallow date components to be adjacent to another non-date digits; allow month-year dates,
      but coerce day to be 1.
    - Change: scriptlet MakePlaylist now accepts a grouping field via --args, but defaults to 'album'.
    - Change: scriptlet RandomPlaybackRange now accepts a playback duration via --args, or defaults to
      either a column named 'playback duration' or the default value of 30 (seconds).
    - Change: sample template scriptlets Sample and SamplePostProcessing were uppdated and renamed to
      SampleSimple and SampleComplete to reflect the new pre-processing callback and new arguments,
      and the comments were updated.
    - Internal: added code to the sample modules to breakpoint the perl debugger when it is enabled,
      making it easier to debug scriptlet modules.


@winniew - the updated GetCoverArtSize scriptlet is included.  You'll add an additional option and argument:

   ... -a WW_ImageFileWidth,WW_ImageFileHeight

See the comments above regarding both -a and GetCoverArtSize.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

[ Edit: I've removed the link from this page, as I didn't want it posted here.  MrC ]
Logged

winniew

  • World Citizen
  • ***
  • Posts: 102

I'm just testing the new version 1.08 with the new scriptlet GetCoverArtSize.
It works for both:
1. Getting the Coverartsize as "width x height"
2. Getting separate fields for width and height

The only (no big) problem is, that it updates the fields in MC even when you use the switch "-t".

Even with switch -t it updates the fields in MC.
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor108>perl pscriptor.pl -v -t -c pscriptor-config.txt -f "WW_CoverArtSize" -E GetCoverArtSize
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\01 - Bolero.mp3
        in(ww_coverartsize): ''
        old(ww_coverartsize): <unknown>
        new(ww_coverartsize): '344x511'

Here the same for the separate fields (no comment in.. old.. new..)
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor108>perl pscriptor.pl -v -t -c pscriptor-config.txt -f "WW_CoverArtSize" -E GetCoverArtSize -a WW_Image_File_Width,WW_Image_File_Height
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\01 - Bolero.mp3

C:\Users\Winnie\Desktop\Pscriptor108>

So for me this is no real showstopper, but you should know the behaviour.
Perhaps I'm using the wrong switches in wrong order...
Thank you for your quick implementation.

WinnieW
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

@theoctavist - I removed the posted codegen link.  Using the same link posted at the top of the scriptlet, replace the .exe with .zip.  Sorry, I forgot to update the link there.  Nice drawing.

@winniew - good to hear of the success.  I'll correct the missing -t functionality.  Thanks for mentioning the issue.

The order of options generally doesn't matter (with the exception of where you place the -c <config file> option, and for my debugging purposes, where -d or -v is located).
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Re: Using Fingerprinting to Deduplicate
« Reply #66 on: March 27, 2014, 04:30:22 pm »

I've merged the topic into this pscriptor thread.  It really belongs here since it is only relevant to users of pscriptor.

Let's work on one file until you have it working correctly.

I hope you are using the wrong terminology.  How exactly are you defining the fields?  They should not be Calculated Data fields - they are just user fields.

   - Show one of your field definitions.
   - Show how you have fields defined inside AcousticFingerprint.pm (if you have changed the mappings).

Use -d to get more debug output.


From your output in the thread above:

            <Response Status="Failure" Information="Field not found."/>

this would indicate a field is not defined, and looking at the query:

        MCWS write failed:
        http://localhost:52199/MCWS/v1/File/SetInfo?Value=need%20codes%20in%20qu
ery%20for%20fingerprint%20matching&Field=&FileType=Filename&File=X%3A%5Ct%5Cmusi
c%5Cclassic%5C(Various%20Artists)%5CUnknown%20Artist%5CUnknown%20Album%5C11-TUDO
%20CABE%20NUM%20BEIJO(FT-cancao)%20Manoel%20Reis%20com%20Dianas%20do%20c%C3%A9uV
ic%2034335-2-1938.mp3

there is no Field name here (it is empty - nothing but an & after the = char).  Now I also see the text "need%20codes%20in%20qu
ery%20for%20fingerprint%20matching". Replacing the %20 with space it reads "need codes in query for fingerprint matching".  So this appears that something is not working correctly.  Have you successfully used this before?  If I recall correctly, that error indicates codegen is not running successfully.

Try running codegen directly on a file, such as:

   codegen C:\path\to\audio\file

What results do you get?

By the way, increase the width of your console so that your copy/paste looks better.

please forgive me my ignorance, but I am attempting to get the Acoustic Fingerprint of my entire media library for the removal of duplicates and lossy compression algorithm files.  what fields *must* be in jrmc? i am foggy on this point. i am entirely new with command line, perl, etc. so baby steps(although i did get the discogs scraper to work wonderfully! (thanks btw)

Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Using Fingerprinting to Deduplicate
« Reply #67 on: March 27, 2014, 04:38:52 pm »

please forgive me my ignorance, but I am attempting to get the Acoustic Fingerprint of my entire media library for the removal of duplicates and lossy compression algorithm files.  what fields *must* be in MC? i am foggy on this point. i am entirely new with command line, perl, etc. so baby steps(although i did get the discogs scraper to work wonderfully! (thanks btw)

No worries.  Although the identification returns 20 items, you only need to define/use the fields you want.   You can customize the field names in the scriptlet (soon I'll add per-scriptlet config file options so you don't have to muck w/source code).

The video in a previous post shows the field of interest for de-duping - the MD5 value.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I've posted version 1.09.  It adds the ability to configure scriptlet options in the config file, so you don't have to edit the scriptlet source anymore.

Version 1.09
    - Fix: option --testonly | -t was not working correctly.
    - New: scriptlet options may be specified in the confile file.  See Scriptlet Options in the
      the config file for more informtion.
    - New: scriptlet GetIDV3Tag will use the tag name supplied via -a, or else from the field
      contents of the -f field.
    - Fix: scriptlet RandomPlaybackRange would incorrectly output a warning when -a was no used.
    - Fix: scriptlet RandomPlaybackRange was not setting the duration properly from -a.

@theoctavist: you no longer have to edit the AcousticFingerprint code to customize your fields - see the new config file, specifically the Scriptlet Options.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

thanks Sir!

something dumb I know..but entering perl pscriptor.pl -E AcousticFingerprint -v -c pscriptor-config.txt -f Name


gives me "codegen not recognized as valuable blah blah or batch command

also 500 internal server error

nd "need codes in query for fingerprint matching.

ill get there, promise. just need gentle nudges ;)
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

Enable QuickEdit Mode in your command shell.  Right-click the title bar, and select Properties and then the Options tab.  Ok your way out, and then increase the size of the window as much as you want.

Then, run the pscriptor command again. Copy the output - console is wierd, you select a block of text rather than lines, so be sure to select all the text.   Once selected, press Enter (this copies).  Now you can paste the error text.

It seems to me that you don't have the codegen program in your path.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

C:\Users\me\Desktop\pscriptor>perl pscriptor.pl -E AcousticFingerprint -v -c pscriptor-config.txt -f Name
File: H:\MY MUSIC LIBRARY\Giant Sand...proVISIONS(2008)[FLAC]\01. Stranded Pearl.flac
'codegen' is not recognized as an internal or external command, operable program or batch file.

...
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I trimmed the extraneous output, and marked in red the problem.

Where did you place the codegen program?  It must be located in one of the directories in your Windows PATH variable, or you have to specify the full path in the config file.

You can see your path by typing:

   echo %PATH%

in the command shell.

Also, it looks like you don't have the latest version of pscriptor?
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

hey, I *thought* I did.  i took that codegen zip and created a directory for it, re pscriptor, i took 1.09 and unzipped it to the folder i am using on the desktop, simply replacing the files with updated versions

full path is here.


C:\Users\me\Desktop\pscriptor> echo %PATH%
C:\Perl64\site\bin;C:\Perl64\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\Syste
m32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\WinS
CP\;C:\ffmpeg-20140327-git-62094e2-win32-static\bin\ffmpeg.exe;C:\ENMFP_codegen\
codegen.windows.exe;C:\codegenffmpeg\codegen.exe;C:\codegenffmpeg\ffmpeg.exe;C:\
codegen\codegen.exe
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.


C:\Users\me\Desktop\pscriptor> echo %PATH%
C:\Perl64\site\bin;C:\Perl64\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\Syste
m32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\WinS
CP\;C:\ffmpeg-20140327-git-62094e2-win32-static\bin\ffmpeg.exe;C:\ENMFP_codegen\
codegen.windows.exe;C:\codegenffmpeg\codegen.exe;C:\codegenffmpeg\ffmpeg.exe;C:\
codegen\codegen.exe

You've added the name of the executable to the path.  Remove the red stuff.  PATH contains a list of directories.  It seems you've done likewise with several other executables too.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

haha!  :D

thank you! i am an idiot!  ;D
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

thank you! i am an idiot!  ;D

You're welcome.  Don't pick a fight with yourself.  The other guy is sure to lose.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

it works now, thank you! the only thing is, im not seeing the .md5 hash show up in jrmc field..maybe i set it up wrong? but i didnt see audio_md5 in the output ..just valence, loudness..well several others, no md5 though
 
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

Run with -d -v and you'll see output like:

    read_config_file    : scriptlet AcousticFingerprint option field: audio_md5 ==> 'EN MD5'
    ...
    do_perfile          : Key/Value: audio_md5 = 89ddebea3ac60fb73c55d905cb81eb63
    ...
    new(EN MD5): 89ddebea3ac60fb73c55d905cb81eb63

The first one indicates the mapping from the config file.  This shows your field name.

The second one shows you the returned value.

The third one shows you the value assigned to your field.
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

Here I am again with Scriptlet GetCoverArtSize.
I have a strange issue with filenames containing french accents (for instance accent aigu).

Here copy-pasted from the command shell:
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor109>perl pscriptor.pl -v -t -c pscriptor-config.txt -f "WW_CoverArtSize" -E GetCoverArtSize -a WW_Image_File_Width,WW_Image_File_Height
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - PrÚlude Ó l¦aprÞs-midi d¦un faune.mp3
        in(ww_coverartsize): ''
        old=new(ww_coverartsize): <unknown>     field(ww_coverartsize) UNCHANGED: null value returned from script

Copy-pasting the field "Filename" directly from MC19.124 I get this (looks OK):
Code: [Select]
E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - Prélude à l´après-midi d´un faune.mp3
I think it's something with encoding the clipboard, but I have absolutely no idea.
I am working on Windows7 64Bit German.


Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I have a strange issue with filenames containing french accents (for instance accent aigu).
   ...
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - PrÚlude Ó l¦aprÞs-midi d¦un faune.mp3
   ...
Copy-pasting the field "Filename" directly from MC19.124 I get this (looks OK):

E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - Prélude à l´après-midi d´un faune.mp3

I figured out what was causing this, and have solved the problem, but the solution leads to yet another issue for which I'm trying to find an optimal solution.

Technical FYI: The clipboard data is fine.  The problem is elsewhere, and is tricky.  It reared its head due to this scriptlet spawning a sub-process and needing to deliver a file path to a Windows binary.  The Perl-to-Windows bridge in ActivePerl has to convert internal Perl character representations into a byte sequence suitable for Windows, but there are cases where the automatic conversion cannot map correctly (due to implementation choices made for backwards compatibility reasons).  So I have to force the bridge into UTF-8 mode and force a wide-string (Variant BSTR) across the bridge.
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

I figured out what was causing this, and have solved the problem, but the solution leads to yet another issue for which I'm trying to find an optimal solution.
..
Thanks, there is no hurry. I will be patient.
Logged

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

[Edit: I fixed your link]

Mr. C is there a way to make things better with regard to my issue here, that is ..could a scriptlet be made to improve the random sampling of tracks>

http://yabb.jriver.com/interact/index.php?topic=88428.0
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

Mr. C is there a way to make things better with regard to my issue here, that is ..could a scriptlet be made to improve the random sampling of tracks>

Sure, I suppose it could create a random playlist of size N and play that.  You want a random ordering of the tracks, and then selecting N of them?

I'm concerned about the amount of files you'll be throwing at it.  45,000 files is a lot of stuff to place on the clipboard, and I've not tested pscriptor with anything that large.  I suppose you could make a view with just the Filename field in it, and that will certain reduce the load.

I'll take a look at this sometime in the next day or two; I'll probably base it off the existing MakePlaylist scriptlet, and just drop the playlist to your desktop (and it could cause MC to start playback of that playlist too I suppose).
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

Thanks, there is no hurry. I will be patient.

Ok, good news.  My 18000+ file test passed, so I'll post an update tomorrow.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

Sure, I suppose it could create a random playlist of size N and play that.  You want a random ordering of the tracks, and then selecting N of them?

I'm concerned about the amount of files you'll be throwing at it.  45,000 files is a lot of stuff to place on the clipboard, and I've not tested pscriptor with anything that large.  I suppose you could make a view with just the Filename field in it, and that will certain reduce the load.

I'll take a look at this sometime in the next day or two; I'll probably base it off the existing MakePlaylist scriptlet, and just drop the playlist to your desktop (and it could cause MC to start playback of that playlist too I suppose).


yes sir, that would be perfect.
Logged

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

Run with -d -v and you'll see output like:

    read_config_file    : scriptlet AcousticFingerprint option field: audio_md5 ==> 'EN MD5'
    ...
    do_perfile          : Key/Value: audio_md5 = 89ddebea3ac60fb73c55d905cb81eb63
    ...
    new(EN MD5): 89ddebea3ac60fb73c55d905cb81eb63

The first one indicates the mapping from the config file.  This shows your field name.

The second one shows you the returned value.

The third one shows you the value assigned to your field.



well, i see an md5 value in the command shell output,.  but when it finishes there is no value displayed in the MC field. i should see that checksum string just like id see the name, album, etc right? im sorry to be so slow MrC
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.


well, i see an md5 value in the command shell output,.  but when it finishes there is no value displayed in the MC field. i should see that checksum string just like id see the name, album, etc right? im sorry to be so slow MrC

If the field is not being written to your Library, then it means the field name is not the same as the field specified in the config file (very that in from the output).
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

Ok, good news.  My 18000+ file test passed, so I'll post an update tomorrow.
Cool.
..
So I am looking for my own testcases to validate your coming update.

One problem feeds the next: I am trying to find all my files with "Filename contains character à".

Search-Wizard-Rule

show 90% of my library, this is definitively wrong.
Not really intuitive. What am I doing wrong (again)?

Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I've posted version 1.10.  It includes the Numberize and MakeRandomPlaylist scriptlets, and should also resolve the problem with Unicode characters in the AcousticFingerprint scriptlet.

Version 1.10
    - Fix: a script return value of 0 was not updating a cell.
    - New: scriptlet Numberize will return a numerical sequence on a per-group basis.  See the
      comments in the scriptlet for usage.
    - New: scriptlet MakeRandomPlaylist will select N random tracks from the selected files for
      more randomness of playback than MC provides.  Will also auto-start playback by default.
      See the comments in the scriptlet for more info.
    - Fix: scriptlet AcousticFingerprint failed on some Unicode file names.
    - Internal: Use common MCUilts clipboard read/parsing.
    - Internal: the 'perfile' scriptlet callback may be set to undef if no per-file work is required.

@theoctavist - please give the MakeRandomPlaylist scriptlet a trial.  If you are going to select all of your 45k tracks, it would be more efficient of you create a new view or smartlist that only includes the Filename column (so that less is posted to the clipboard).  Example usage, picking 100 random tracks from the selecting and playing them back in zone 2.

    perl pscriptor.pl -c pscriptor-config.txt -E MakeRandomPlaylist -a "n=100 zone=2"

@winniew - please test to see if this update resolves the AcousticFingerprint filename issues you had.

One problem feeds the next: I am trying to find all my files with "Filename contains character à".
...
Not really intuitive. What am I doing wrong (again)?

The problem is that MC is automatically ASCII-fying your characters, since most people want that behavior with Search.

Try typing this into the Search box:

   [=regex([filename],/#[é]#/)]=1

You can add characters inside the [ ] brackets - any of them will be matched.
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

I have just tried the new version 1.10 with GetCoverArtSize and my Ravel-Testfiles.
With empty sizefields I get the following:
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor110>perl pscriptor.pl -v -c pscriptor-config.txt -f "WW_CoverArtSize" -E GetCoverArtSize -a WW_Image_File_Width,WW_Image_File_Height
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\03 - Clair de lune.mp3
        in(ww_coverartsize): ''
        new(ww_image_file_height): 511
        new(ww_image_file_width): 344
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - Pr+®lude +á l-¦apr+¿s-midi d-¦un faune.mp3
        in(ww_coverartsize): ''
        old=new(ww_coverartsize): <unknown>     field(ww_coverartsize) UNCHANGED: null value returned from script
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\05 - La Mer I. De l'aube +á midi sur la mer.mp3
        in(ww_coverartsize): ''
        old=new(ww_coverartsize): <unknown>     field(ww_coverartsize) UNCHANGED: null value returned from script

First file is OK but again the files with the french accents don't work for me. Sorry.

I will send you a PM with a link to my problemfile.

The regex in the Search box works perfect, that would have taken me hours.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I have just tried the new version 1.10 with GetCoverArtSize and my Ravel-Testfiles.
With empty sizefields I get the following:
Code: [Select]
C:\Users\Winnie\Desktop\Pscriptor110>perl pscriptor.pl -v -c pscriptor-config.txt -f "WW_CoverArtSize" -E GetCoverArtSize -a WW_Image_File_Width,WW_Image_File_Height
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\03 - Clair de lune.mp3
        in(ww_coverartsize): ''
        new(ww_image_file_height): 511
        new(ww_image_file_width): 344
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\04 - Pr+®lude +á l-¦apr+¿s-midi d-¦un faune.mp3
        in(ww_coverartsize): ''
        old=new(ww_coverartsize): <unknown>     field(ww_coverartsize) UNCHANGED: null value returned from script
File: E:\mp3_rw_2009\(Multiple Artists)\Ravel - Bolero\05 - La Mer I. De l'aube +á midi sur la mer.mp3
        in(ww_coverartsize): ''
        old=new(ww_coverartsize): <unknown>     field(ww_coverartsize) UNCHANGED: null value returned from script

First file is OK but again the files with the french accents don't work for me. Sorry.

Oh, I'm a doof.  Of course it won't work any better.  I updated the AcousticFingerprint scriptlet, not the GetCoverArtSize scriptlet!  Somehow i started using that script for testing the problem and had forgotten entirely that you were using the other one.   I'll make the changes today.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

@ winniew - please test the attached version of the scriptlet.
Logged
The opinions I express represent my own folly.

winniew

  • World Citizen
  • ***
  • Posts: 102

Perfect, now GetCoverArtSize works, even with my worstcase-filename:
"04 - Prélude à l´après-midi d´un faune_áàéè_äÄöÜß_°^§$%&#€.mp3"

Thank you for quick help!
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

You're welcome, and I'm really sorry about the time this took.  This is one of those weak areas with ActiveState's ActivePerl; they chose to be incompatible for ease of use for most users years ago, and that decision in certain areas has created issues (other perl versions do not suffer these limitations).  I also didn't realize that the Image module I use would botch the filenames too under ActivePerl.

Thanks for your patience.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I've posted pscriptor version 1.11.  The new WhichPlaylists scriptlet allows retrieving the list of all playlists that a file resides in.  See the comments in the scriptlet.

Version 1.11
    - Fix: scriptlet GetCoverArtSize failed on some Unicode file names.
    - Change: Sample scriptlets renamed with leading underscore (helps filename autocompletion
      and sorting).
    - New: scriptlet WhichPlaylists returns a list of every playlist that a file exists in.  The
      fullpath of the playlist is returned, suitable for use in building paths, etc.
    - Change: obsoleted scriptlet CopyPlaylistName, which could only copy the name (not path) of
      a playlist, and therefore had limited utility.  See WhichPlaylists above for superior
      functionality.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

I've posted version 1.10.  It includes the Numberize and MakeRandomPlaylist scriptlets, and should also resolve the problem with Unicode characters in the AcousticFingerprint scriptlet.

Version 1.10
    - Fix: a script return value of 0 was not updating a cell.
    - New: scriptlet Numberize will return a numerical sequence on a per-group basis.  See the
      comments in the scriptlet for usage.
    - New: scriptlet MakeRandomPlaylist will select N random tracks from the selected files for
      more randomness of playback than MC provides.  Will also auto-start playback by default.
      See the comments in the scriptlet for more info.
    - Fix: scriptlet AcousticFingerprint failed on some Unicode file names.
    - Internal: Use common MCUilts clipboard read/parsing.
    - Internal: the 'perfile' scriptlet callback may be set to undef if no per-file work is required.

@theoctavist - please give the MakeRandomPlaylist scriptlet a trial.  If you are going to select all of your 45k tracks, it would be more efficient of you create a new view or smartlist that only includes the Filename column (so that less is posted to the clipboard).  Example usage, picking 100 random tracks from the selecting and playing them back in zone 2.

    perl pscriptor.pl -c pscriptor-config.txt -E MakeRandomPlaylist -a "n=100 zone=2"

@winniew - please test to see if this update resolves the AcousticFingerprint filename issues you had.

The problem is that MC is automatically ASCII-fying your characters, since most people want that behavior with Search.

Try typing this into the Search box:

   [=regex([filename],/#[é]#/)]=1

You can add characters inside the [ ] brackets - any of them will be matched.


im getting there.

ok, lets say i dont want to change the default...i just want to always select all the audio files and start the playback immediately.

I have on one of my drives... 23,638 files. just want to play them , not worry about zones, or specific path to put the playlist in. 

how would I type it out?  i mean what would come after this...
perl pscriptor.pl -c pscriptor-config.txt -E MakeRandomPlaylist  -a n=23638

is that right or? again, just want to autoplay all random all the time, to include all the audio tracks I have.

im terribly slow with all this code stuff.

Logged

stiv32

  • World Citizen
  • ***
  • Posts: 162

I've posted pscriptor version 1.11.  The new WhichPlaylists scriptlet allows retrieving the list of all playlists that a file resides in.  See the comments in the scriptlet.


    - New: scriptlet WhichPlaylists returns a list of every playlist that a file exists in.  The
      fullpath of the playlist is returned, suitable for use in building paths, etc.
    - Change: obsoleted scriptlet CopyPlaylistName, which could only copy the name (not path) of
      a playlist, and therefore had limited utility.  See WhichPlaylists above for superior
      functionality.

Could you give me a sample command to test it? Can we now merge the playlists at the 'Playlists field' of each track?
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I have on one of my drives... 23,638 files. just want to play them , not worry about zones, or specific path to put the playlist in. 

how would I type it out?  i mean what would come after this...

You have it right, but it is even simpler.  Select all your files and enter the command:

   perl pscriptor.pl -c pscriptor-config.txt -E MakeRandomPlaylist

Could you give me a sample command to test it? Can we now merge the playlists at the 'Playlists field' of each track?

   perl -c pscriptor-config.txt -E WhichPlaylists -f some_field

where some_field is the name of the field you want to be the target of the list of playlist paths. WhichPlaylists will place in that field all the full paths of each playlist that includes the file (Smartllsts are omitted).   MC's Playlist file list column is not used.
Logged
The opinions I express represent my own folly.

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.

You have it right, but it is even simpler.  Select all your files and enter the command:

   perl pscriptor.pl -c pscriptor-config.txt -E MakeRandomPlaylist

   perl -c pscriptor-config.txt -E WhichPlaylists -f some_field

where some_field is the name of the field you want to be the target of the list of playlist paths. WhichPlaylists will place in that field all the full paths of each playlist that includes the file (Smartllsts are omitted).   MC's Playlist file list column is not used.

it works fantastically well!! AWESOME! thank you so much! used with 23,832 files today.  (One of my music drives)

so happy! the random before has nothing on this.
Logged
Pages: 1 [2] 3 4 5 6   Go Up