INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 2 3 [4] 5 6 7 8 9   Go Down

Author Topic: Expression functions [HELP WANTED]  (Read 62724 times)

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #150 on: May 18, 2020, 02:46:51 pm »

Thanks.  Lepa and I were talking about this, as he was having issues with slowness of these functions.

I thought it odd, as these functions perform a search essentially the same as filtering in a panes view, and filtering in a panes view is extremely rapid.

Are these Group...Query functions using a totally different search methodology?
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #151 on: May 20, 2020, 05:45:13 pm »

Matt, there's a lot that could be done to make Regex() more usable, but for starters, could it be changed to either return all non-overlapping matches to a single capture group (like Boost normally would) populating R1..R9 as appropriate, or at least, using the existing multi-explicit-capture-group syntax, not throw away the first 5 matches if match 6 isn't there, because the string was too short? Regex matching against lists of variable length is problematic.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #152 on: May 21, 2020, 11:33:55 am »

I'll make the next build have this:
NEW: The Regex expression function takes a -2 mode that outputs all the captures as a semi-colon delimited list.

Thanks for the idea.
Logged
Matt Ashland, JRiver Media Center

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #153 on: May 31, 2020, 01:19:41 am »

I'll make the next build have this:
NEW: The Regex expression function takes a -2 mode that outputs all the captures as a semi-colon delimited list.

Thanks for the idea.
I thought at first that this would allow for producing a list, from a list, like so...

(I really can't wrap my head around regex, so forgive the laborious explanation coming...)

If the source list contains, amongst other things:

A/1;A/2;A/3;A/4

MrC kindly provided regex to search for "A anything ;" and then remove the "A/". This much I can just about keep up with.
The result, is 1, because the regex stops as soon as it finds a match.
I want, "1;2;3;4"

When I first read this change, I thought this would do it, but, I can't get there. Is this because I misunderstood and it's still not possible, or because I'm not constructing correctly?

I initially thought that by changing the mode here to -2, it would place a string of all matches into [R1], but, I now think that what this actually does is return a list from [R1];[R2];[R3] etc.?
As my expression only ever returns an [R1], my goal is still unattainable, right?
Code: [Select]
if(regex([keywords],/#(!People[^;]+)#/,0),replace([R1],!People\,),)&datatype=[list]
I sat down ages ago to add this to the wiki, and that still hasn't happened yet! Reminds of when we used to send our kids to tidy their rooms.... Never happened :D

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #154 on: May 31, 2020, 03:43:24 am »

Marko,
I'm not sure I understand what you're trying to do, but I can see several ways of doing what I think you want.
the following expressions assume you want to take out the prefix "!People\" from list elements:

1. Why not just do replace([Keywords],!People\,) ?

2. You can use ListMix() to apply a function to each element of the list:
ListMix(/#Replace([L1],!People\,)/#, 0, [Keywords])
ListMix(/#Regex([L1],!People\(.+), 1)/#, 0, [Keywords])

3. Regex with mode -2:
Regex([keywords],/#!People\([^;]+)#/,-2)
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #155 on: May 31, 2020, 05:57:20 am »

Thanks for replying... Here's exactly what I want...

Take this string, a [Keywords] example...
Code: [Select]
!Events\Annual Christmas Get-Together;!People\Family\MarkW;!People\Family\Nettie;!People\Friends\Alex;!People\Friends\David;!People\Friends\John Stevens;!People\Friends\Louise Stevens;!People\Friends\Megan;!People\Friends\Tracy C;!Places\!Edinburgh\Platform 5
Extract all instances of !People\...\...;
Remove !People\ from each instance
return new list: Family\MarkW;Family\Nettie;Friends\Alex;Friends\David;Friends\John Stevens;Friends\Louise Stevens;Friends\Megan;Friends\Tracy C

Is that possible?

lepa

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2033
Re: Expression functions [HELP WANTED]
« Reply #156 on: May 31, 2020, 06:04:16 am »

Thanks for replying... Here's exactly what I want...

Take this string, a [Keywords] example...
Code: [Select]
!Events\Annual Christmas Get-Together;!People\Family\MarkW;!People\Family\Nettie;!People\Friends\Alex;!People\Friends\David;!People\Friends\John Stevens;!People\Friends\Louise Stevens;!People\Friends\Megan;!People\Friends\Tracy C;!Places\!Edinburgh\Platform 5
Extract all instances of !People\...\...;
Remove !People\ from each instance
return new list: Family\MarkW;Family\Nettie;Friends\Alex;Friends\David;Friends\John Stevens;Friends\Louise Stevens;Friends\Megan;Friends\Tracy C

Is that possible?
I thought this was exactly the point of -2 although havent test it yet myself.
wer gave here some regex example how to catch all the instances between similar patterns. Of course at that time there wasn't -2 option yet so it didn't work then.
https://yabb.jriver.com/interact/index.php/topic,119385.msg867397.html#msg867397
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #157 on: May 31, 2020, 07:05:46 am »

Quote
I thought this was exactly the point of -2 although havent test it yet myself.
Me too, untill I got up close and personal with it...

I want to capture Everything between !People and ;, and then remove all instances of !People\
My issue being, the regex function stops after it has captured the first instance. I want it to carry on, finding each instance, placing each into [R1] to [Rn], then, I believe, the -2 mode could output my list. As it is with what I have at the moment, I only ever get as far as [R1], so have to fall back on 20 nested if(isequal( functions...
Code: [Select]
listbuild(1,;,if(isequal(listitem([keywords],0,;),people,8),replace(listitem([keywords],0,;),!People\,),),if(isequal(listitem([keywords],1,;),people,8),replace(listitem([keywords],1,;),!People\,),), and so on...20 being an arbitrary number that ensures all people are always captured. I think this is having a pretty bad performance affect in theater view, where on-screen responses to navigation from the remote in my image library take a good three seconds a press.

I was hoping regex could provide a more streamlined and efficient solution, but think that the answer is still, no, hoping someone can say, actually, yes, try this... :)

-marko

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #158 on: May 31, 2020, 08:59:25 am »

Regex mode -2 should work for that, but I just tested it and it looks like -2 is not yet available/working.
I made that work with this:

Code: [Select]
Replace(ListMix(/#if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),)#/, 0, [keywords]),; ;,;)
- Uses ListMix to evaluate each item in the list [keywords]
- for each item, if it contains "!People\", replace the "!People\" with nothing and return the rest; else return blank;
- cleanup the resultng list with Replace() to remove empty list items (I thought there was some other function to do this...)

Logged

lepa

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2033
Re: Expression functions [HELP WANTED]
« Reply #159 on: May 31, 2020, 10:39:51 am »

Code: [Select]
Replace(ListMix(/#if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),)#/, 0, [keywords]),; ;,;)
Hah, that's clever  :) I also thought there was a function to exclude empty items but I couldn't find any. ListClean() could be a good canditate for that...
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #160 on: May 31, 2020, 12:18:39 pm »

One new function proposal that would help here:

ListGrep(list, text, mode)

Returns all elements of list which contain text.
Mode 0 - case insensitive
Mode 1 - case sensitive

ListGrep(redneck;blue;red roses, red) = "redneck; red roses"

Then the above code could be reduced to
Code: [Select]
Replace(ListGrep([Keywords], !People\), !People\,)
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #161 on: May 31, 2020, 12:55:26 pm »

Looks interesting, thanks zybex. Might be a day or three before I can play and feedback, back to work in the morning.

ListGrep would be great too 👍

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Expression functions [HELP WANTED]
« Reply #162 on: May 31, 2020, 06:22:54 pm »

Regex mode -2 should work for that, but I just tested it and it looks like -2 is not yet available/working.

The new mode isn't available until MC26.0.84, which isn't public yet. However, Marko has been working with a version that includes the mode.

It would be great if someone could test and provide examples for the use of Mode -2 when a Public Release comes out that includes it. Then we can update the Wiki.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72413
  • Where did I put my teeth?
Re: Expression functions [HELP WANTED]
« Reply #163 on: May 31, 2020, 06:37:23 pm »

zybex, you have mail.
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #164 on: June 01, 2020, 12:51:05 pm »

Had a very quick play with this, and, it works... with a few 'buts'...

I had to wrap your expression in two expressions... ListClean, removing duplicates still left some trailing semi colons, so, I tested for that, removing if necessary, resulting in quite a cumbersom expression, that nonetheless, worked...
Code: [Select]
if(isequal(Right(ListClean(Replace(ListMix(/#if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),)#/, 0, [keywords]),; ;,;),1),1),/;),removeright(ListClean(Replace(ListMix(/#if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),)#/, 0, [keywords]),; ;,;),1),1),ListClean(Replace(ListMix(/#if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),)#/, 0, [keywords]),; ;,;),1))
I think there's a bug in the the ListMix function because, if there are brackets in the list items, a syntax error is produced, only for that item, the rest are processed correctly. For example...
After running through the expression above, this keywords string:
Code: [Select]
!Events\!Concerts\Take That (Caroline & Vada); !People\Family\Caroline; !People\Family\Vada; !Places\!Glasgow\The Hydro
Returns:
Code: [Select]
Expression Error (Syntax Error); Family\Caroline; Family\Vada
I won't be able to test this against Theater View performance on the HTPC until at least Sunday. Thanks for your help with this zybex.

Do you really think the regex "-2" mode will catch these? How would you get it to to place all the "People" list items into separate captures? My understanding was, that once it dealt with the first instance, it would consider its work done, and stop.

-marko.


zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #165 on: June 01, 2020, 01:45:46 pm »

Try this for some magic:
Code: [Select]
Replace(Replace(ListMix(if(Compare(find(/#[L1]#/,!People\),>=,0),Replace(/#[L1]#/,!People\,),@@), 0, [keywords]),; @@,),@@; ,)
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #166 on: June 01, 2020, 02:13:17 pm »

I think there's a bug in the the ListMix function because, if there are brackets in the list items, a syntax error is produced, only for that item, the rest are processed correctly.

It looks like parenthesis were causing problems.  Next build should fix it.  Thanks for the report.
Logged
Matt Ashland, JRiver Media Center

Doof

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5908
  • Farm Animal Stupid
Re: Expression functions [HELP WANTED]
« Reply #167 on: June 01, 2020, 02:48:24 pm »

Still taking requests? If so, if there isn't already a way, I'd love to be able to read environment variables into a couple of my expressions.

Also, I've recently hit a situation where the ability to do date calculations would be super helpful. Even if you could just clue me in to how to manipulate the internal floating-point value to add/subtract seconds?
Logged

mattkhan

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 4216
Re: Expression functions [HELP WANTED]
« Reply #168 on: June 01, 2020, 03:01:14 pm »

Even if you could just clue me in to how to manipulate the internal floating-point value to add/subtract seconds?
It is days since epoch where epoch is December 30th, 1899 at 00:00:01

+1s is therefore +1/86400
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #169 on: June 01, 2020, 08:10:25 pm »

Still taking requests? If so, if there isn't already a way, I'd love to be able to read environment variables into a couple of my expressions.

I'll add this tomorrow.  Thanks for the suggestion.
Logged
Matt Ashland, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Expression functions [HELP WANTED]
« Reply #170 on: June 01, 2020, 08:12:52 pm »

Also, I've recently hit a situation where the ability to do date calculations would be super helpful. Even if you could just clue me in to how to manipulate the internal floating-point value to add/subtract seconds?

Also, if you want to do math on the floating point number, you need to use the raw format of the field. i.e. [Date,0] shows the raw floating-point number.

So to add 35 seconds to a date stored in the [Date] field: Formatdate(Math([Date,0]+(35/86400)), Date HH:mm:ss)
Or to add 35 seconds to the current time: Formatdate(Math(Now()+(35/86400)), Date HH:mm:ss)  That works because Now() is a function not a field, and returns a floating-point number and not a formatted date.

Also, I think the epoch begins at December 30th, 1899 at 00:00:00, as best I have determined it from past testing in MC. The floating-point number calculation never yields an exact zero value. But what is a second among friends?  8)
I always thought that was a strange date and time to start from, rather than December 31th, 1899 at 24:00:00 (January 1st, 1900 at 00:00:00), but maybe that was an error in counting leap years or something... Oh, I see it is the Microsoft COM DATE epoch, and the day shift was because of a leap year!
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Expression functions [HELP WANTED]
« Reply #171 on: June 01, 2020, 09:15:21 pm »

The tooltip for ListMix() includes information and examples from FormatNumber. A copy/paste error.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Expression functions [HELP WANTED]
« Reply #172 on: June 01, 2020, 11:52:12 pm »

Try this for some magic:
Code: [Select]
Replace(Replace(ListMix(if(Compare(find(/#[L1]#/,!People\),>=,0),Replace(/#[L1]#/,!People\,),@@), 0, [keywords]),; @@,),@@; ,)

I tested and deconstructed this expression because I thought I would learn something.

Some notes;
1. This expression will only work correctly on MC26.0.84 and earlier, because of this change in MC26.0.85.
Quote
26.0.85 (5/27/2020)
5. Changed: Standardized on ";" for the list delimiter instead of "; " for all the expressions.
2. While it might be the safest thing to do, and doesn't seem to change the result, I found that the [L1] components didn't need to be escaped using the /#...#/ pair.
3. This works on the sample [Keyword] list Marko provided. I didn't test beyond that.
4. The expression "fails" and returns "@@" when no instances of "!People\" are found. Fixed by wrapping another Replace() function around it.

What I learned;
a. I was trying to work out what the @@ operators were supposed to do... then realised that they are just placeholders for the Replace functions later.
b. This only works because ListMix() is a list manipulation function, and passes the [L1] value down to functions within its scope.
c. This type of expression does my head in, and I have to read it s.l.o.w.l.y.
d. Version 2 of the Syntax Highlighter for JRiver Expression Language makes reading this code so much easier. Thanks Wer!

So on MC26.0.85 and above, use;
Code: [Select]
Replace(Replace(Replace(ListMix(if(Compare(find(/#[L1]#/,!People\),>=,0),Replace(/#[L1]#/,!People\,),@@), 0, [keywords]),;@@,),@@;,),@@,)or
Code: [Select]
Replace(Replace(Replace(ListMix(if(Compare(find([L1],!People\),>=,0),Replace([L1],!People\,),@@), 0, [keywords]),;@@,),@@;,),@@,)

Thanks for sharing Zybex.

But we really need to get the Regex mode -2 tested and working... which is what I was going to look at today... but Rabbit Hole, down I went.  ;D
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #173 on: June 02, 2020, 02:06:01 am »

But we really need to get the Regex mode -2 tested and working... which is what I was going to look at today... but Rabbit Hole, down I went.  ;D

The reason I proposed what has been christened the -2 mode in the first place was for exactly situations like this.  Without it, the Regex() cannot take an industry standard regex expression and return all matches at once against a variable length string.  The -2 mode should make operations like this simple.

I haven't been able to test it however because it's still in beta only.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #174 on: June 02, 2020, 02:16:20 am »

Thanks RoderickGI, nice analysis :)

The change to ";" will surely break some code out there. I agree with it though.
I really dislike chained functions like Replace(Replace(Replace...), but most times it's the only way. For me it highlights the need for some extra functions/modes:

- Trim(expression) could take a second argument to Trim that instead of whitespace. Trim(expression, ; ) would trim semicolons on both sides

- ListClean() needs a mode 2 to remove empty elements from a list

- ListGrep(list, value) to return list elements containing "value". It would simplify this expression tremendously. Alternative name: ListFilter()

Regex mode -2 can be used to do this, but it would need some really awkward regex expressions in most cases.

Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #175 on: June 02, 2020, 02:54:32 am »

The -2 mode should make operations like this simple.

It should work, but the regex expressions needed can become quite tricky. For this case, it would be something like this (The RemoveRight() it to remove the last semicolon):
Code: [Select]
RemoveRight(Regex([Keywords],/#[;^\s]+!People\\([^;]+;)#/,-2), 1)
Or a general expression to filter a list for elements containing "word":
Code: [Select]
RemoveRight(Regex([Keywords],/#[;^\s]+(.*word[^;]*;)#/,-2), 1)
I've just tested this on the latest beta and mode -2 only returned the first match. Seems it's not working 100% yet.

EDIT: Perhaps I misunderstood how -2 works or how it's implemented. After some more tests, I see it's currently working like this:
a) Regex(this 123 sample 456 text, /#(\d+)#/, -2)               => this outputs 123 (expected: 123;456)
b) Regex(this 123 sample 456 text, /#(\d+)\D+(\d+)#/, -2)       => this outputs 123;456

I expected that -2 mode with a) would output all \d+ occurrences.
The b) case is not as useful as it requires you to know the exact layout of the string you're searching. It's not possible to apply -2 to a list of random length to filter the elements, for instance - my regex examples above to filter the lists will not work with this current -2 mode.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #176 on: June 02, 2020, 03:37:31 am »

This is a teaser for something I'm doing - yesterday I started building an interactive Expression Language editor where you can just type and immediately see the results.
This is still very unpolished (half-day work), but check out the 2 screenshots. One shows a rendered Tooltip expression as it would show on the Movie Tooltip view, the other one is a scratchpad I used to test several different expressions.

Planned features:
- instant evaluation (by MC via the API, I'm not reimplementing the language)
- syntax highlighting, color customizable
- rendered (HTML) and Text output views
- HTML tag support: font, b/u/i, img with full src path or "tooltip:" for thumbnails, logos, etc
- multiple scratchpad tabs (like notepad++)
- test expression against any movie (currently this in running inside ZRatings, and when you change to some movie in the main Datagrid the expression re-evaluates for that movie)
- function argument syntax/help
- insert fields and function templates from a list
- color selector for <font> ... WYSIWYG is possible, but too much work for now.

Hopefully next weekend I'll have some test version available.
Feedback so far?
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #177 on: June 02, 2020, 04:03:22 am »

EDIT: Perhaps I misunderstood how -2 works or how it's implemented. After some more tests, I see it's currently working like this:
a) Regex(this 123 sample 456 text, /#(\d+)#/, -2)               => this outputs 123 (expected: 123;456)
b) Regex(this 123 sample 456 text, /#(\d+)\D+(\d+)#/, -2)       => this outputs 123;456


Zybex, if -2 is working as I intended, it would output 123;456 as per your example A.  That was the whole point of it.

The functionality described in B is useless, because it cannot be used against a string with an unknown number of matches (and it uses non-standard syntax of course).  What you describe in B is also redundant with the already existing R1..R9 functionality.

I can't comment on what Matt actually implemented because I haven't seen the beta, but returning an arbitrary number of matches with a single regex was the goal.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #178 on: June 02, 2020, 09:20:52 am »

One new function proposal that would help here:

ListGrep(list, text, mode)

Returns all elements of list which contain text.
Mode 0 - case insensitive
Mode 1 - case sensitive

ListGrep(redneck;blue;red roses, red) = "redneck; red roses"

Then the above code could be reduced to
Code: [Select]
Replace(ListGrep([Keywords], !People\), !People\,)

Next build:
NEW: Added a ListGrep expression function to filter a list to a matching string.

Thanks for the idea.
Logged
Matt Ashland, JRiver Media Center

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #179 on: June 02, 2020, 11:13:15 am »

Next build:
NEW: Added a ListGrep expression function to filter a list to a matching string.

Thanks for the idea.

Brilliant. This will be useful in three or four areas for me. Thank you both.

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #180 on: June 02, 2020, 11:17:46 am »

This is a teaser for something I'm doing - yesterday I started building an interactive Expression Language editor where you can just type and immediately see the results.
<snip>
Hopefully next weekend I'll have some test version available.
Feedback so far?
Looks amazing. Would help enormously when compared to tweaking in notepad++, copy, paste into MC, go back, tweak more, copy, paste, etc.

Will it only work on movie fields?

@Rod,
Thanks for the breakdown and amended expression, which works flawlessly.

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #181 on: June 02, 2020, 11:26:21 am »

It works on any [field], but it's always in relation to some "File" entity (movie/episode/song/...). Not sure how it behaves with the GroupQuery functions.

The expression is evaluated by MC, so whatever works on MC should work here - unless MC has some extra functionality which is not exposed on the API, which is very much possible.

Still, even like it is I find it's already very useful.
Logged

Doof

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5908
  • Farm Animal Stupid
Re: Expression functions [HELP WANTED]
« Reply #182 on: June 02, 2020, 03:15:47 pm »

It is days since epoch where epoch is December 30th, 1899 at 00:00:01

+1s is therefore +1/86400

you need to use the raw format of the field. i.e. [Date,0] shows the raw floating-point number.

D'oh! I had found an earlier thread and/or wiki article that clued me into the 1/86000 of the epoch value but I couldn't ever get the the math to work. It was this last bit about the raw value that I was missing! Thanks guys! This is going to VASTLY simplify a few of my expressions.

I'll add this tomorrow.  Thanks for the suggestion.

Thanks Matt!
Logged

Doof

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5908
  • Farm Animal Stupid
Re: Expression functions [HELP WANTED]
« Reply #183 on: June 02, 2020, 04:06:56 pm »

Is there any way to output information from a different file by referencing its key or another identifier?
Logged

Doof

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5908
  • Farm Animal Stupid
Re: Expression functions [HELP WANTED]
« Reply #184 on: June 02, 2020, 09:53:22 pm »

GroupCountQuery() seems to be a pretty taxing operation for MC to perform on a... largish (200k) library. Could it be sped up at all if we could limit the number of files the query needs to run against? For instance, a query like GroupCountQuery([Album], [Track #]) could be limited to only searching within music files by doing something like GroupCountQuery([Album], [Track #], [Media Type]=[Audio] -[Media Sub Type]=[Audiobook]). Would something like that speed it up?
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #185 on: June 02, 2020, 10:03:17 pm »

It's puzzling why GroupCountQuery should be significantly slower than the filtering done in a panes view.  It's essentially the same search against the db, and filtering in a panes view is extremely speedy.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #186 on: June 02, 2020, 10:24:52 pm »

I can confirm Zybex's findings, that Regex -2 is not working as intended.

The intention was to return all matches of an arbitrary length string that contained an indeterminate number of matches.

For example, in the Boost regex engine (used by MC) we can do the following:

Input: Humphrey Bogart [Role One];Sydney Greenstreet [Role Two];Mary Astor [Role Three]
Regex: [\w\s]+(?=\s\[)
Output(semicolon delimited): Humphrey Bogart;Sidney Greenstreet;Mary Astor

That one regex would output all the actor names in such an input, regardless of whether there was one actor or 20 listed.

The above is exactly what Regex-2 should do.

However, the Regex-2 function only outputs the first match, Humphrey Bogart.

As implemented, to use Regex-2, you must still possess precognition, because you have to know in advance the number of matches that will exist.  This does not work with lists (strings) with a variable number of elements.  You must still provide a number of redundant capture groups EXACTLY equal to the number of matches to be found in that particular string. If you have too few groups, you will not get all matches. If you have too many groups, the Regex() function returns null.  (And that was exactly the problem with the existing R1..R9 functionality.)

This is not helpful, as it is redundant with the pre-existing Regex() functionality and the way it loads results into the R1..R9 variables. Plus, in addition to requiring prior knowledge of the number of captures present, you have to produce a somewhat ridiculous regular expression with multiple redundant capture groups.

The purpose of this enhancement to the Regex() function was to get around those constraints.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #187 on: June 02, 2020, 11:55:43 pm »

I've found some issues with GroupSummaryQuery, Matt...

GroupSummaryQuery does not work correctly sometimes IF the field name has spaces in it. 

GroupSummaryQuery(Album Artist,Duration) can be used, but GroupSummaryQuery(Artist - Composition,Duration) can not. [Artist - Composition] is a custom field. Another example, custom field [Artist-Album] can be used, but the identical field [Artist - Album] can not.

The [Artist - Album (Year)] field also can not be used.

My personal favorite because it is so strange:
GroupSummaryQuery breaks (returns null) if the VALUE of the grouping field contains a letter with an umlaut (e.g. Tannhäuser).  So, GroupSummaryQuery(Composition, Duration) works correctly if [Composition]=Tannhauser but breaks if [Composition]=Tannhäuser    A double quote character " has the same effect.
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9133
Re: Expression functions [HELP WANTED]
« Reply #188 on: June 03, 2020, 01:09:40 pm »

One new function proposal that would help here:

ListGrep(list, text, mode)

Returns all elements of list which contain text.
Mode 0 - case insensitive
Mode 1 - case sensitive

ListGrep(redneck;blue;red roses, red) = "redneck; red roses"

Then the above code could be reduced to
Code: [Select]
Replace(ListGrep([Keywords], !People\), !People\,)

This is working an absolute treat. Thank you very much to all involved.

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #189 on: June 03, 2020, 05:24:48 pm »

Maybe the regex needs to empty the array between runs?
Logged
Matt Ashland, JRiver Media Center

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #190 on: June 03, 2020, 05:29:50 pm »

Maybe the regex needs to empty the array between runs?

I'm not sure what this is in response to.  I don't see the connection to what Zybex and I were seeing with Regex-2.  Is there something we need to explain better?
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #191 on: June 03, 2020, 06:29:40 pm »

Could someone post a simple RegEx(..., -2) example that isn't working properly?
Logged
Matt Ashland, JRiver Media Center

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #192 on: June 03, 2020, 06:34:41 pm »

Sure, that's what my previous post had.  Here it is:

Input: Humphrey Bogart [Role One];Sydney Greenstreet [Role Two];Mary Astor [Role Three]
Regex: [\w\s]+(?=\s\[)
Output(semicolon delimited): Humphrey Bogart;Sidney Greenstreet;Mary Astor

That one regex would output all the actor names in such an input, regardless of whether there was one actor or 20 listed.

The above is exactly what Regex-2 should do.

However, the Regex-2 function only outputs the first match, Humphrey Bogart.

Another example from Zybex's post:

a) Regex(this 123 sample 456 text, /#(\d+)#/, -2)               => this outputs 123 (expected: 123;456)
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: Expression functions [HELP WANTED]
« Reply #193 on: June 03, 2020, 06:46:09 pm »

I'm not clear on this example:
Regex(this 123 sample 456 text, /#(\d+)#/, -2)

Isn't there only one capture, so outputting 123 is right?
Logged
Matt Ashland, JRiver Media Center

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #194 on: June 03, 2020, 06:52:17 pm »

No.  \d+ matches all numbers in the string.  We're trying to return ALL matches.

The current behavior of -2 is functionally equivalent to the existing R1..R9 behavior, in that you have the following problem:

Write a single Regex function that obtains all the numbers from the [Data] list. The data field may have a different number of elements for each file, as follows:

123 Main;1000 Oak;500 people
2000 years;2500 dollars;12 cars;13 ghosts of scooby doo
4 calling birds;3 french hens;2 turtle doves;11 lords a leaping;5 gold rings; 6 geese a laying

Currently, that is impossible. In MC, a different Regex expression is required for each of those lists, to match the exact number of numbers present in each string.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #195 on: June 03, 2020, 06:55:39 pm »

Please look at this example:

https://regexr.com/5628d

All numbers are matched (highlighted) with the single expression.

Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #196 on: June 03, 2020, 06:55:56 pm »

Matt,
Check the boost::regex_iterator:
https://stackoverflow.com/questions/2593288/how-to-use-c-boosts-regex-iterator

This is about repeating the regex capture on the string until there are no more matches.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Expression functions [HELP WANTED]
« Reply #197 on: June 03, 2020, 08:54:56 pm »

Exactly.  It's about returning all matches for a capture group, as the /g (global) flag would.

There is also the  match_results::captures function.  Take a look here, in the section on repeated captures:
https://www.boost.org/doc/libs/1_33_1/libs/regex/doc/captures.html

Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10925
Re: Expression functions [HELP WANTED]
« Reply #198 on: June 04, 2020, 02:28:09 am »

We don't use boost. Haven't for years.
Logged
~ nevcairiel
~ Author of LAV Filters

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2591
Re: Expression functions [HELP WANTED]
« Reply #199 on: June 04, 2020, 02:55:52 am »

We don't use boost. Haven't for years.
Good for you, that thing is getting bloated.

std::regex_iterator / std::regex_token_iterator does the same.
Logged
Pages: 1 2 3 [4] 5 6 7 8 9   Go Up