INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2] 3   Go Down

Author Topic: Feature Request: Writing to Expression Fields  (Read 8173 times)

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #50 on: September 22, 2020, 12:49:13 am »

So, with all of this in mind, I think actually describing the change to the UI of the Library Field Manager might be the simplest way to explain the idea fully. And, looking at it closely, I think you can fix a few other organizational issues in there that have crept up over time, without going too crazy and revamping the whole thing (which is good because UI programming is a PITA).

How about this?

_Name_ - Unchanged
_Data_
 [✓] User Data - when unchecked, disable indented below and Input Expression.
     Data Type [__]
     Relational [__]
     Acceptable Values [__]
     [✓] Save in file tags (when possible)
 [✓] Input Expression - when unchecked, disable indented below
     [expression editor] - See Note 1
 Edit Type [__] - Always enabled.
 [✓] Show a Search Link - Always enabled. See Note 2
 [✓] Output Expression - when unchecked, disable indented below and enable User Data above.
     [expression editor] - See Note 3
_Search_ - Unchanged


Notes:
1: This is the Setter expression, which allows you to transform the user's input before it is saved. This expression is evaluated when the user edits the value, and the result is saved to the User Data variable.

In an Input Expression, if you use square bracket [This Field's Name] notation, it substitutes the input from the user. If you use [This Field's Name,0] notation, then it substitutes the current value of the User Data variable (before the edit).

Since you can only enable this if User Data is also checked, you could put it "under" User Data in the grouping hierarchy. But, doing it this way allows you to get rid of the gigantic Expression: label and that wasted space. Instead you can expand the width of the expression edit box to nearly the whole width of the panel. Which also probably lets you make them a bit shorter, and so easier to fit the two expression editor boxes in the same overall Library Panel size. And, it'll look nice and symmetrical with the Output Expression box.

2: This is the existing [✓] Show a link checkbox item, which is clumsily named (and it isn't even a checkbox anymore, it is an arrow).

3: This is just the existing Calculated Data expression editor box, exactly as it works now, except you can enable it along with User Data. When you do, the field outputs the output of the expression instead of the User Data value.

If you use square bracket [This Field's Name,0] in the expression then it substitutes the "underlying" User Data value. I'd say that if you use [This Field's Name] notation (without the ,0) then it should just do the same thing. But maybe returns blank? In any case, if [✓] User Data is unchecked (so, the equivalent of a Calculated Data field now), then both substitutions return a blank value.

You can save the space of the giant Expression label here too, and make the box wider (and possibly a bit shorter).

I think that would do it. It would be clean, and allow cleanup of some strange enabling/disabling behavior and hierarchical positioning in the current Library Field Manager. And it is easier to understand than the current setup.

It also isn't a dramatic change in the UI. It would be easy to convert the settings upon upgrade, because the all current Calculated Data expression just become [✓] User Data: unchecked and [✓] Output Expression: checked. Everything else stays the same.

I like it. A lot.

Edits: The forum list formatting was making me crazy. I'm sorry, I'll stop poking at it now.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #51 on: September 22, 2020, 02:18:06 am »

Great, my brainwashing finally worked  ;D

I admit I also skimmed some of the longer posts, and while trying to write small/concise explanations for the idea I probably didn't explain it as well as I could. In the end we got to this second page with a consensus, which is nice  8)

I saw that the "Calculated Field" type could be generalized as you did above, but didn't want to say it because I thought it would further complicate the discussion. It's an obvious corollary though, glad you saw it too.

The mockup above is cool! Notes on that:
- "Edit Type" is currently always enabled, but I think it's a bug in the current UI. It makes no sense to have it for Calculated fields. I tried it, and it does allow you to edit the value of a Calculated field... but then nothing happens, the value is just discarded. So this option should be under "User Data" and be enabled/disabled along with the rest.
- I prefer "Display Expression" and "Commit Expression", but Input/Output also works
- "Show a Search Link" could go under the Search section

Other notes and problems:
- Backup/Restore work at the DB level so they should always bypass the expressions and just backup/restore the raw field values;
- same for sidecard export/import
- However, CSV export is ambiguous... we might want to export the Display value or the Raw value, depending on needs/context/field.
- What about when Importing files? The imported tags should be considered Raw values, or should they go through the Commit Expression?
- Conversely, which value should go into an MP3 when tagging it from Field values?
- Same for the File Move&Rename tool, though there we could use the [field, 0] syntax as needed I guess.

I can see other problems with the Setter implementation, but I'll leave those aside for now.

[img width= height=]https://i1.wp.com/blogs.perficientdigital.com/files/2011/07/treecomicbig.jpg[/img]
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #52 on: September 22, 2020, 02:27:24 am »

I think the developers would need to fully evaluate all the implications of rearranging the dialogue, but I'd be happy enough if that was done.

Some feedback:
I'm not real happy with the name "Output Expression". I am thinking of the majority of users, and how they would interpret that name. We have been using "Display Expression", or at least I have, because it immediately tells me what this expression does: It defines what is displayed for the field, based on an expression, which implies on the screen. "Output" could mean the output of my data entry. The question is bound to arise, "Which direction is output?" I'm not going to get on my soapbox over the name, but that is my feeling about the currently proposed name.

"Input Expression" is far less of a concern. I used "Commit Expression" because I have a tiny bit of a database background, and Commit to me means something being saved to a database. "Save Expression", as in "What is saved to the Library" might be more meaningful to the majority of users. But I could live with any of those three.

I am not as convinced that the "Output Expression" with the "User Data" flag turned off is exactly the same as the current "Calculated Data" functionality. Your terminology is part of the problem here. I don't really know what you mean by "User Data Value" in Note 3. I think it should mean the current content of the field. But it could be what the user just entered on screen, or it could be what the user put in the field previously.

Regardless, the current "Calculated Data" functionality is, as you have noted, read-only and any data input by the user into a Calculated field is basically ignored and discarded. The contents of such a field is always equal to the result of the expression defining it. As such, it isn't just a manipulation of the data stored in the Library field for display, but is in fact what is stored in the Library field. For a Calculated field called xxx, there is no difference between the value returned using [xxx] and [xxx,0]. The Raw data is always the same as the Display data.

Now, if not having the "User Data" field checked changed the way the "Output Expression" box was used, and it now acted exactly as the current "Calculated Data" functionality, I would be happy with that. Re-using dialogue realestate is a good thing, when functionality is mutually exclusive. However, if the box was used that way, I would like to see the box heading, "Output Expression" or "Display Expression" set to "Calculated Data" until the "User Data" field is checked, to avoid confusion. Or maybe it would be easier to just continue using a radio button for "User Data" and "Calculated Data", and just have the expression entry box collapse when appropriate. Again, a JRiver decision I think.

My main point there is that the "Display Expression" as we have discussed it is not equivalent to the "Calculated Data" functionality as it exists today.


Otherwise, I'm happy.  :D  Except you kept editing your post as I was reading it Glynor! I had to keep refreshing to be sure nothing changed!  8)


PS: Zybex beat me. It seems we need to discuss "Calculated Data" further. Cool. We are getting there.

My starting point with the export, import and tagging question would be that they always use the raw data. But they would all need to be looked at carefully. Basically, we are just looking at taking raw data and changing how it is displayed, and taking data entry and changing how it is saved. The underlying data should be consistent with external data sharing. Think about the [Date] field as an example, where it is the raw data written to tags, except where there is special code to write that data to different file tags, which may alter it before writing. Usually though that is just reading and writing the raw data to different tag names, rather than changing the data.

PPS: I'm the Business Consultant Zybex.  ;)
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #53 on: September 22, 2020, 02:35:17 am »

Quote
PPS: I'm the Business Consultant Zybex.
I'll get started on the documentation.  ;D
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #54 on: September 22, 2020, 02:40:30 am »

After thinking more on the "Calculated Data" issue, you might be able to convince me.

I always thought of Calculated fields as holding content, being the calculated value. But maybe they don't, or maybe they don't need to. They are always re-calculated whenever they are accessed, so essentially thinking about it that way, it can be just a Display Expression. As long as the [field name] and [field name,0] functions work correctly, with the field being recalculated for display every time it is referenced, I can see that there would be no real difference.

I guess that depends on how the "Calculated Data" functionality works under the hood now. I don't know what it is doing, whether it is saving a value to the field for use in Views and so on, or recalculating every time.


Zybex, the documentation seems like a fair share of the workload for us.  :D
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #55 on: September 22, 2020, 02:46:13 am »

I meant Documentation as in the cartoon above ;)

Calculated fields are calculated on access, there's no stored value. Perhaps MC does cache the latest calculated value for performance, but that's an implementation detail. In effect, it's as if there's no stored value.

I don't even think that's the case - try this: add an Expression column with expression="counter()". This just increments the value each time the expression is recomputed. Now... just HOVER the mouse over this expression column, moving it up and down over a few files, or click on some files... and the values change like crazy, indicating that the expression is being recomputed. Highly inefficient, but perhaps there's a reasoning for that that we are not aware of.

Anyway, for all intents and purposes, the traditional "Calculated Field" is the same as just setting the "Display Expression".
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #56 on: September 22, 2020, 03:16:57 am »

I meant Documentation as in the cartoon above ;)

Yes, I understood that. No work seems fair to me!


try this: add an Expression column with expression="counter()".

Yes, I was aware of that. But that is an expression column, which could be handled differently to a Calculated field. Let me check something... Okay, a Calculated field works the same way. I set up a field with an expression of Now() and it continuously updates. In fact, it updates on any screen refresh, such as moving a divider, not just on mouse over.


Anyway, for all intents and purposes, the traditional "Calculated Field" is the same as just setting the "Display Expression".

As that does sound like the case, I guess my concerns are satisfied on that point.

I wonder what JRiver thinks of all this?
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #57 on: September 22, 2020, 03:33:47 am »

I wonder what JRiver thinks of all this?

[Jim] Hmm, there seems to be some heated discussion around this feature request, doesn't look like everyone agrees. I'll skip this thread.
[Matt] Jeez, so many huge posts! Can't these guys make simple requests??? I'll wait for Jim to say something.
[Yaobing] CTRL+F, "TV"... no hits, I can skip this thread.
[Hendrik] *** users! ... back to Warcraft.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #58 on: September 22, 2020, 08:53:08 am »

I'm not real happy with the name "Output Expression". I am thinking of the majority of users

I didn't finish reading, but I just wanted to make two comments:

1. I don't care very much what names they use.
2. I do think, Display Expression, is particularly bad. Because that is no small part of what it was confusing me all this time. I was hung up on the word "Display".

It isn't just for display. That's why I thought it needed "new functionality" to implement the idea. Because I thought you were saying "apply this expression ONLY when the field is displayed" (in the panes and whatnot), but in other expressions treat it as the Value. So, in a pane, grid-view, etc [Artist] would show as "Swift, Taylor" but in an Expression, it would substitute as "Taylor Swift".

MC can't do that now (output of the field is the output of the field, and a field only has one default output which applies both to display and to expressions). And that would have been very confusing and difficult to use. Because it implies that the expression will only be used "on display", which it can't be (or the whole thing falls down - see basically all of my arguments for this entire thread).

I chose Output Expression because that's what it does: An expression that determines what the field outputs. And it matches nicely with Input Expression, which is an expression that filters the user's input.

Again, though, I'm not that picky. But, based on my own experience here, I think "Display Expression" is a particularly bad name.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #59 on: September 22, 2020, 09:06:30 am »

- "Edit Type" is currently always enabled, but I think it's a bug in the current UI. It makes no sense to have it for Calculated fields.

Sure does. I use it all the time.

Make an expression that outputs 0 or non-zero (usually one, but it doesn't matter, blank or something works too). Set the Edit Type to Check. You have an expression field that outputs a row of checkboxes. Same goes with Five and Ten Stars, Large Value, and List. Clear-Only isn't useful now, but if you could input on Expressions, it would be (so that you could use it like an informational display + reset control).

It is just annoying that the system doesn't "know" they're read-only checkboxes (which is part of what I was complaining about above). But, this would solve that. If you have only Output Expression checked and not User Data (the equivalent of current Calculated Data fields) then you display using the selected Edit Type but always read-only the editors.

You used to not be able to access that setting for Calculated Fields, and they added it back in on purpose (after much complaining by, in no small part, me). That is part of the reason the current Library Field Manager layout is so wonky (same goes with the Search Links thing).
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #60 on: September 22, 2020, 10:07:23 am »

- "Show a Search Link" could go under the Search section

Sure. That would work too.

Other notes and problems:
- Backup/Restore work at the DB level so they should always bypass the expressions and just backup/restore the raw field values;

Well, a Library Backup also includes all Field definitions (including current Calculated Fields), Views, and everything else too, so yes, but it would include both.

- same for sidecard export/import
- However, CSV export is ambiguous... we might want to export the Display value or the Raw value, depending on needs/context/field.
- What about when Importing files? The imported tags should be considered Raw values, or should they go through the Commit Expression?
- Conversely, which value should go into an MP3 when tagging it from Field values?

If Save to File Tags is enabled, it should save the raw values to Sidecars and file-tags. That's the "stored data". You don't save current Calculated Fields to tags (that option stays off if User Data isn't enabled) so, that stays the same.

CSV exports should get the final, formatted values, IMHO. That's how it works when you copy/paste to Excel from a view onscreen, and CSV export is the same thing. If you want to export the "raw" User Data values, then you export [Field Name,0].

- What about when Importing files? The imported tags should be considered Raw values, or should they go through the Commit Expression?

My opinion here, but it isn't super-strongly held, is that it should absolutely follow the Input Expression, just as it would when entering User Data "manually". In fact, doing this would make my Tag on Import rules quite a bit simpler, because I could just put the logic for all of the stuff it does now to massage "it's own field" in the Input Expression instead.

Tag on Import wouldn't have to change, mind you. You just wouldn't need to do those kinds of things there.


Nope. All of that is wrong and backwards. You'd NOT want to apply the Input Expression for new file imports, as that would double-convert values like "Taylor Swift" (already in the right format).

Imports ignore Input Expressions (which only apply to user-edits).

- Same for the File Move&Rename tool, though there we could use the [field, 0] syntax as needed I guess.

Definitely use the [,0] notation to access the "raw" User Data values everywhere in MC, including RMCF. I absolutely want to use this for RMCF. That's a big hunk of what I want to accomplish with it, and I want to be able to access both raw User Data (with [,0] and "normal output" via []).

[Field Name] always accesses the formatted Output. [Field Name,0] always accesses the User Data (if enabled). If User Data isn't enabled, then [Field Name,0] outputs blank.*

The only exception would be my two notes, in the expressions used where it refers to itself. There, you can't actually reference the field's own output in the Output expression, as that's a loop, and you need a way to "grab" the user's input for the Input Expression. So, my suggestion for the syntax for these is:
  • Input Expression - [Field Name] is user input, and [Field Name,0] is the current User Data value (before the edit).
  • Output Expression - [Field Name] and [Field Name,0] are equivalent and both show the User Data value (or blank if User Data is disabled).
For the latter, you could always "force" the author to use [Field Name,0] and have [Field Name] just return blank. But I'd say just make them both do the same thing for simplicity of use.

* Edit: Or you could have [Field Name,0] output the equivalent of [Field Name] if User Data is disabled. I think it might be nice to be able to get "blank" for these in some cases (and more consistent throughout MC), but I don't have a strong opinion here.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #61 on: September 22, 2020, 11:19:30 am »

[Jim] Hmm, there seems to be some heated discussion around this feature request, doesn't look like everyone agrees. I'll skip this thread.
[Matt] Jeez, so many huge posts! Can't these guys make simple requests??? I'll wait for Jim to say something.
[Yaobing] CTRL+F, "TV"... no hits, I can skip this thread.
[Hendrik] *** users! ... back to Warcraft.

I think this is probably fairly accurate.

Once we decide on everything and agree on all of the little edge points (and I think we're very close, if not there now), I intend to start a new thread with our final proposal (and link back to here for the original discussion). Or just poke them and point to a single, definitive post at the end, if we can make one.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #62 on: September 22, 2020, 12:28:13 pm »

Make an expression that outputs 0 or non-zero (usually one, but it doesn't matter, blank or something works too). Set the Edit Type to Check. You have an expression field that outputs a row of checkboxes.

Thanks, TIL.

Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #63 on: September 22, 2020, 12:43:16 pm »

I always thought of Calculated fields as holding content, being the calculated value. But maybe they don't, or maybe they don't need to. They are always re-calculated whenever they are accessed,

You were thinking about it wrong. They are not now the way you envisioned it in your mind. There is no data in the database for Calculated Fields (other than the definition of the field). They are calculated when accessed.

Calculated Fields as currently implemented already are the Output Expression we've been discussing. They're identical.

That's why my original idea would have worked. But this is better and cleaner.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Feature Request: Writing to Expression Fields
« Reply #64 on: September 22, 2020, 12:44:17 pm »

I actually like the idea you all have come up with. It would definitely be useful in certain situations, and a lot of good thinking has gone into this thread.

I'd be a little trepidatious about really asking for it though. It is a complicated request, that would serve a niche purpose for a limited number of people.  The change goes deep. There are many other things that have been needed and ignored for years that would be more beneficial to a larger number of people. Just a couple of examples: making dsp presets work more effectively; and providing better built in support for the multi-track compositions of classical music. There are others of course. Not to mention the D word, which will never happen anyway.  If a group of influential individuals is going to come together around a big idea to push for it, I would want it to be something that would provide a big benefit to a lot of people.

I'm just worried about the distraction of resources it would cause. Cost/benefit.  I like the idea though, so please don't think I'm denigrating it.  Thanks to all of you for the effort you put in on this thread.

-Will
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #65 on: September 22, 2020, 12:48:52 pm »

I'd be a little trepidatious about really asking for it though. It is a complicated request, that would serve a niche purpose for a limited number of people.  The change goes deep.

Not really.

We need three basic things:
1. The Input Expression functionality.
2. Change the Library Field Manager Calculated Data and User Data setups from radio buttons to checkboxes (so that you can check one, the other, or both simultaneously.)
3. The syntactic sugar for [Field Name]. And this only applies to the Input and Output expression definitions, really. The [Field Name] and [Field Name,0] syntax already exists and already does what it does.

The rest of the rearranging of the dialog I put above is all to fix issues which already exist in the setup as things have changed over time and Calculated Fields got access to the Edit Type and Search Link functions.

That's pretty much the beauty of the idea. It doesn't go extremely deep. It is the Input Expression (the Setter functionality) and the ability to enable both Output Expression (aka Calculated Field mode) and User Data at the same time.

Everything else stays exactly the same. All of your existing Field Definitions stay exactly the same (just the radio buttons turn into checkboxes, but unless you go in and fiddle them, they'll be the same as they were).
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #66 on: September 22, 2020, 01:17:07 pm »

3. The syntactic sugar for [Field Name]. And this only applies to the Input and Output expression definitions, really. The [Field Name] and [Field Name,0] syntax already exists and already does what it does.

It is worth discussing this briefly.

So, the idea would be if you define a field where you check both: [✓] User Data and [✓] Output Expression that you'd use:
[Field Name]: result of Output Expression
[Field Name,0]: value of the stored User Data variable

That's clear, and does exactly what it does now (you just can't actually "check both boxes"). That is the key piece that makes the idea work (hence my "ah-ha moment" earlier).

The only part where it gets clouded is using self-references to the field in your Input and Output Expressions. For the Input Expressions, [Field Name] wouldn't make sense. For the Output Expressions, using [Field Name] also doesn't make sense (it would be a self-referential loop). But that's already the case now (so no change there).

But, you need a way to get input from the user in order to act on it in your Input Expression. So, I figured re-use the otherwise useless [Field Name] notation on the Input Expression to allow you to access the user input. But that's just syntactic sugar.

But, maybe that is confusing the issue? Underneath the covers, it is effectively going to point to a new Input() function, and wrap it in square-bracket [Field Name] notation. You could make it use [Input] notation instead and leave the [Field Name] in input and output expressions alone. The only problem with that is it does introduce a "breaking" change for some people, because if you already have a field called [Input] then you're screwed. And, the only place where you'd ever need the Input() function and its square-bracket version would be in the Input Expression definition.

So, doing the sugar and using the [Field Name] notation to refer to Input() is better, IMHO: It reads better in the expressions, and it doesn't cause that issue.

Thoughts?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #67 on: September 22, 2020, 01:50:04 pm »

I think it's fine to reuse the [Field] notation. Something like this:

                      [Field]            [Field,0]
Input Expression:     newValue           Field.Value
Output Expression:    Field.Value        Field.Value
Elsewhere:            Field.Output()     Field.Value

- newValue is the user input being processed for storing
- Field.Value is what is currently stored (raw value)
- Field.Output() is the Output function (or Field.Value if no Output expression is defined)
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #68 on: September 22, 2020, 01:59:30 pm »

Wer, it's not that complicated to implement as it is now - it's simpler than the initial proposal from Glynor; it's fundamentally just adding a couple of new Field options, tweaking the Field Getter/Setter, and some minor UI design.

In the end it's up to JRiver folks to prioritize their TODO list. Personally, I don't think this feature will rate highly on their queue... but it's worth a shot.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #69 on: September 22, 2020, 03:09:14 pm »

I think it's fine to reuse the [Field] notation. Something like this:

                      [Field]            [Field,0]
Input Expression:     newValue           Field.Value
Output Expression:    Field.Value        Field.Value
Elsewhere:            Field.Output()     Field.Value

- newValue is the user input being processed for storing
- Field.Value is what is currently stored (raw value)
- Field.Output() is the Output function (or Field.Value if no Output expression is defined)

Yup. That's exactly what I was thinking.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #70 on: September 22, 2020, 04:53:41 pm »

I agree CSV exports and Copy/Paste should be the formatted value, [Field Name], which are the displayed values as a result of the Output Expression. Users would expect the displayed value to be what shows up in a CSV file, or the clipboard.

The Input/Output, Commit/Display Expression naming doesn't matter too much to me, as long as it is properly documented. I was coming from a user perspective, Glynor I think your deep knowledge of MC meant you were thinking of the existing underlying behaviour of MC and terms used to describe it. Zybex is a programmer, so "Yeah, whatever works, I understand it".  ;)


The subtle difference that the Display Expression may be used when the field isn't actually displayed wouldn't mean much to most users.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #71 on: September 22, 2020, 05:29:28 pm »

Reading and writing tags from/to files needs a little thought, although there may not be an issue.

Consider the [Date] field in MC again. It is a numeric value, but it is written to the "DATE" tag in human-readable format. In fact, MC currently only writes the year to the "DATE" tag, and it isn't the [Year] field, which is a calculated field that formats the [Date] field. It derives from the [Date] field directly, or appears to.

The raw numeric data in the [Date] field is written to another file tag, the "JR_DATE" tag, but only if the [Date] contains a decimal number rather than an integer. i.e. If the [Date] field only contains a year value, then the "JR_DATE" tag isn't written.

On importing a file the human-readable format "DATE" tag is written to the [Date] field, but in numeric format. So while we agree that the Input Expression is bypassed, there is still processing done by existing hard coded rules.

Will this proposal affect the tag mapping in MC?


While I am thinking of the [Year] field, it is an example of some exceptions that occur in MC to the normal rules. It is a calculated field, and as we have said, when a user enters data into a calculated field it is normally ignored and discarded.

However, enter data into the [Year] field and it is validated and saved, and at the same time the [Date] field is updated with the new data. There is some special programming around those two fields. The expression used for [Year] cannot be edited. [Date] is flagged as a User Data field, and that cannot be change. [Year] cannot be flagged as User Data. [Name] is another field that is treated specially.

How will this interact with the new functionality?
What would happen if this functionality allows [Year] to be flagged as User Data, and an Input Expression added to it?
Will some fields be locked so this functionality can't be used with it?
What fields will be locked out of using this functionality?
Will locking out some fields reduce the value of the proposal?
Will this change, or require change, to the definition of USer Fields?
(As I mentioned previously, JRiver has been adding fields as User Fields so that the database didn't need to be changed, which would impact Library Backup compatibility. For example, [HDCD] is a User Field.)

Basically, there is some internal coding around field types which is not visible to the user. More than just Hidden fields.

So, when using the Output Expression, are we assuming that hard coded formating built into MC will still be applied, and that it will be applied after the Output Expression? Or before? If so, are there any other consequences, for any of this proposal?

Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #72 on: September 22, 2020, 06:31:56 pm »

I have expanded your table Zybex. I don't know if the additional information is correct, or agreed, but it is based on posts above. I may have missed something. Comment?

                                User Data selected                      User Data unselected   
                           [Field]            [Field,0]             [Field]            [Field,0]
Input Expression:          newValue           Field.Value           N/A                N/A
Output Expression:         Field.Value        Field.Value           <blank>            <blank>
Elsewhere:                 Field.Output()     Field.Value           Field.Output()     Field.Value*

- newValue is the user input being processed for storing
- Field.Value is what is currently stored (raw value)
- Field.Output() is the Output Expression (or Field.Value if no Output Expression is defined)


I have asterisk one cell, because currently "Calculated Data" does work differently when [Field Name,0] is used instead of [Field name]. Some formating is applied to [Field name], but is not applied to [Field Name,0]. For example, I have a numeric Calculated field that displays to six decimal places using [Field Name,0], but only to two decimal places using [Field Name].

This may be true for non-calculated fields as well.

Basically, the issue here is that MC currently does some output formatting, and the Output Expression also does output formatting, so it should be decided which happens first. Probably the same issue for Input Expressions. We should make some recommendation.
Raw Data -> existing MC formatting -> Output Expression
User Input -> Input Expression -> existing MC formatting


BTW, I can think of some mathematical and financial examples of where the Field.Output() value would want to be used in the Input Expression. i.e. Store currency value in US$, but enter and display currency value in AU$, and allow incremental user additions. So;
Input Expression would be: (([Field] + input value) * exchange rate).
Output Expression would be: ([Field] / exchange rate)
So the overall expression would be: ((([Field] / exchange rate) + input value) * exchange rate)

A bit contrived, and I don't know if there would be real example in MC data. Just throwing out ideas for consideration, so no roadblocks are left.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #73 on: September 22, 2020, 07:16:41 pm »

Your updated chart for Output Expression is definitely wrong. [Field] would not be blank in that case!

Didn’t read the rest yet.

Nevermind. I shouldn't have tried to read it on my phone.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #74 on: September 22, 2020, 07:20:07 pm »

Consider the [Date] field in MC again. It is a numeric value, but it is written to the "DATE" tag in human-readable format. In fact, MC currently only writes the year to the "DATE" tag, and it isn't the [Year] field, which is a calculated field that formats the [Date] field. It derives from the [Date] field directly, or appears to.

[Date] like many other built-in fields, has always had special casing, and would be unchanged.

(The other "dates" are internally just expressions that parse out just the Year, or Month, or Day, or whatever, for ease-of-use. But, in any case, any internal fields which are currently grayed out and special cases, would remain as they are.)

Also, it is worth mentioning that your updated chart above (while it appears to be correct), really makes the situation seem more complicated than it is. [Field Name] and [Field Name,0] are already things. They work the way they do now, except for the User Input stuff and the ability to access the User Data variable value.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #75 on: September 22, 2020, 07:25:28 pm »

I have asterisk one cell, because currently "Calculated Data" does work differently when [Field Name,0] is used instead of [Field name]. Some formating is applied to [Field name], but is not applied to [Field Name,0]. For example, I have a numeric Calculated field that displays to six decimal places using [Field Name,0], but only to two decimal places using [Field Name].

Correct. The [,0] square-bracket notation option is designed to provide raw values. [Date,0] shows you the real floating-point date value, for example. Which, makes sense when accessing the User Data variable value. That's why it works, because it already is a "show me the real, underlying value without the formatting" option. The Output Expression is just treated the same way as MC's own internal formatting that it does on fields in this way.

You would "lose" access to the ability to use [Calculated Field,0] to get raw, unformatted data from your expression (if User Data is enabled for that expression). That's easy to work around though, just append &datatype=[string] to your expression, removes the special formatting coming out of expressions too. Or, if you really can't (because it is a number) you can just make a second Read-Only Expression Field (current Calculated Field) and point it to the original value with &datatype=[string], and then use that one anywhere you need the raw, unformatted output from an Expression Field that also has User Data enabled.

Other than that, I don't believe that conflicts with anything, and that is pretty edge-case oriented. It wouldn't, in any case, impact any existing functionality because it would only impact Fields that have both Output Expression AND User Data enabled, which isn't currently possible.

So, really, it could be like this:

                                User Data selected                      User Data unselected   
                           [Field]            [Field,0]             [Field]           [Field,0]
Input Expression:          newValue           Field.Value           N/A               N/A
Output Expression:         Field.Value        Field.Value           Unchanged         Unchanged
Elsewhere:                 Field.Output()     Field.Value           Unchanged         Unchanged

And, hence, pretty pointless.

Remember, Output Expressions with User Data unselected ARE the current Calculated Data fields. Behavior there doesn't need to change.

I don't actually know what it spits out if you reference a Calculated Field in its own expression. It might spit out Expression Error? I don't know. I could test it, but I don't really care. What it should do there is keep doing whatever it is doing now.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #76 on: September 22, 2020, 08:35:31 pm »

It's never pointless to document how something works, or is expected to work, particularly when it isn't obvious, there are several options, and you are making a proposal to someone.

Earlier you were saying a couple of those conditions should output blanks, now you are saying unchanged. Okay, I was summarising the discussion, and asking questions. It would be nice if this worked first time it was implemented.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #77 on: September 22, 2020, 08:52:16 pm »

It's never pointless to document how something works, or is expected to work, particularly when it isn't obvious, there are several options, and you are making a proposal to someone.

Earlier you were saying a couple of those conditions should output blanks, now you are saying unchanged. Okay, I was summarising the discussion, and asking questions. It would be nice if this worked first time it was implemented.

Oh, I know. I was working it out in my head now too. I wasn't saying it was pointless that you brought it up! (Far from it.) I was saying, in the end, after realizing that all the things on the right are either (a) impossible with the new system or (b) already exist, that "thinking" about how to change them is pointless. The correct answer is unchanged for everything on the "right side" of the chart.

You can't access the Input Expression box if User Data is unchecked. So, there, the N/A is correct.
For the other ones, which are all currently possible, then you just leave the existing behavior intact.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #78 on: September 22, 2020, 09:01:26 pm »

And, I should add further, you brought up an important point I hadn't fully considered. If you make one of these new fields that contains both an Output Expression and a User Data variable, then reusing square-bracket [,0] mode to access the underlying value of the User Data variable, means:

1. For those "new types" of Fields, you wouldn't be able to access the formatted value of the "underlying" User Data variable. Because, I'm assuming throughout all of this, that [Field Name,0] would continue to output the unformatted data from the variable (as it does now for all standard User Data fields).

If you need it formatted, you'd have to wrap it in another Read-Only Expression Field, as discussed above.

2. Likewise, you wouldn't be able to access the unformatted value of the Expression Output itself, as discussed above. However, this doesn't matter because you control the expression itself, so if you don't like what it is outputting, then just change it to output it with better formatting.

Or, if you really need both, one of them is going to have to be wrapped in a separate Read-Only expression field with a different expression, and you won't be able to write to that one. But, again, that's no different than today.

But, all of it only applies to using those new Read/Write Expression Fields. And you control the expressions in this case, so you can do anything you want with them. So, really, hitting that should be very edge-case-y, don't you think?

EDIT: After re-reading this and thinking about it, it is worth leaving here but really... [Field Name,0] really should result in the unformatted data from the Field, and [Field Name] the formatted data. Your Output Expression is, in this case, the formatting.

If you're intentionally trying to get to that underlying value you'd expect it to be the unformatted version. And, more importantly, the unformatted version is "safer" to receive because you can always just re-do the formatting if it is stripped out. You can't always do the reverse easily (think of a Date value).
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #79 on: September 22, 2020, 09:25:35 pm »

I thought of something else critical, that is worth bringing up again. I mentioned it before but, there is still one last benefit to my original idea that this new one doesn't address.

For this idea to work to solve my original problem (and a few others to boot), you'd have to be able to enable the Output Expression and Input Expression options on the [Artist] field itself, and other stock fields like it.

The Artist field, like many others in MC, is now locked down, and can't be changed. For this idea to work, those fields can still keep Data Type, Relational, Edit Type, and Acceptable Values locked. But Input and Output Expression would have to be unlocked. If not, you'd be able to do it with all of your custom fields, but you couldn't use it to format [Artist] or [Cinematographer] automatically, which would defeat my entire point.

My original idea doesn't require JRiver to let you make any changes at all to the stock fields. A separate, custom field can write into a stock field, but it doesn't actually have to change. And my original idea does all of this other stuff too. So, if JRiver is not willing to allow that, can we agree that some version of my original request would be the best alternative?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #80 on: September 22, 2020, 10:01:45 pm »

Yep. That is why I was asking these and other questions.

Will some fields be locked so this functionality can't be used with it?
What fields will be locked out of using this functionality?
Will locking out some fields reduce the value of the proposal?

I was certainly thinking of the [Artist] field and Taylor Swift, even if I didn't include it.

That you would be able to use this functionality on stock fields was implicit to agreeing with this proposal. But even then, there would be some fields that it shouldn't be used with, even if it could be.

If we have to create a new field and can't write back to the original field with all the functionality built around it, the current proposal wouldn't meet your original request. It still might be neat for user-created fields, but less useful.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #81 on: September 23, 2020, 12:09:25 am »

Yep. That is why I was asking these and other questions.

I missed that when you said it earlier, but it is an important point. If we were given the option between:
  • My original Writable Calculated Fields idea, and...
  • The Unify Calculated and User Data Fields proposal outlined above, except that: you can only use it with user-created custom fields and not stock ones
Then I'd take Writable Calculated Fields in a heartbeat.

Because I can't just switch away from those stock fields: I want my files to actually be usable in other applications (the [Artist] tag needs to be set properly so that other applications can see it), and I want to apply it to all of the "personal name" auto-metadata filled fields (like Director, Cinematographer, Actors, etc). Those are my core "designs" and Writable Calculated Fields do not suffer from that restriction.

With that, I think we're back to two proposals:

1. Unify Calculated and User Data field types, as outlined above.
This has two potential issues:
   * It requires a little bit of syntactic sugar with the [Field Name,0] notation stuff, which has one edge case when using the new functions.
   * It requires that it be possible to apply this change (optionally) to [Artist], [Album], [Actors], [Director], [Composer], and similar built-in stock fields.

But, I do like how easy to understand this idea is, and it does simplify the overall quantity of fields you'd need to create to accomplish some tasks.

2. Writable Calculated Fields. If the second of the two points above fails to meet JRiver's requirements, then this would do all of the same things, and would avoid both of those issues.

It requires no special syntax other than a reference to the User's input in the Input Expression. And you can just use your Write-To User Data field directly when you need to access the raw value (so no mucking with [Field Name,0] required). The major downside is that it requires you to define two fields (a custom Calculated Field, which then writes to a standard User Data Field) instead of one.

Writable Calculated Fields Mock Library Field Manager Implementation:

_Name_ - Unchanged
_Data_
 [•] User Data - See Note 1. Radio button (like now)
     Data Type [__]
     Relational [__]
     [✓] Save in file tags (when possible)
 [•] Calculated Data - See Note 1. Radio button (like now)
    Output Expression - cannot be disabled.
        [expression editor] - See Note 2.
    [✓] Input Expression - when unchecked, disable indented below
        [expression editor] - See Note 3.
        Data Storage Field [___] - See Note 4.
 Edit Type [__] - Always enabled.
 Acceptable Values [__] - Always enabled if User Input is allowed.
_Search_
 [✓] Show a Search Link - Always enabled. - Same as above, but I moved it here this time.
 (Section otherwise unchanged.)


Notes:
1. User Data and Calculated Data: These work exactly as they do now, only the "always available" options are moved out of the "User Data grouping".
2. Output Expression: This is the exact same thing as the current Calculated Data Expression box, just relabeled Output Expression. Cannot be disabled.
3. Input Expression: This is identical to what was described above. Enter an expression here which is evaluated on any write to the Calculated Field, the results of which are saved to the Data Storage Field. Square-bracket notation [My Field Name] (or alternatively Input() or [Input]) expands to user's input data when the expression is evaluated.

By default when Input Expression is enabled, but left blank, the user-entered data is passed through to the Data Storage Field unmodified (alternatively, you could just pre-fill [My Field Name] in the expression box, which would also probably serve as a handy tutorial tool for users).

If Input Expression is unchecked, then Calculated Fields work exactly as they do now.*

4. Data Storage Field: This allows you to select a single User Data type field from a drop-down combobox. Other Calculated Fields cannot be selected (preventing loops). The selected field will be transparently used for storage any time data is saved to the Calculated Field, after the input is filtered through the Input Expression (if provided).

* But, one other little thing: If Input Expression is unchecked, please "Read-Only" the star-editors, checkboxes, and whatnot if Edit Type is set to Check, 5-Stars, 10-stars, etc (to be clear, do not disable these options in Edit Type, just make the checkboxes that show up in the Views read-only). I love that we can have a checkbox-style Calculated Field, but it stinks that you can click it and check the box when it is read-only.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #82 on: September 23, 2020, 12:18:25 am »

So, looking at the above, especially Zybex and RoderickGI and others, do those look like two proposals that would both work equally well if implemented, at JRiver's option?

I can re-write it up if we get general consensus.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #83 on: September 23, 2020, 02:20:51 am »

Quote
It requires a little bit of syntactic sugar with the [Field Name,0] notation stuff, which has one edge case when using the new functions.
Do you mean [FieldName]?
The small table above is enough - you might want to add simply that the return value for [FieldName] keeps all existing behavior, except when processing the Input and Output expressions (if defined). [FieldName,0] always returns the field's raw value in all contexts (or "circular reference" error, as it does now, when trying to use it on a pure calculated field to reference itself)

Agreed than locked fields such as [Artist] would need to be semi-unlocked to allow adding Input/Output expressions. Some other critical fields might have to remain locked to prevent users from messing them up.

Your summary of both ideas looks good.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #84 on: September 23, 2020, 02:45:42 am »

Time to throw a few more problems into the discussion  ;D

I've mentioned before that I have some reservations about the InputExpression, and I've tried to explain it on the previous page. Let's see if I can explain it better now.

Problem #1 - The Multi-Field Output:
If the OutputExpression for a user-data field references other fields than itself, there's no way to build a corresponding InputExpression that works seamlessly for the user.

Example 1 - normal case
Let's use the [Artist] field example:
- set Output="swap([Artist])"
- set Input="unswap([Artist])"

So "Gaylor Swift" displays as "Swift, Gaylor", and when the user edits it and changes to "Swift, Taylor" the value "Taylor Swift" will be written to the field. All good.

Example 2 - bad case
If the Output expression references other fields than itself, we have a problem. Let's use a contrived example; imagine you want to show the Artist and their country:
- set Output="swap([Artist]) ([Country])"
- set Input= ??

The Output expression will display like "Swift, Taylor (US)".
The problem here is that the InputExpression cannot write back to [Country], only to [Artist]. So you might define an Input expression that removes the (US) part and unswaps the name to get "Taylor Swift" back; however, any edit to the Country part is lost, which kind of breaks the magic:
- if the user edits "Swift, Gaylor (US)" and changes to "Swift, Taylor (US)", the Input Expression can store the correct artist name, it works fine;
- if the user edits "Swift, Taylor (AU)" and changes to "Swift, Taylor (US)" ... nothing happens, the field will still display "Swift, Taylor (AU)" as the [country] cannot be changed by the Input Expression of [Artist].

This behavior might be weird to the user, who will not easily understand why some edits work and others don't. WAF is the problem here!
Note that this problem exists in both proposals.

One solution is to prevent using other fields except itself on the OutputExpression of user-data fields. To do that the user would need to use regular Computed fields without Input expression (not editable, read only). Another, more radical, is to do away with the InputExpression completely and just show the actual/raw field value when editing starts (I know you dislike this).
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #85 on: September 23, 2020, 02:57:39 am »

Problem #2 - The Debug Method:
Another issue with InputExpression is ... the debugging of it until you get it right. Who here can get an Expression Language expression right on the first try?

While playing with the InputExpression to get the desired results, each time it runs it will overwrite the underlying field content. If the expression isn't right on the first try it will mess up the saved value and the user will have to manually fix it before trying again with a new/fixed expression.

This is annoying.

Worse, you might enter an InputExpression that appears to work just fine, but does not in fact cover all edge cases. Then you start using the field and in some cases it might be corrupting data when the user edits the field value. If the user doesn't notice this corruption immediately he might end up with lots of corrupt data that will need manual fixing, or even a backup restore. Even if the user notices it immediately he might become suspicious of the feature and will want to check all his data for possible corruption.

This is annoying and dangerous.

These are usability/reliability issues that make using this feature very cumbersome and even dangerous if you're not an expert.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #86 on: September 23, 2020, 03:32:08 am »

Problem #3 - The Input Bypass:
Lastly, another usability problem: on a field with an InputExpression defined, how can the user edit the raw value directly?

By definition, any edit to the field would go through the InputExpression. But if the user wants to enter some raw data (either individually or via a multiple-file/field Paste), he needs an easy way to bypass the InputExpression. This is not so much an issue with the LinkedField idea as the user can just go and edit the back-end field directly, but it would be nice to be able to do so directly on the new editable field which holds the InputExpression.

Suggestion:
- When editing a field, press ENTER or click somewhere else to finish the edit (as usual) -> InputExpression is triggered
- Press CTRL+ENTER to finish the edit and write the value directly without processing -> InputExpression is bypassed
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Feature Request: Writing to Expression Fields
« Reply #87 on: September 23, 2020, 06:35:39 am »

Yep, the proposal summaries look fine for both.

Problem #1, Example 2: I can't think of any practical way to solve that. I can think of at least one fancy programmatic way to update just the main field (re-read the field, parse the Output Expression to only include the functions affecting the main field, the Input Expression only handles that main field), but I don't think that would fly.

Problem #2: Yes, it could be a can of worms. Library Backups, regularly. Maybe users only get access to this functionality with special licence conditions, for expert users. Sorry, no better ideas just now.

Problem #3: Your suggestions sound fine.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #88 on: September 23, 2020, 09:22:34 am »

The problem here is that the InputExpression cannot write back to [Country], only to [Artist]. So you might define an Input expression that removes the (US) part and unswaps the name to get "Taylor Swift" back; however, any edit to the Country part is lost, which kind of breaks the magic

Correct, and yes, if you merge multiple fields into a single output, you'd have to pick only one value that can be written. My [Watched] expression above is a similar example. That would apply to the Unified User Data and Calculated Fields example too, as you noted.

Again, not really an "issue" because you control the expressions. I think it is fine. Just design your system well.

The perfect does not have to be the enemy of the good. There are lots of things like this in MC's expression design.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #89 on: September 23, 2020, 09:23:42 am »

Problem #2 - The Debug Method:
Another issue with InputExpression is ... the debugging of it until you get it right. Who here can get an Expression Language expression right on the first try?

This would be equally true of both systems, and of all data entry done with =Expression. I think that was your point, but I don't see how it is any different than stuff we've had for years.

Yes, if you're writing to field values with an expression, you should be careful. I wouldn't mass tag my entire Library with anything until it was thoroughly tested.

But that also applies to mass tagging them with the =Expression editing method. So... As Roderick said, make Library Backups (which it does automatically now, thank god) and test.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #90 on: September 23, 2020, 09:27:34 am »

Problem #3 - The Input Bypass:
Lastly, another usability problem: on a field with an InputExpression defined, how can the user edit the raw value directly?

Unnecessary. And that's why that idea still has merit (I think you missed this important point). You just edit the Data Storage Field directly. The real field is still there and is still accessible like it always was. While the Calculated Field can write to the Data Storage Field, this does not block you from writing/reading to/from the Data Storage Field yourself directly.

So, for the Artist Name Swapping example, while the system does not require you to display the "real" [Artist] field in order to edit it anywhere, the real [Artist] field is still sitting there and can be used like it always was, directly, whenever you want.

That is the one advantage the Writable Calculated Fields example has over the Unified approach. It doesn't touch the original fields. They're "regular" like they always were, and so no special syntax is required. Want the formatted value? Use [Artist Display]. Want the original unformatted value? Use [Artist]. You can write to either one, in the format that seems appropriate and then both change.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #91 on: September 23, 2020, 09:33:30 am »

Quote
The real field is still there and is still accessible like it always was.

I'm raising issues for both proposals - #1 and #2 are common to both, #3 is exclusive to the unified approach.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #92 on: September 23, 2020, 09:35:17 am »

I'm raising issues for both proposals - #1 and #2 are common to both, #3 is exclusive to the unified approach.

Yep. I got it, I was just making it clear for other readers. But #3 isn't an issue. Or I don't understand what you mean.

You don't need any special syntax or controls to bypass the Input Expression, because if you want to bypass the Input Expression, you can open up the Tag AW and edit the "original" field.

You can't do it through the Calculated Field. But that's... By design. It does what it does, and if you don't want it to do what it does, then don't use it.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #93 on: September 23, 2020, 09:38:00 am »

I mean that on the unified approach, if you set up a field with an Input Expression you also need a way to write a raw value directly to the field, without any massaging by the Input Expression. I was suggesting CTRL+ENTER.

There is no "original field" in this case. It doesn't link to any other field.

Edit: Are you saying that the Tag panel should display the raw value instead of the OutputExpression() value? I fully disagree there, if there's an Output+Input expression pair defined, then the Output-processed value is the one to be displayed pretty much everywhere. Hence the need for a workaround.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #94 on: September 23, 2020, 09:45:24 am »

I mean that on the unified approach, if you set up a field with an Input Expression you also need a way to write a raw value directly to the field, without any massaging by the Input Expression. I was suggesting CTRL+ENTER.

There is no "original field" in this case. It doesn't link to any other field.

OOOOOOH. Sokath, his eyes uncovered. #3 is specific to the Unified Approach. Which you wrote, I think twice, and I read it backwards both times.

You're right.

So, that's another big knock on the Unified approach that I'd forgotten in my excitement over [Field Name,0] yesterday. In addition to the requirement for the [Field Name,0] syntactic sugar to access the unmodified field for reading, you need something to let you access the un-modified field for writing.

Makes me think Writable Calculated Fields is overall better again:
  • Less syntactic sugar required.
  • Doesn't require JRiver to unlock stock fields.
  • Provides easy-to-use access to the underlying storage fields directly.
I think your proposal for the solution is fine, but... It feels like it needs a lot of special casing, which is (1) harder to use and (2) more work for JRiver to implement, and so less likely we'll get anything.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #95 on: September 23, 2020, 10:02:40 am »

Both of them are finicky solutions, frankly, likely requiring more code changes than we can perceive. They will have limited audience, not many people would play with this feature. I think the chances of it getting implemented are very slim. Remember that the only added gain relative to what MC already does, is the ability to edit computed fields; but... one can already edit the base fields directly, so in the end it sounds like a lot or work just to create an editing shortcut.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #96 on: September 23, 2020, 10:07:46 am »

Out of the blue, here's an alternate idea to allow editing of Computed fields (let's call it "Smart Tag Editor" for reference):
  • keep field definition as-is, no change required at all
  • when user edits a computed field, MC looks at the expression for that field, extracts the names of all Fields used in the expression (following expression chains if needed), and pops up a mini-tag editor JUST with the fields that make up that Computed field.
  • the mini-editor has a cell for each field, and may show underneath what the Computed expression will output in real-time as the edits are being made.
  • Optionally, a new checkbox could be added the the Computed field definition to enable it: "[_] enable editing of related fields"

That's it. They could probably reuse the existing Tag AW for this.


Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #97 on: September 23, 2020, 10:35:23 am »

A decent idea, but:
(1) That wouldn't help with tagging in the Pane Checkboxes (or it would be very clumsy). Enabling that is a core design-focus of my original idea. For me, and my (now 15-year long) workflow, it wouldn't help at all.
(2) That sounds like more implementation work than my Writable Calculated Fields or the Unified approaches (we can't know for sure, but it would be inventing a whole new GUI editing function of some kind, if possibly small-scale).
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2619
Re: Feature Request: Writing to Expression Fields
« Reply #98 on: September 23, 2020, 10:47:14 am »

A decent idea, but:
(1) That wouldn't help with tagging in the Pane Checkboxes (or it would be very clumsy). Enabling that is a core design-focus of my original idea. For me, and my (now 15-year long) workflow, it wouldn't help at all.
(2) That sounds like more implementation work than my Writable Calculated Fields or the Unified approaches (we can't know for sure, but it would be inventing a whole new GUI editing function of some kind, if possibly small-scale).

Creating a new window/panel is a couple hours work... contrary to the other 2 ideas, this would be a self-contained feature, wouldn't impact anything else. And it could reuse the existing TagAW code/look.

Pane checkboxes, are you referring to this example you gave? If so, it would work the same... you can have the [artist is a person] field/column, and the [Artist-Display] would use it to compute the displayed name. When editing the [Artist-Display] field, one of the related fields that would show up is the [artist is a person]. You could even select multiple files and popup the Smart editor for the [Artist-Display] - it would show <varies> where needed.

Not sure I understand correctly though.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Feature Request: Writing to Expression Fields
« Reply #99 on: September 23, 2020, 04:46:14 pm »

EDIT: Read my next post before you read this one.

Now I'm confused. I thought you were talking about that as an add-on for the Unified Approach. And you wouldn't be able to use the Pane Checkboxes to tag those underlying fields.

Eg: If it was based on writing to [Field Name,0] which is what I was thinking initially, then you could make a "plain [Artist]" pane, if you wanted one. But you can't, because then that expression wouldn't be writable. And you couldn't point more than one expression pane at the same field and have them both be writable.

Are you instead talking about an extension to the Writable Calculated Fields idea? Because, again, you don't need anything fancy to write to the underlying fields with that idea. If you want to write to the underlying fields directly (and bypass the Input Expression) you can: Just add [Artist] to your Tag AW. In fact, it is already there. Just don't hide it.

My concern over the implementation time is about:
Out of the blue, here's an alternate idea to allow editing of Computed fields (let's call it "Smart Tag Editor" for reference):
MC looks at the expression for that field, extracts the names of all Fields used in the expression (following expression chains if needed), and pops up a mini-tag editor JUST with the fields that make up that Computed field.

That seems terrifying. It might not be easy to follow some of my expression chains, filtering in and out all of the fields. Here's one of them I use as a Pane in all of my music views:
Code: [Select]
Regex(NoArticles(ListItem([Artist],0)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4))If(IsEmpty(ListItem([Artist],1)),,;Regex(NoArticles(ListItem([Artist],1)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],2)),,;Regex(NoArticles(ListItem([Artist],2)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],3)),,;Regex(NoArticles(ListItem([Artist],3)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],4)),,;Regex(NoArticles(ListItem([Artist],4)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],5)),,;Regex(NoArticles(ListItem([Artist],5)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],6)),,;Regex(NoArticles(ListItem([Artist],6)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],7)),,;Regex(NoArticles(ListItem([Artist],7)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],8)),,;Regex(NoArticles(ListItem([Artist],8)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))If(IsEmpty(ListItem([Artist],9)),,;Regex(NoArticles(ListItem([Artist],9)), /#([0-9]{0,2})([a-zA-Z]?)([a-zA-Z]?)#/,-1)IfElse(And(IsEmpty([R1]),IsEmpty([R2])),/(other/),!IsEmpty([R1]),0-9\[R1],1,[R2]\FixCase([R2],3)FixCase([R3],4)))&DataType=[list]
Now, that particular one would stay read-only and wouldn't need the editor at all. But, it has Regex replacements (which look like square bracket notation), and has to operate on multiple ListItems. Possible certainly. That one wouldn't really be that hard (the answer is only [Artist]). But a couple hours to make and validate all of that across the entire scope of the Expression Language?

I never like to assign a time estimate to code I can't see. It is hard enough (often nearly impossible) estimating timeframes on code you CAN see. But that still seems like more work than my original idea. Which is self contained.

But, that's not for us to decide. I do just want to keep the proposal as simple and as "bare necessities" oriented as is necessary. And that seems like a bigger ask than a rearrangement of the Library Field Manager and adding in Input Expressions (which is, as discussed, effectively just adding a Setter to an existing system that already works in the proper way).

I could be wrong, of course. But, I don't see how it is that complicated.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/
Pages: 1 [2] 3   Go Up