Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[glynor]

So, with the arrival of MC27 comes my annual considerations about how to handle proper names in MC (especially [Artist] but also [Director], [Composer], [Actors], etc).

* What I'm doing now: Manually "swapping" Firstname Lastname into Lastname, Firstname for many of these fields. For others, like [Director] and [Actors] which are generally auto-filled by metadata lookup, I just suffer with the list being stupid-sorted and generally useless.
* The Dream: Have this happen automatically, so that the tag is stored Firstname Lastname, but (when the value contains a proper name) the swapped version is used and displayed everywhere in MC, especially for sorting.

With the new stuff in the expression language, I'm so very, very close to being able to implement the latter in an extremely clean an efficient manner. But there is one major stumbling block I always hit: Expression Fields are read-only.

So, if I do this, I have to display the original, non-swapped versions in all of my Panes, Columns, and Categories otherwise I can no longer use them to edit these fields. I could do this and sort by the swapped versions, of course, but then the display of my [Artist] and other fields in Panes would look all crazy, and it is difficult in some cases to get MC to sort categories by another field. That's non-ideal, and hence I've never actually implemented The Dream.

To fix this, What I'd Love To Have:

When defining an Expression Field in MC, if you could have two new "boxes" in the UI:
1. Write-To Field: Combobox allowing you to select any other writable field in MC. If blank (the default) then the field is read-only, like now.
2. Expression to Apply to Input: An "expression editor" textbox (disabled when Write-To Field is blank). This would allow you to enter an expression that will be applied to input before the results are written out. For this to work, I think it would need a special field of some kind that would represent the "input" (like [Input] or something similar) so that it could then be used in Expressions.

The idea of #2 is to allow pre-formatting of the data entered to make it match the expected output field format. In other words, since I'm going to be displaying Lastname, Firstname for many of these, I could put something like this in the Expression to Apply box: If([Artist is Proper Name],Unswap([Input]),[Input]) and then the user could enter "Swift, Taylor" (or, more likely, check one of the boxes in the Panes) and it would automatically become "Taylor Swift" when it is written out to [Artist].

This system, or something similar, would be extremely useful in a wide variety of locations throughout MC. There are many places where I use Expressions to "massage" display of field data, where I have to also display the "real field" because I'm otherwise unable to modify that field. Or, worse, where I want to but don't use an Expression to massage the output, because that'll block me from using the thing to enter data, which seems to crimp the power available in the Expression Language and formatting in MC.

I can think of a wide variety of uses for this way-beyond displaying reversed names:
* It would fully unlock the ability to pre-filter and/or format data for display (including fonts, colors, etc) without blocking your ability to edit the underlying data.
* It would enable fancy "data validation" by using the Expression to Apply to block invalid edits to fields, if you got really into it.
* It would make "non-blank" defaults for fields viable (without having to rely on Tag on Import seeing each and every file as it comes into the database).

Thoughts? Would this be a nightmare to implement?

[zybex]

The main problem I see with this idea is the danger of creating loops: [FieldA] writing to [FieldB] using an expression that uses the current value of [FieldB]; a change in B would cause A to be recalculated, writing a new value to B, which would trigger A reevaluation again, and so on. Stack overflow, unless MC did all kinds of checks to prevent loops (that's your nightmare scenario because the user could create a loop using any number of fields, A->B->C->D->A ...)

There's a simpler solution, I think:
- for any regular field [Field], add an optional "Display Expression" box (default implicit value is "[Field]")
- when displaying the field value, show the output of the expression
- when editing the field, show/edit the actual written value

Simpler, no chance of loops, no need to point field A to B.

[glynor]

I've thought about that over the years, and yes, that's a valid alternative. That would block some of the utility of the tool though, which is why I didn't suggest it.

That would solve the "display something different than what is written" issue. But that's pretty much the only thing it would solve, and it would be extremely troublesome in a variety of places because you wouldn't have the Expression to Apply Before Writing function.

A simple example: How do I check the checkbox in the panes that says "Swift, Taylor" and have it automatically tag it properly as "Taylor Swift"? That's my main goal with the Name Reversing stuff. I use the checkboxes to edit my tags extensively (never type something when you can pick it from a list, because that's how typos happen). Even ignoring that, it would be counter-intuitive for an input field to "change" to something (possibly DRAMATICALLY different) when you go to enter data there, and would make usability difficult. What about when you need to enter completely new data? How would the user know what format to use to enter it? You'd just have to "know" not to type it in matching all the other stuff in the field.

Being able to write out to a different field (with the ability to pre-apply an expression on input) could solve that, and opens up all sorts of other possibilities not enabled by the simpler "massage the display only" method. I listed some in my original post (data validation would be sweet) but here's another: You could auto-set other fields based on input with that expression, by pointing to chains of Expression Fields. This is a dumb, contrived example, but imagine if you could auto-set an [Artist Rating] (star field) down by 1 when you add the 3rd 1-star rated track, or up a notch when you add the 3rd 5-star track? You could do that by chaining input and output to various "display fields".

You're right, of course, that the users could get themselves into trouble if they write to fields which cause a loop. But, honestly, there are tons of places in the Expression Language where you can get yourself into trouble. With Great Power Comes Great Responsibility and all that. I don't think that's necessarily a reason not to implement it in this instance, as you'd have to do some serious mucking about to get yourself into that situation.

[zybex]

A simple example: How do I check the checkbox in the panes that says "Swift, Taylor" and have it automatically tag it properly as "Taylor Swift"?

The field actual value would be "Taylor Swift", so it would tag like that. It would be "Swift, Taylor" just when printing the field.

it would be counter-intuitive for an input field to "change" to something (possibly DRAMATICALLY different) when you go to enter data there, and would make usability difficult.

Those are the breaks. I think it much more awkward to have to remember that to change field X, you actually need to edit field Y. And you would still need to have both on screen - the one you want to see, and the one you need to edit.

You could auto-set other fields based on input with that expression, by pointing to chains of Expression Fields.

Update chains are a nightmare, that's not likely to ever be implemented.

With Great Power Comes Great Responsibility and all that.

+1

[glynor]

You could also "solve" the looping writes situation in other ways (which might be good even if the user didn't create an infinite loop, just for optimization purposes): Only allow a field to be written-to once by a single input "event".

In other words, when [Artist] gets written in the code, you flip a boolean. If the same input event tries to come through and write to the same field again, the execution stops and the write gets discarded. This would "short circuit" any foolish write-loops simply without the hard-lock.

[glynor]

The field actual value would be "Taylor Swift", so it would tag like that. It would be "Swift, Taylor" just when printing the field.

Right, but how does the user know this is what will happen? For reversed names, this is pretty simple and obvious when you're applying an existing tag to a different file.

But what does my wife type into the box when she selects the New Artist checkbox, and how does she know what to put there? If she goes by the values of the other items shown in the list, she'd probably manually type "Swift, Taylor" and then it'd end up displaying as Taylor, Swift or some kind of nonsense and I'd get yelled at.

I don't like getting yelled at. 

Also, that assumes the modification to the field is pretty simple in order for it to be intuitive. Many of my "display fields" are compound expressions that combine the values of multiple fields (displaying things different ways depending on the "state" of the file in question). In most cases, these would still make the most sense to keep read-only, but there are other circumstances. Maybe you add a "star" character to favorite artists so that anywhere they display, they show a pretty star. But if the user manually enters the star, you could have your Expression to Apply "filter that out" (or maybe even, if you got super-fancy, set the Favorite Artist status too).

[zybex]

WAF is always a consideration   

Anyway, it's also tricky to have field X being changed out of the blue without her even touching it. That would have her mumbling around in shock and you would be sleeping on the couch until you fixed that nonsense

[glynor]

Those are the breaks. I think it much more awkward to have to remember that to change field X, you actually need to edit field Y. And you would still need to have both on screen - the one you want to see, and the one you need to edit.

I think you misunderstood my original request. That's exactly what I'm trying to avoid. Sure, behind the scenes it would be writing to [Artist] when you edit my [Artist (Display)] expression-field, but for everything you see, it would just look like she was editing a field directly. You wouldn't need to display the real [Artist] field anywhere. I would, in some limited places where it would look nicer, like the Playback Bar and whatnot, but you wouldn't have to show it anywhere.

You'd just edit [Artist (Display)] directly in place, and input your data just like all of the other entries in that field look on screen, and the Expression to Apply would "fix" it before it gets written out to [Artist] behind the scenes.

Update chains are a nightmare, that's not likely to ever be implemented.

If they did what I suggested, you could implement them. Just point your Field to Write to another expression-field, which itself writes elsewhere.

I'm not suggesting you'd do this everywhere, obviously, or go crazy with it, but it could be handy in some limited circumstances to do clever things.

In any case, either option would be better than what we have now. I'm not 100% sure I'd use the Display Expression method to actually solve my Reversed Artist Names issue (though I'd sure-as-heck try it out). If they are able to implement something more like my suggestion, though, I think it could open up possibilities to fix issues we aren't even considering now though!

[RoderickGI]

You are entering into the realm of real relational databases which use input scripts and field validation, built into the database itself and not in application code. While MC's "Manage Library fields" functionality provides a little of that capability, a glimpse, the database MC uses isn't relational and doesn't have that capability. That is all written in MC code, not in the database itself. Adding it would be a large project for a flat-file system such as the MC library.

This is better...

There's a simpler solution, I think:
- for any regular field [Field], add an optional "Display Expression" box (default implicit value is "[Field]")
- when displaying the field value, show the output of the expression
- when editing the field, show/edit the actual written value

Simpler, no chance of loops, no need to point field A to B.

... with the addition of an Input Template, which provides a little of the form-based data entry some databases use. The template would simply be text defined against a field in "Manage Library fields".

So the Display Expression is used when the field is just being displayed. i.e. Swift, Taylor
When the field is edited the real underlying data would be shown. i.e. Taylor Swift
When an empty field was edited the input template would be displayed as greyed out text which would disappear when any content was entered into the field. i.e. Firstname Lastname

The Google search entry field is a simple example. The field says "Search Google or type a URL" until you click in it. First image. A better example would be where the text didn't appear until an empty field was clicked on (edited), and remained until some data was entered. There are plenty of those in web form data entry, which is often driven by a database.

Or the Input Template from the fields "Manage Library fields" entry could just be prefilled in the field and highlighted, so any data entry overwrites the content. But there is a danger then of the field being filled with the Input Template text for lots of records. The "Add New Field" function in "Manage Library fields" is an example. Second image.

[zybex]

You'd just edit [Artist (Display)] directly in place, and input your data just like all of the other entries in that field look on screen, and the Expression to Apply would "fix" it before it gets written out to [Artist] behind the scenes.

This would require the user to enter 2 expressions on a calculated field A:
- the expression that yields the field's value, usually dependent on the value of B
- the expression that is used to process any input and then write it to field B

These two expressions would need to precisely complement each other so that the system worked seamlessly - and it would be up to the user's proficiency in Expression Language to achieve that. Plus, it wouldn't really work with expressions using more than one field; if the field's output expression is calculated using 3 other fields, but the input expression can only write to one of them... editing the field would immediately break the expected output.

Example:
- Field A output = "[Artist] ([Band])"
- Field A writes to [Artist] (because you would have to choose one linked field)
- how would the Input/validation expression be defined? Editing the field would never be seamless, since any change to the second part (Band) would be lost; the edited value would be written to [Artist] only, potentially messing up the output expression value.

[glynor]

I don't understand what you mean at all. Perhaps you're missing my point entirely. You'd define the expression once per field in the Library Fields setup part of Settings. I'd do that when setting up the Expression Field. The "users" would never have to.

The field example I gave [Artist (Display)] was an entire field name, and was just a placeholder (though that's probably what I'd call mine). It could have just as easily been [My Expression Field]. It doesn't matter.

When you edit the field in the Tag Action Window, in-line in the list of files, or in the Panes, you'd just edit them like any other field. Then what you enter would be filtered against the expression defined in settings for that field, and then the result saved to the designated output field.

Which is also, incidentally, why what I'm suggesting does not require any fancy relational database features (though with clever use of Expressions, you could hack some on, like you can now with Expressions). It would just be: (1) calculating expression results (which MC is VERY good at, and does VERY fast) and then (2) saving it to the designated destination field (which is no different from a UI element saving directly to the [Artist] field (or any other "normal" field). What would be actually saved to the output field would just be text, like any other field data.

[zybex]

By users I mean you and me, whoever defines the fields.

Maybe I'm misunderstanding. I'm assuming the field [Artist (display)] would be defined as an expression field, with expression = swap([artist]), and then, on top of that, we would link it to [artist] as the write-to field, with a second expression to pre-process the input. Is this correct?

If it is, then my reasoning above applies.

If you mean the field would not be an expression field, but instead a regular field with only that write-to link+expression, then It would mean you could have totally unrelated values in [artist] and [artist (display)], by simply editing the [artist] directly. I don't think this is desired. However, this is what is commonly called a "trigger" in relational DBs, and it can also be useful.

[mattkhan]

You are entering into the realm of real relational databases which use input scripts and field validation, built into the database itself and not in application code. While MC's "Manage Library fields" functionality provides a little of that capability, a glimpse, the database MC uses isn't relational and doesn't have that capability. That is all written in MC code, not in the database itself. Adding it would be a large project for a flat-file system such as the MC library.

I don't think this has anything to do with the underlying storage mechanism and a relational database doesn't have input scripts or validation, it's just a storage engine that may have some programming API built on top.
 
whether the way MC is implemented makes this easy or not is another Q but it's just a question of constructing a graph of nodes which can operate on values as data passes through it, validate that the resulting graph is not cyclic and you don't have a problem with loops

I'd have thought that usability would be quite bad though without a way to understanding where such graphs exist in your setup & understanding what is going on could be quite challenging for non trivial cases. I mean MC doesn't even have any sort of syntax highlighting, code formatting or autocomplete or any way to publish/share/import fragments that use expressions so adding yet more functionality to the underlying instead of making what is there more accessible seems like the wrong path to me (though more and more excel style expression functionality without any work on other APIs seems like the wrong path full stop to me).

[zybex]

@Jim/Matt,
Please check if this feature request is doable/desirable.

[just bumping the thread as I'm aware that it's annoying to the OP when other users derail a feature request with 3rd party discussion ]

[RoderickGI]

I don't think this has anything to do with the underlying storage mechanism and a relational database doesn't have input scripts or validation

Oracle and Progess have functionality built into the database as part of its definition, and are not part of the application code.

Anyway, noting Zybex's point, I made a suggestion I called an Input Template which seemed to cover Glynor's concern about users not knowing what format to enter data into an empty field. Along with the suggestion of a Display Expression added to Manage Library Fields, as per Zybex's Reply #1, I think there would be a workable solution, without needing to write to a second field, which I think would be problematic from a usability perspective.

[glynor]

Well, it seems like all of us are in agreement that the issue does need to be solved in some way. The long and short of it is that the Expression Language provides all the tools needed to manipulate and display data from the Library, but doesn't yet provide a way to cleanly use Expressions while preserving edit-ability of the underlying fields.

To make it simple, I thought I'd briefly summarize the two proposals outlined in this thread:

Display Expression and Input Template
In Tools > Options > Library & Folders > Manage Library Fields, for all standard User Data type fields, provide two new settings:

• Display Expression: Shows a Expression-editing textbox. When blank (the default) the value of the field is displayed, exactly as they work now. When filled with an expression, this value is substituted anywhere the values of the field are displayed (panes, grid-view, Tag AW, etc).
  
• Input Template: Allows the user to define a tooltip-style "informational" display when the users are editing this field to describe the required input format.

Note: The purpose of the Input Template is to prevent the situation where the user enters the data as it appears for all of the other fields, and receives unexpected results due to the Display Expression transformation.

Writable Calculated Fields
In Tools > Options > Library & Folders > Manage Library Fields, for all standard Calculated Data type fields, provide two new settings:

• Write-To Field: Combobox allowing you to select any other User Data type field in MC. If blank (the default) then the field is read-only, like now. If filled, when the user edits the value of this field, the input is saved to the designated field exactly as it would be for any other standard User Data field.
  
• Expression to Apply to Input: Shows a Expression-editing textbox (option disabled if Write-To Field is blank). If filled, this expression is applied to any input before that input is saved out to the designated Write-To field. This effectively does the "reverse" of the Input Template option described above. It allows the user to enter data exactly as the other values of the field are shown, but transform the input automatically to the proper "storage format".

Note: The restriction of the Write-To Field combobox to only User Data fields will prevent users from inadvertently creating a loop. Alternatively, (if this is easy with the way MC is currently coded) you could lift that restriction but allow a single "edit operation" to write to a field only once, thereby short-circuiting any loops the user does create. But that's probably dumb.

While (given only these two options) I'd slightly prefer the Writable Calculated Fields version, that's really about minor semantic quibbles. Any solution would be greatly appreciated, and either of these choices would meet my needs (and, I believe the needs of the others who commented here). They're really just two different methods to accomplish the same thing, and JRiver can certainly decide how best to implement it based on the underlying architecture of MC.

[glynor]

It is worth noting that I thought of one additional simplification to the Expression to Apply to Input option since I wrote my original responses: I'd originally thought you'd need to define a new, special field for the expression engine to "capture" the user's input (and so allow it to be transformed before output to the designated field). I proposed [Input].

But that isn't needed. You can simply use the calculated field's name for this purpose. So, that gives us these usage examples to solve the "Artist Name-Swapping Issue":

Writable Calculated Fields Version

• Field: Artist Display (custom Calculated Field)
  
• Expression: If([Artist is Personal Name],Swap([Artist]),[Artist])
  
• Write-To Field: Artist
  
• Expression to Apply to Input: If([Artist is Personal Name],Unswap([Artist Display]),[Artist Display])

Display Expression and Input Template Version

• Field: Artist (standard User-Data field)
  
• Display Expression: If([Artist is Personal Name],Swap([Artist]),[Artist])
  
• Input Template: I'm not positive what the markup for this would look like, but you could use formatting similar or identical to that used for the Track Info tooltips.

For both versions, [Artist is Personal Name] would be an integer-type field set to Relational: Artist with an Edit Type of Checkbox. You'd check the box if the Artist in question is Taylor Swift and leave it unchecked for "band names" like Pink Floyd.

[glynor]

I should note as well, I may have misunderstood RoderickGI's idea for the Input Template function. It is possible that, alternatively, this could operate exactly as my Expression to Apply to Input proposal: where you simply define an expression that is automatically applied to any input entered into the field, rather than explaining to the user what they should enter in the field.

The only downside to that is that you'd lose access to any method to write to the User Data field "directly" (without having this transformation applied). But, I'd live with that.

RoderickGI, if you could clarify that would be great, and if I misunderstood, I'll edit the posts above to reflect it.

[RoderickGI]

You understood my suggestion pretty well, only I wouldn't want the Input Template to be a Tooltip, as they take a moment to display, and they don't show in-line, inside the field.

• I would want the Input Template to show when a field was edited.
• The text would be greyed out to indicate that it wasn't editable, as is common practice for forms entry fields.
• When the field that was opened for editing, using F2 or double-clicking, the cursor would be positioned at the beginning of the field. Again, to show that the text is not content, but a template.
• When the user started typing content into the field, the Input Template text would disappear.
• The content would just be text defined in the field definition dialogue, in Manage Library Fields.

My concern with your Writable Calculated Fields solution is that you are using two fields, one for data entry and one for writing the value to. This would cause me issues with Views, as you would need to have one View for data entry and one for day to day use, or you would need both fields in any View you used. I may not be understanding your solution correctly though. But I think I am. Perhaps if you outlined how the functionality would work in Views. My solution uses just one field, which means all data entry and editing is done in the one View, with the one field.

I can think of one potential big advantage with your solution though. If you expanded the concept of the [Artist is Personal Name] to cover the instance of where Artists have three or more names, that could be very useful. So for example, with different Artist names;

For "Taylor Anne Swift" the [Artist is Personal Name] field would be set to 2 (for a zero based field) or 3 for human readability, to indicate that the last name (surname) is Swift, so the swapped version should be "Swift, Taylor Anne".

For "Taylor Swift" the [Artist is Personal Name] field would be set to 1 (for a zero based field) or 2 for human readability, to indicate that the last name (surname) is Swift, so the swapped version should be "Swift, Taylor".

For "Pink Floyd" the [Artist is Personal Name] field would be set to 0, to indicate that the Artist field should not be swapped.


But I would still be concerned with using two fields, and how that would work in Views.

If this could all be done in one field, without the use of a Write-To field, I would be happy for the solution to be implemented. Just re-reading your last post, it seems that still needs an [Artist Display] field in addition to an [Artist] field.

[glynor]

My concern with your Writable Calculated Fields solution is that you are using two fields, one for data entry and one for writing the value to. This would cause me issues with Views, as you would need to have one View for data entry and one for day to day use, or you would need both fields in any View you used. I may not be understanding your solution correctly though.

You are absolutely NOT understanding my idea. I think because you're making it more complicated in your mind that necessary. But, to be clear, you would not have to show "both fields" anywhere in MC. You could use exclusively the Calculated Fields in all of your Views. That's the whole idea.

The Calculated Field would work exactly like they do now, except you'd be able to edit values and save to them. If you do save to them, it would save to the field specified as the Write To Field. Like a "write redirect". You could have it save-to a field that isn't shown anywhere in MC at all, nor used for anything at all, and is only defined in Library Fields as a User-Data type field. If you leave the Expression to Apply to Input option empty, then that is absolutely it. You edit your custom-made [Artist Display] Calculated field, and whatever you put in the edit box (in the Tag AW, Panes, inline in the grid, via MCWS, whatever) would save to the field you specify as the Write-To Field.

It might make it easier to forget about all of the Name Swapping stuff, which is making it seem more complicated than it is, and think of a simplified example.

Dumb [Comment Light] Example
Say you have an Expression field that is super dumb, like an indicator light, that tells you if [Comment] contains anything or not. You can do this now, it doesn't require any additions to MC.

Make a Calculated Field in MC called Comment Light and put this expression in it: If(IsEmpty([Comment]),Empty,Not Empty)

Then, if you display this new [Comment Light] field in a View, it would show "Not Empty" for any file that has something in [Comment], and "Empty" for any file that does. Dumb, but simple.


The problem is, when you edit the text in the [Comment Light] field for a file, it doesn't do anything. What you type just gets discarded (and you really should have changed that Edit Type to Cannot be written).

Allowing Edits to [Comment] Through the Dumb [Comment Light] Expression Field
Now say you want to be able to edit that [Comment Light] field and type some crap in there, and have whatever you type in the box go to the [Comment] field. Here you would need the new stuff, but it would be easy. You'd just set the Write-To Field to Comment.

Now, when you have a file that is showing "Empty" in your fancy new [Comment Light] field in a file list view, you could "edit" it and type in "purple" and hit return.


Once you hit return, the text you typed would be saved to [Comment] instead of being thrown away (as it is now). And the value shown onscreen in [Comment Light] would change to "Not Empty" because now [Comment] is no longer Empty. You would not need to have the [Comment] field visible in any way. But, if you opened the Tag Action Window and looked at [Comment] it would contain "purple".

If you're not actively editing the thing, the new functionality does nothing. Everything works exactly like it does now. All of the magic happens when you edit the Calculated Field and then hit Enter.

Controlling What Gets Written Through the Dumb [Comment Light] Expression Field
But, what if you don't want to just write out exactly what the user enters in the edit-box? What if you want to change what they entered into something else? That's what the Expression to Apply to Input box is for. Enter an expression there and then that expression would be evaluated whenever someone makes an edit to the field and then whatever the expression spits out would be written to the Write-To Field. So, continuing the dumb example from above:

Say you want it just like above, but if the file in question happens to be in the Classical Genre, you want the comment that gets written to always be "Mozart Rules". For all other files, you just want it to keep working like described above, but if the Genre is Classical, then only that text can be written out. (I don't know why, it is a dumb example.) So, to accomplish this, you'd just put this expression in the Expression to Apply to Input box: If(IsEqual([Genre],Classical,1),Mozart Rules,[Comment Light])

When you enter something in the box and hit Enter, this expression would be evaluated and its output goes into [Comment] instead of what you typed in the box. So, if [Genre] is "Classical" then it would ignore what you put in the edit box, and always write out Mozart Rules. And, to preserve the original functionality from above, you would "pass through" the input entered by referring to "itself" in the expression.

It would not "go through" and write-out "Mozart Rules" to all of your Classical files. It would only do anything at all if you edited the [Comment Light] field and put something (anything) in there. The two fields would not be tied together in any way (other than that one-way writing behavior). So, you could still tag your Classical files with any comment you want, but you'd have to actually write to the "real" [Comment] field (by opening it up in the Tag AW, for example). You just couldn't do it through the [Comment Light] field you made.

Conclusion
In most real-world cases, of course, the expression that the Calculated Field displays would have some relation to the Write-To field and probably contain it, as it does in my Artist Swapping example from way above. But it wouldn't even have to reference it at all. It would be weird, but you could have a Calculated Field that doesn't even use or display the value stored in the field it writes to at all. So, using the example above, if you set the Write-To field to [Keywords] you could use it to enter Keywords, but the value displayed in [Comment Light] after editing (since it doesn't reference [Keywords] in its expression) would always stay the same. You could also have two (or more) different, completely unrelated Calculated Fields which write to the same User Data field. That's not a problem because it only happens on write, when you hit Enter.

And that really brings up one of the main benefits of this method over the Display Expression suggestion discussed above. My way would be much more flexible. You wouldn't be limited to just displaying that [Artist] in one way. You could have 5 or 6 different Calculated Fields, which show [Artist] in a variety of different ways (maybe formatting it one way for Theater View and a different way for Standard View), but use all of them to allow you to edit the [Artist] field "underneath".

The other way would work, but you'd be limited to a single Display Expression for each particular User-Data field. If you want to make another Calculated Field that uses the values in a User-Data Field to display the contents in a different way, you could, but then you couldn't use those to edit it. You'd only be able to edit the "original", and the "original" would always display with the Display Expression applied. So, that would be another issue. What about when you want to show the original value in your User-Data field, without that expression applied? You wouldn't be able to directly. You'd have to make an expression to point to it, which is silly. And that workaround assumes that the output of the Display Expression would only be used for display, and not actually impact other uses of the User-Data field in the Expression Language. That's not something MC can do now, so it would be entirely new functionality. Without it, the actual stored value in the User Data field would become completely inaccessible.

And, perhaps most importantly, modifying the Calculated Fields to allow writing (instead of modifying the display of User-Data fields with an expression) would not require JRiver to "muck with" the way standard User-Data fields work at all. Which, for architectural reasons I'm guessing, they've often been loathe to do.

Everything would work exactly like it does now, except that when you edit a Calculated Field (which you can do now, if Edit Type isn't set to Cannot be edited) it would actually do something useful, instead of just throwing the edits away.

Again, though, I'm okay with either solution. For the Artist Name Swapping business, they're basically functionally equivalent (I just happen to think the other way is a bit better, more flexible, and a little more elegant).

[glynor]

Okay, so now that you (hopefully) understand what I'm proposing, how does it work for the Artist Name Swapping example?

First, you'd want to make two new fields:

[Artist is Personal Name]

(Click to embiggen.)

[Artist Display]

(Click to embiggen.)

Then, you could go through your Library and put all of the Artists back to "normal" and check the boxes for any Artists where the value is a "person's name" rather than a "band name". You could hide the "real" [Artist] field from everywhere (unless you want to use the "raw value" of it somewhere).

I'm showing the "real" [Artist] tag in the Tag AW here so you can see what it really contains in the example (and because I didn't want to screw up my whole Tag AW layout to make the example screenshot), but you wouldn't need to have it displayed anywhere at all, even in the Tag AW. You could just do your edits through the [Artist Display] field instead.

Everything up to now works fine with MC as it is currently. The only issue is that you can't edit values in the [Artist Display] field you made. If you do, they're ignored. If you want to actually be able to modify what shows in the [Artist Display] field, you need to edit the separate [Artist] tag. That stinks.

So, with the new stuff, there'd be two new boxes down below the Calculated Data text box when you're setting up the [Artist Display] Calculated Field. You'd have Write-To Field and Expression to Apply to Input.

• Write-To Field would be a combobox (drop-down picker) where you can choose from any of the existing "regular" Data-Type Fields, and you'd pick [Artist].
  
• Expression to Apply to Input would be a textbox where you could enter a different expression (just like the the one that is already there, but for input filtering). Here you'd put basically the "opposite" of your original expression (so that you could "de-transform" the entries): If([Artist is Personal Name],Unswap([Artist Display]),[Artist Display])

So that way, if you edit the [Artist Display] field and type something, or choose from the picker (which works now, as shown in my screenshot above) it would save whatever you choose into the [Artist] field. BUT, since you DON'T want to write it out as "Swift, Taylor" it would first apply If([Artist is Personal Name],Unswap([Artist Display]),[Artist Display]). You'd have to make sure to check the box for [Artist is Personal Name] first, but then whatever you enter (or pick) would get saved "un-swapped".

The beauty of this is that most of the time, you wouldn't need to do anything at all. When you first import a track from a new solo artist, you'd just have to check the [Artist is Personal Name] checkbox. But you'd usually leave the text in [Artist] alone (assuming it came in well tagged from whatever place you downloaded it or whatever). Since that [Artist is Personal Name] field is relational, it would be automatically checked for any new files you import, after the first one, with the same [Artist] name going forward (as shown with Taylor Swift in my example above).

Presto, all of your Views would show it reversed, it would sort reversed without having to manually swap anything or do any edits at all.

And, since the default value of [Artist is Personal Name] is unchecked, you wouldn't even need to go through your entire Library and fix the ones you did manually swap before right away. They'd keep displaying right because they'd be treated as "band names" until you fixed them and checked the box.

I'm honestly not as excited to get the [Artist] issue in particular fixed. It is annoying that each time I import new tracks from a solo artists that I downloaded from Amazon or whatever, that I have to manually swap the names on them. But, it is only one-time pain. I just type =Swap([Artist]) into the Artist edit box and it goes where it should.

Where I really want it is all the stuff that gets filled in automatically. Especially [Actors], [Director], [Screenwriter], etc that get filled by TVDB lookup. Those all come in "firstname lastname" style. You could manually swap those too after you import, I suppose. But that's super-annoying, and you'd have to remember to re-do it every time you re-do the lookup (and I do "refresh" mine from time to time because often the metadata available in the databases improves over time).

Lastly, the stuff in your comment about multiple names (3 and 4 names) doesn't make any sense to me at all, because Swap() and Unswap() already work on those. So that was confusing, but... Doesn't really matter. If you wanted to do fancier things with the [Artist is Personal Name] field, you could. Or, maybe you have more than one "setting" for how the [Artist] is handled? Whatever.

[RoderickGI]

You just posted while I was writing. I'll post this and read your new post.


Your [Comment Light] example is too simple, and it still has the same concern about having to display two fields in a View.

For example, if I only have the [Comment Light] field in the View, it says Not Empty, and I want to add something to the comment, I need to see the [Comment] field in order to do so. I don't want to just blindly overwrite the field. I may want to just edit it a little, or add something to the end, or whatever. I need to see the field to do that.

I guess if the [Comment Light] field used the expression: If(IsEmpty([Comment]),Empty,[Comment])
Then I could see the Comment before starting an edit. But would the edit give me the text from the [Comment] field as a starting point? I would need that. Checking... Okay, if I currently create a calculated field and edit it, the content of the field is carried into the edit, so I guess that would work.

So looking back at your Artist example, for values in the [Artist] field of "Taylor Swift" or "Pink Floyd":

Writable Calculated Fields Version

• Field: Artist Display (custom Calculated Field)
  
• Expression: If([Artist is Personal Name],Swap([Artist]),[Artist])  > [Artist Display] would show "Swift, Taylor" or "Pink Floyd"
  
• Write-To Field: Artist
  
• Expression to Apply to Input: If([Artist is Personal Name],Unswap([Artist Display]),[Artist Display]) > Then "Taylor Swift" or "Pink Floyd" would be written to the [Artist] field.

Okay, that makes sense. As long as the "Expression to Apply to Input" doesn't result in a loop. As it is writing out to the [Artist] field, then I guess it shouldn't.

Also the "Expression to Apply to Input" would need to write back a value consistent with the original field value format. For example a error such as the expression:
If([Artist is Personal Name],([Artist Display]),Unswap[Artist Display])

Would be a problem, as it would write "Swift, Taylor" or "Pink Floyd" to the [Artist] field instead of "Taylor Swift", which would then be processed by the [Artist Display] expression to become "Swift, Taylor," I think, and each time the [Artist Display] expression was evaluated that would get worse, adding commas.

[glynor]

Your [Comment Light] example is too simple, and it still has the same concern about having to display two fields in a View.

How? Explain exactly why you'd ever need to show [Comment] in the View? If you only want to know if [Comment] contains some text, and need to be able to edit [Comment] occasionally, you could just use [Comment Light] in all of your Views everywhere.

For example, if I only have the [Comment Light] field in the View, it says Not Empty, and I want to add something to the comment, I need to see the [Comment] field in order to do so. I don't want to just blindly overwrite the field.

Of course, if you need to see exactly what is currently in the [Comment] field, you'd need to be able to see it. But that's what the Expression to Apply thing is for. You could edit them just as they look in your Calculated Field, and they'd be "right" when they go into their destination.

And there are lots of situations I can think of where I don't care what is currently in the field, but I might want to write to it. I'll provide an example in a separate post.

[RoderickGI]

Our posts align.

Well, not on the simple [Comment Light] example. I explained why I would want to see the comment. Because I want to see what I am about to edit. Sure, I could see the "Not Empty" value, and then use the Tag AW to edit it, but that is an extra step, and that wouldn't be true for a more realistic example. As a user I would also need to remember that this is a special calculated field, and so not to just directly edit it if it says "Not Empty".

But I guess I understand your [Artist] example now.

However, in the specific [Artist] example there would be a problem if the [Artist] field held the value "Taylor Swift; The Sweetback Sisters", as one is a personal name and the other is not. As [Artist] is a list type field, that could be a problem. Swap and Unswap will handle List type fields though, so as long as all Artists were either personal names or not, then the example should work fine. I'm not sure how the relational field would handle that situation.

Without testing everything, muy understanding was the Swap and Unswap still fell over with some name formats. In fact, it can't handle "Bary Grant Flowerdale Jr", returning "Jr, Bary Grant Flowerdale". It is similar with any three element name with a suffix. But I guess those are rare and could be handled by exception.

However, a couple of more examples and I think you would have me convinced, even given the concerns above. This would be a power user's tool only, in all likelihood, so caveats are fine.

Oh, I think you updated Reply #19 after I loaded the page. I've skimmed it but haven't thoroughly read it yet... Looks okay. Doesn't change my thoughts. Other activities call, even in Stage 4 lockdown!

[glynor]

I have now a field called [Watched] which displays a checkmark when files have been previously fully-played, blank when they've never been played at all, and a percentage when they've been started, but never completed:


It works great. But, one problem is that if I watch an episode of a show in Plex on my iPad, they stay "unchecked" in MC.

But, with this, I could set the Write-To Field to [Number Plays] and the Expression to Apply to: If(IsEqual([Watched],✔)Math([Number Plays] + 1),Math([Number Plays] - 1))

Then, when I "edit" that field, I'd get a dropdown showing a checkmark, blank, and a bunch of percentages (just as I do now):


I pick the Checkmark, and Number of Plays increments by 1, which causes the [Watched] field to spit out a checkmark anyway, so it looks like I edited it. If I pick any other value, it decreases my number of plays by 1 (which would be handy to "mark as new" the files). Yes, I know I can do this by right-clicking on the files and choosing Library Tools > Mark as Played/Not Played, but it would be much nicer to do it right inline (and we could probably edit these values in Theater View, which would be huge).

I'd probably make mine more complex to make entering a blank value actually set [Number Plays] to zero, and a percentage (any percentage) de-increment the counter like that (or maybe I'd have it ignore these, not sure). But you get the idea, I hope.

Bringing it back around to the [Artist] swapping example, since [Artist Display] displays this:
If([Artist is Personal Name],Swap([Artist]),[Artist])
But when you write to it, it does this (which is NOT the same expression, it is the opposite):
If([Artist is Personal Name],Unswap([Artist Display]),[Artist Display])

Also the "Expression to Apply to Input" would need to write back a value consistent with the original field value format. For example a error such as the expression:
If([Artist is Personal Name],([Artist Display]),Unswap[Artist Display])

Would be a problem, as it would write "Swift, Taylor" or "Pink Floyd" to the [Artist] field instead of "Taylor Swift", which would then be processed by the [Artist Display] expression to become "Swift, Taylor," I think, and each time the [Artist Display] expression was evaluated that would get worse, adding commas.

No it wouldn't. It would UNswap the input when you write to it, so it would convert Swift, Taylor back to Taylor Swift and write that to the [Artist] field.

[glynor]

However, in the specific [Artist] example there would be a problem if the [Artist] field held the value "Taylor Swift; The Sweetback Sisters", as one is a personal name and the other is not.

That is correct. That is an issue, but you'd just leave those set to "band mode" (the default) and name-swap them yourself. Which is no worse than what I'm doing now. Just only necessary once in a while.

There's a perfect set of examples in my screenshot above (I picked Taylor Swift on purpose, in fact):
Swift, Taylor;Sheeran, Ed would work just fine (and could have the [Artist is Personal Name] box checked).
Swift, Taylor;Bon Iver would not, so you'd have to actually enter Swift, Taylor in that manually and uncheck the box. Since Swift, Taylor isn't Taylor Swift, and the relational-ness keys on [Artist] and not [Artist Display], the system treats those separately:


Same with things Swap() and Unswap() don't handle well. (Though I have a regex that fixes up some of the Jr., Sr., III, II, etc stuff too, and if you search for old posts here, you can find details, and then you could include that in your system too.)

For anything that isn't "simple" you'd just put them in "band mode" ([Artist is Personal Name] unchecked, which is the default anyway). For my other auto-metadata lookup fields, I'd make override checkboxes (hidden just in the Tag AW for when needed) that would avoid the swapping but which would default the other way. [Director is Special], [Actor is Special], and so on and so forth.

[RoderickGI]

No it wouldn't. It would UNswap the input when you write to it, so it would convert Swift, Taylor back to Taylor Swift and write that to the [Artist] field.

I switched around the expression so it would fail. Just a demonstration of what could go wrong.

[glynor]

I switched around the expression so it would fail. Just a demonstration of what could go wrong.

Well, sure. If you muck up your expression. But that would be true of lots of the things in MC. If I muck up my expression when I type into the box with =Swap([Artist]) it'll mess up my data too.

But you'd only have to type the "opposite" expression out once. So, you know, be careful. And... I wouldn't do it with a crapload of files until it was well tested (which is part of why I like the system too, because it defaults to "band mode" so I could test with a few files before I converted my whole library over to the new scheme).

[RoderickGI]

The Watched example is a good one.

(and we could probably edit these values in Theater View, which would be huge).

Unfortunately last time I checked you still can't select a Checkmark for a field in Theatre View. Though maybe with your custom Watched field, and a list of percentage values in addition to the Checkmark, this wouldn't be the case, as the field doesn't have an Edit Type of "Check"? I haven't looked back at your Watched setup thread to confirm.

[glynor]

It does not have an edit type of Check. It usually has an edit type of Cannot be edited. I had to change it for the screenshot.

All of my Calculated Fields do, in fact, except the ones that use Checkmarks and stars. It annoys me that MC can't still show a checkbox or stars or whatever, but not allow editing on it (have the checkbox visible, but disabled).

This could solve that problem too, because if Write-To Field is blank, then the field would be by-definition, read-only, and they could just always disable them in the UI (but still let you pick different Edit Types for display). But if you leave it set to Standard edit type, then it shows all the values available in the dropdown (see my screenshot).

And, if they unlock the Acceptable Values box for Calculated Fields too (like Edit Type is unlocked now), I could make it so that dropdown would only contain a checkmark or an anti-sign, and then my Expression to Apply would look for those to do it's logic.

Which would be even better.

I have LOTS of examples like that. Things where I'd love to use my nice "combined for display" Expression Fields (used all over the place in my Views) to give me toggle or increment up/down types of control.

[EnglishTiger]

I'm curious as to how it would cope/work with tracks that were performed by Multiple Artists as in:- an individual artist + a group; or more than one individual artists. given there are a multitude of ways those combinations can be input/stored in the database and tags?

In addition how would it cope with artists who frequently use a "Stage Name", i.e. Sting; as the performer but their Proper Name, Gordon Sumner, as composer/song writer?

[glynor]

I'm curious as to how it would cope/work with tracks that were performed by Multiple Artists as in:- an individual artist + a group; or more than one individual artists. given there are a multitude of ways those combinations can be input/stored in the database and tags?

In addition how would it cope with artists who frequently use a "Stage Name", i.e. Sting; as the performer but their Proper Name, Gordon Sumner, as composer/song writer?

It wouldn't deal with either one directly. Unchecking [Artist is Personal Name] would be the override, effectively. The way I do that now, for what it is worth, is I add both names as artists. So, right now, for Tom Petty and the Heartbreakers tracks, I have them tagged as:
[Artist]="Tom Petty and the Heartbreakers;Petty, Tom"

I'd leave them that way, and I'd leave [Artist is Personal Name] unchecked.

For [Composer], I'd absolutely use it, just like I described above for [Director], [Actors], etc. All of my composers for my Classical music collection are in with their actual names in "swapped" order now. I'd instead:
1. Make the [Composer is Special] field which would be just like [Artist is Personal Name] but "reversed" in logic.
2. Make [Composer Display] with If([Composer is Special],[Composer],Swap([Composer]))
3. Make Expression to Apply to Input for [Composer Display]: If([Composer is Special],[Composer Display],Unswap([Composer Display]))
4. Fix all of my current [Composer] values by using =Unswap([Composer]) on them (or just control-A and check the [Composer is Special] checkbox on all of them until I felt like doing it).

So, it would work the same as [Artist] but the "default mode" would be reversed. That would be nice because most of my values in [Composer] are personal names. Currently they're manually swapped, so I'd have to un-do that for those, but that wouldn't be too hard because there would be few outliers. And once done, I'd just need the checkbox for your occasional weird name that Swap() and Unswap() puke on for whatever reason.

I'm not sure if this is what you meant, but...

If you want your [Artist Display] to include the Composer, even if those value isn't explicitly included in [Artist], you could certainly do that. My custom [Artist Album (Smart)] field does stuff like that. (I use that because I don't like the behavior of the built in [Artist Album (Auto)] field.)

Your [Artist Display] field Expression could use IsEmpty() to test if [Composer] is blank or not, and if not use ListBuild() to include those values in the display. You wouldn't be able to use it to write to [Composer] because you couldn't pick two different fields in the Write-To Field combobox (and wouldn't want to, because how could it know which goes where). But you could still use it to write to [Artist].

I probably wouldn't do that. I use composers mostly for classical (and some for Jazz). But you could if you want to see all of those values combined in a Pane or a Theater View Category.

Again, all of that stuff you can already do in MC. It just isn't very viable because you can't edit Calculated Fields, and so you are forced to include the "real" [Artist] or [Composer] or [Director] fields in the Tag AW (at least), if you ever want to be able to edit the contents by hand. And you have to include them in panes if you want to use the Pane Checkboxes to tag them (which I do extensively).

[glynor]

I should note: I left off &datatype=[list] from all of my [Artist Display] expressions above in basically all of my posts. If you're doing it for real, you'd have to append that.

Including that makes the forum code angry because list makes a list, and I always forget about the [nobbc] tag.

[glynor]

Okay, that makes sense. As long as the "Expression to Apply to Input" doesn't result in a loop. As it is writing out to the [Artist] field, then I guess it shouldn't.

Yeah. Bringing up the potential for looping above was a very good point from zybex. I'm glad they (and you) made me think about it more.

Limiting the fields you can choose in the Write-To Field combobox to only Data Type fields is the best solution for that, I think. That does cut out the potential use-case of having "cascading sets" of Calculated Fields to do super-fancy things (like transform the input in multiple ways, depending on conditions).

If it is possible to let us go whole-hog and pick any field we want, including other writeable Calculated Fields, that would be great. I'd take it (and I might do something clever with it someday). But I can't even think of a real world usage for that right now that I'd ever actually want to implement. And they'd have to make darn sure that it can protect itself against a user short circuiting it into an infinite loop with a poorly thought out set of expressions. You could do what I said originally and make it only allow writes to an individual field once per edit, but... That seems like special casing and a bunch of work for an edge-case usage, that maybe nobody will ever use.

So, I'd say just make it so it can only write to "vanilla" fields. That's safe because they can't trigger another write, and so you can't build chains. And it is simpler.

[glynor]

Well, not on the simple [Comment Light] example. I explained why I would want to see the comment. Because I want to see what I am about to edit. Sure, I could see the "Not Empty" value, and then use the Tag AW to edit it, but that is an extra step, and that wouldn't be true for a more realistic example. As a user I would also need to remember that this is a special calculated field, and so not to just directly edit it if it says "Not Empty".

You could edit [Comment Light] in-line in a file details listing too.

But, yeah... That's why it was a dumb example. I was just trying to get the point of the system across, without all of the complexity of list data types, swapping names, and the like. You could do that, but it would be stupid to actually do it.

In real-world usages, you'd edit them either by typing something that looks just like the other values in the field (like with [Artist Display]) or you'd pick from the drop-down menu.

[zybex]

To make it simple, I thought I'd briefly summarize the two proposals outlined in this thread:

Display Expression and Input Template
In Tools > Options > Library & Folders > Manage Library Fields, for all standard User Data type fields, provide two new settings:

• Display Expression: Shows a Expression-editing textbox. When blank (the default) the value of the field is displayed, exactly as they work now. When filled with an expression, this value is substituted anywhere the values of the field are displayed (panes, grid-view, Tag AW, etc).
  
• Input Template: Allows the user to define a tooltip-style "informational" display when the users are editing this field to describe the required input format.

Note: The purpose of the Input Template is to prevent the situation where the user enters the data as it appears for all of the other fields, and receives unexpected results due to the Display Expression transformation.

There is no need for an "Input Template" in my suggestion, just the Display Expression. If you mean just some text to be shown as a tooltip/info, that's fine - but it's not a requirement.

Also, the Display Expression would be available on non-calculated fields (user-writable). Calculated fields would still be read-only, there would be no need to make them pseudo-writable as in your proposal.

[zybex]

Let me try to formalize both approaches:

Your original idea - WriteRedirect™:
FieldA: normal read/write field
FieldB: calculated field + writeRedirect to FieldA:
    read/displayValue = Expression(FieldA, others)
    write(value) -> FieldA = InputExpression(value, otherFields)

Simplified idea - DisplayExpression™:
FieldC: read/write field with DisplayExpression
    read/displayValue = DisplayExpression(FieldC, otherFields)
    write(value) -> FieldC = value
   
    Standard field syntax could be used:
       [FieldC] = output of DisplayExpression for FieldC  => same as [FieldB] in your idea
       [FieldC, 0] = raw value without DisplayExpression => same as [FieldA] in your idea

These ideas are actually equivalent:
- Value of FieldA -> shown via expression of FieldB -> Edits written to FieldA (after massaging with optional InputExpression)
vs
- Value of FieldC -> shown via expression of FieldC -> Edits written to FieldC (optional massaging can also be added - I dislike it)

The difference is only where the "Display Expression" is stored - FieldB is in effect the holder of that expression, and nothing else. The simplified version stores it all on the same FieldC. So the second one is a self-contained/simpler implementation of the same concept, doing away with the need to have 2 expressions for the same field. The rest is eye candy and user assistance - the "input expression" or "tooltip" is in effect a separate concept that can be added to either implementation to help the user when entering data.

For JRiver the implementation is also simpler - they can reuse the existing Field Expression box to enter/store the field's Display Expression.

The need for that InputExpression is problematic in my view, and would not work well in practice, as it requires (as I said above) well balanced expressions that complement each other, and wouldn't really work when the wanted display value is calculated based on multiple input fields.

[glynor]

These ideas are actually equivalent:
- Value of FieldA -> shown via expression of FieldB -> Edits written to FieldA (after massaging with optional InputExpression)
vs
- Value of FieldC -> shown via expression of FieldC -> Edits written to FieldC (optional massaging can also be added - I dislike it)

The difference is only where the "Display Expression" is stored - FieldB is in effect the holder of that expression, and nothing else.

They aren't equivalent. I get that you might not care, or it might not matter for your workflow, but without a way to automatically "massage" the input data, I probably wouldn't implement it at all on [Artist]. I'd use your suggestion for my auto-metadata-filled fields, but only those.

Because:

• How would I implement my [Watched] idea with your system described above?
• How would I tag [Artist] on my files using the Panes checkboxes or "dropdown suggestions" in the [Artist] combobox? All of that functionality would become useless. The only way to tag the files would be to manually type values into the box, and even then, you'd have to just "know" that you had to type them opposite of how all the other values look in the field (ignoring all suggestions to the contrary)?

Because of the necessary mechanisms for inputting data into MC. The input data wouldn't match what would be displayed by the field. That way would be "worse" (in my mind) than the perfectly-possible-now method of just showing the Calculated Field in your Views and editing the underlying tags only in the Tag AW. You'd still have to "give up" on ever using those fields where they have a Display Expression in a View to input data (unless you know the "secret method" and type them all by hand).

Maybe I'm not understanding. If you could outline, like I did above, how to solve those issues maybe it would make more sense? I'm interested if there is really a simpler solution where you "unify" the field types. But if so, I'm not seeing how it would work.

Note: I still don't think JRiver is ever going to change how User-Data fields display. I think that probably endangers optimization and caching, and... Fundamentally, they want those to be "simple data storage fields". The point of the Calculated Fields is to serve that role. But, I could be wrong there.

[zybex]

It is equivalent. As I mentioned, the Input validation/massage expression can be added to both methods. The only difference is the write-to location: In your suggestion you need a second field, on my suggestion the write-to is the field itself. Everything else is the same.

Your [Watched] example has some problems - what happens when you change the value from 50% to 60%? Your [Number of Plays] gets decreased! Then you change from 60 to 85%, and it again decreases; even if you edit and just enter the same value (70 -> 70), it still reduces the [number of plays] because the expression is still executed.

With plan B you can just have [Watched] be a standard field holding a number from 0 to 100 (percentage), with DisplayExpression = If(IsEqual([Watched],100), ✔, [Watched]%). When you edit the field you can still have the dropdown of values - what it then displays is dependent on the current field value. You can also have another field for [Number of Plays], but that's independent.

As for the [Artist] tagging, all tag suggestions/dropdowns/autocompletes would still work with the DisplayExpression values, while the actual underlying stored value would be the raw value. The concept is simple - anywhere where a name is *displayed* onscreen, it uses the DisplayExpression; but the underlying value to be stored would be the corresponding raw value.

[RoderickGI]

Well, I would prefer it all to be done in one field, and to be simpler to understand, but...

Zybex, if I wanted to correct the spelling for an [Artist] field that displayed swapped as "Swift, Tailor", how would I know to enter "Taylor Swift"? That is the level of question that most users would have.

I can think of a simple way, and I'm sure you can too. Just display the raw data when you edit the field. But that doesn't happen in any existing functionality now, so it would require changes to the MC core functionality.
What about in drop-down lists and during Pane editing? You would want both to sort by "Lastname, Firstname", because that is what you see when viewing the data, but if that is the case, how would a user know to enter "Firstname Lastname"?

Also, you would have to read Glynor's whole thread on his custom Watched field to understand why he is suggesting what he is for that example. The [Watched] and [Number Plays] fields are not independent, even in standard MC functionality.

If the issues with using a single field with existing functionality could be resolved, that would be great. But there is some thought required on that. Some detailed thought. Probably by developers who understand the underlying mechanisms used in MC.

[glynor]

Zybex, if I wanted to correct the spelling for an [Artist] field that displayed swapped as "Swift, Tailor", how would I know to enter "Taylor Swift"? That is the level of question that most users would have.

I think it is worse than that. You wouldn't be able to use the Panes or combobox drop downs (in a grid-style File Listing View) at all. They just wouldn't work, at least without a substantial re-coding. And typing in the box manually, even if you could see the "original values" wouldn't always be useful.

I think the fact that "Taylor Swift" and "Swift, Taylor" are related terms (easily "mentally" transformable) is confusing the issue. That's why I tried to explain with a simpler example above, but maybe a different simple example would be more clear.

Imagine you want to make a basic enumerated type field where you can choose between three different pre-selected "audience sizes" with a default value of "empty/unset". So, you create an [Audience Size] Integer Field that can contain 0-5. When this shows in Views in MC, you want to display:
0 = blank
1 = tiny
2 = small
3 = medium
4 = large
5 = huge

Since this is an ordered list, it is handy to do it as an integer-based enum like that because you can then do math on it. So, for example, to search against this field you can do [Audience Size]>[2] to find all the files with medium and larger audience sizes. To determine if the audience for file x is "larger than" file y, becomes if (x > y).

So, you write a Display Expression that does IfCase() and spits out tiny, small, medium, large, and huge when appropriate, and you have this field added as a Category in a Panes-style View. You have a brand-new file that just imported, and so [Audience Size] is set to 0 and it is listed under "Unassigned" in the Pane. You select it in the file list, and check the box next to medium in the Pane.

How does it know to assign it as 3?

The expression language is a one-way text transformation language that works on one file at a time. For each file, your IfCase() expression turns the value into "medium", and then the Pane displays that list of values. It doesn't get any integers at all, and doesn't know that the integers exist.

So, when you go back the other way, and try to write to it by checking the checkbox next to "medium" in the panes, how does MC know to write 3 to the field? To know to do this, it would have to be able to run the expression you defined "in reverse" and re-convert medium back into 3. But because the expression language operates on text, they're not all simple 3 = "medium" dictionaries. They can have very complex logic that can't be trivially "auto-reversed".* Even this one would be difficult. What if the text says "MeDiUM"? Should it be case insensitive and reverse it to 3, or case sensitive and error or spit out 0 for unassigned? You can't define the behavior, because there's no way to include the case sensitivity logic in your original expression, because it doesn't "go in that direction".

So, you can't reverse them without the user providing the method. And without being able to reverse them automatically that would totally break the Panes and "dropdown comboboxes (in grid-view and in the Tag AW). To fix it, they'd almost certainly have to dramatically recode the whole system to pass through both "original values" and "displayed values" and keep track of both as "objects" in the panes (which would also likely impact performance).

Beyond that, if you do just give up on the Panes and combobox pre-filled values, and intend to hand-type everything. It still becomes an issue. So, when you edit the field, your suggestion is to have it just effectively "turn off" the Display Expression functionality and let you "see" the underlying data any time you "edit" that field with a keyboard.

Well, continuing my audience size enum example, you select the field in the Tag AW ready to type in your value. It turns off the display expression, and so your currently-blank file shows 0 in the box. What do you type there to make the file turn into "medium"? How do you know to do that?

Sure, you could write some kind of instructional "how-to" that pops up somewhere. But you'd have to type it up when setting up the field (and maintain it as you add additional sizes to your enum), and it could quickly become unwieldy. What, happens when you're using an enum for colors or something, and there are 32 or 64 of them possible?

Obviously, it would be much better if you could just pick "medium" from the drop down, or type "medium" (or "MeDiUM" because your reversing IfCase() is doing IsEquals() in case-insensitive mode) and have it set the value to 2 for you. And bonus, the Panes and comboboxes work again.

Those are all big problems with doing it in a single field at least as I understand it. You'd have to be able to define both a Getter and a separate Setter function in order to do the transformation both ways.

* The Swap() and Unswap() dilemma is a perfect example: the language doesn't "know" the Swap() and Unswap() are effectively the reverse of each other. So it wouldn't know that when "Swift, Taylor" is coming in "one end" of the expression, how to use Unswap() to turn it into "Taylor Swift" before writing it to the file item in the database.

[zybex]

Glynor, you keep misunderstanding a key part of what I'm saying even though I've repeated it a few times. I'm just saying we don't need the "write-to" functionality (it can be done in a single read/write field), but you can still have the InputExpression to process what the user enters. These are two independent features.

The only difference between proposal A and B is where the raw field value is stored - in your case it's in a different field, in my case it's on the same field. Nothing else. Everything else can be done as per your proposal.

Your idea has a field 1 that is read/write, and a field 2 that is read-only. I'm saying we don't need the read-only field to implement your idea.

Edit: To clarify (I hope) - in your proposal, you need a Field2 of type "Calculated", with some ExpressionX, and pointing to a write-to Field1. What I'm saying is that you can simply enter that same ExpressionX as the "DisplayExpression" of Field1. Everything else of your proposal can be added on top of this. DisplayExpression is the Getter, InputExpression is the Setter (better call it SetterExpression)

[RoderickGI]

Difficult, isn't it?

I think we (Glynor and myself) have been hung up on the current functionality of Calculated fields.

So Zybex I think what you are saying is that you don't use a calculated field (which is more or less read-only) anymore, but use a new field type that has a "Display Expression" and a "Commit Expression".

So in the Audience size example (because examples work way better for me).

• The raw data held in the [Audience Size] field would be integers from 0 to 5.
• The "Display Expression" would convert that raw data to the text equivalents of those integer values, probably using IfCase() expression. Easy enough.
• Theoretically, the text value would be presented in all Tag AW edits, Panes views, or View columns for in-line editing. Drop-down lists would present the text values.
• When I user does as edit, the "Commit Expression" would reverse the text value selected back to an integer, again probably using an IfCase() expression.

The [Audience Size] field, being an integer, can be used in maths expressions as required, probably using the [Audience Size,0] format. That would be consistent with, for example, the [Date] field, which is a decimal number but has a "Display Expression" hardcoded into MC. The [Date] field actually also has "Commit Expression" hardcoded in MC, as I can enter "25/09/2020" (yes, I know, "09/25/2020" for you Americans) and that will be converted to a decimal number representing that date.

In fact, the [Date] field has a sophisticated "Commit Expression" that lets a user enter just "2020", or "25/09", and it will interpret what the user means, most of the time. The [Audience Size] field "Commit Expression" could handle similar though much simpler data entry, such as accepting either the numerical value or the text value, and still select the correct numerical value to update the field with. That could even be facilitated by having the "Display Expression" output combined values, such as "3 - Medium", rather than just the text value. That would allow fast manual data entry, if that was desireable.


So Glynor, forgetting existing Calculated Field functionality, if we had the ability to set up a user defined "Display Expression" and "Commit Expression" for this new field type, and these fields worked much like the [Date] field does now, would that do everything you want?

My brain hurts too much to go back through the discussion and validate that proposition.

[zybex]

So Zybex I think what you are saying is that you don't use a calculated field (which is more or less read-only) anymore, but use a new field type that has a "Display Expression" and a "Commit Expression".

Right, though I think there's no need for a new field Type - just adding optional settings on the existing fields for the Display and Commit expressions (except on Calculated fields - these would remain as-is, read-only). It should work for existing types of string/decimal/percentage/list/etc. The different options for "edit type" might require some thought but should still work, as would the "list of accepted values" for a dropdown.

[RoderickGI]

Yeah, I just meant not using the Calculated field functionality, which is currently activated by selecting the radio button.

Correct on the "Edit Type" and list of "Accepted Values". Even the Relational functionality might work, although that would be a corner case! So basically an extension of standard "User Data" type fields.

[glynor]

Glynor, you keep misunderstanding a key part of what I'm saying even though I've repeated it a few times. I'm just saying we don't need the "write-to" functionality (it can be done in a single read/write field), but you can still have the InputExpression to process what the user enters. These are two independent features.

The only difference between proposal A and B is where the raw field value is stored - in your case it's in a different field, in my case it's on the same field. Nothing else. Everything else can be done as per your proposal.

I got completely that you intended to eliminate the distinction between User-Data and Calculated Fields. I didn't realize you were suggesting the Setter Expression (as opposed to RoderickGI's suggestion of an instructional display) also be added to the standard fields, but... There is one other important difference, and it would be complicated to work around.

How would you access the underlying (unmodified) field? So, with my silly audience size enum example, when I use it somewhere (in another Expression, in a Pane, in the Tag AW, etc), how do I specify if I want the original raw value or the expression-massaged value? How do I tell it I want to read or write "2" or "medium"? If I'm swapping artist proper names, how do I display "Taylor Swift"? You'd need some new Expression and Search Language functionality (and corollaries throughout the UI) in order to "unify" Calculated Fields and User-Data Fields like that, otherwise you could never access the underlying field value.

With my suggestion, the [Artist] field is still just waiting there, working like it always did, with the value in it, so you don't need any new language features, display features, or the like.

And that's why Object Properties in C-based languages are based on private variables. Because sometimes you don't want to go through the Getter and Setter, and sometimes you do. Sure, you could invent a new mode to the field() command that bypasses (or enables) the Getters and Setters. But you've now just reinvented the wheel.

I think it would be simpler to just keep User-Data Fields as they are now: plain old variables.
And Calculated Fields like they are now: Properties. Only instead of having them be always read-only properties with no setters, you can optionally pick your own user-data field as a "backing variable" and define a Setter.

I don't care what the options are called. But that seems to me to require the least overhaul of the current system in MC, and the least new functionality. Asking for incremental change always wins when you're asking JRiver for something new. Don't fix what isn't broken.

But if they really feel like doing it that way, sure. It would work either way so long as they implement all of the things discussed above and eliminate the distinction between user-data fields (simple variables) and calculated-fields (expression based properties). To me, that's a lot of work for no benefit other than to eliminate a single combobox where you pick your backing field for your Calculated Field.

[zybex]

How would you access the underlying (unmodified) field?

This already exists:
[Artist]
[Artist,0]

Also, calculated fields would keep existing just as they are now, no change for them.

As for implementation effort, I have the opposite view - this requires much less effort as it's just simple changes to the field Getter and Setters (which already exist for sure, to handle the many special cases/fields already in MC). Some pseudo-code:

Code: [Select]

Field::Get(bool rawValue=false)
{  return (rawValue || Field.DisplayExpression == null) ? Field.Value : Evaluate(Field.DisplayExpression); }

Field::Set(string value)
{  Field.Value = (Field.CommitExpression == null) ? value : Evaluate(Field.CommitExpression, value); }
The LinkedField idea is more complex as it requires:
- validation of which fields to show on the LinkedField dropdown
- Setter is now recursive, as the user may have chains of linked fields
- Setter must detect and handle loops, and prevent stack overflow caused by excessively long chains (MC already does this when evaluating expressions)
- deleting a field now needs to check if there's some other field linking to it
- FieldA writing to FieldB may even interfere with the clean/dirty detection of Database files - writing to a field now causes multiple files to be updated, instead of a single one.
- other side effects of these chains will need to be carefully evaluated

[RoderickGI]

How would you access the underlying (unmodified) field?

I know I write too much, so things get missed, but I did include that in my Reply #42 (Hey, the Meaning of Life, The Universe, and Everything!)

The [Audience Size] field, being an integer, can be used in maths expressions as required, probably using the [Audience Size,0] format.

Also Glynor, you are still thinking Calculated fields. The Zybex solution isn't a Calculated field, as the value isn't defined by an expression independent of user input. It is a field written based on user input. Those are very different things. User edits aren't thrown away, they are processed by the Commit Expression.

As we know, Display Expressions are used throughout MC, only they are hard coded. My [Date] field example is the most obvious.

Having thought overnight, JRiver may want to flag these fields as different to standard "User Data" type fields, so that MC doesn't have to look to see if the fields have a Custom Display and Commit Expression, just to avoid a potential performance hit. But maybe that isn't even a potential problem.

So, ignoring the technicalities of implementation, my question from last night stands:

So Glynor, forgetting existing Calculated Field functionality, if we had the ability to set up a user defined "Display Expression" and "Commit Expression" for this new field type, and these fields worked much like the [Date] field does now, would that do everything you want?

.
.
.
Okay, one small comment on the Taylor Swift example. When [Artist] is imported from an external source, as is common practice it will be in the format "Taylor Swift". The [Artist] field will need to have a Commit Expression, and the data would be written as "Swift, Taylor". Modifying existing standard fields such as [Artist], which already has a lot of functionality built around it, may be an issue for JRiver. That may not happen, and generally doesn't. The underlying MC database structure hasn't been changed since MC18 I think, which is why Library Backups from MC18 still Restore to MC27. New fields have been implemented as User fields. So, all that could be a stumbling block.

Also, there is potential that the Zybex solution would clash with existing hard coded MC Commit Expressions. Either you would have to drop the hard coded processing and just use the user version, or MC would need to accommodate both, which may not be possible. You could not use this functionality on the [Date] field, for example. There may be other restrictions. Hence, another reason why fields that use Commit Expressions may need to be explicitly identified, and if they can't be, it means that JRiver has decide the field isn't allowed to use them.

[glynor]

This already exists:
[Artist]
[Artist,0]

You're totally right. I forgot about that, and that would work. And I finally see what you mean.

- Setter is now recursive, as the user may have chains of linked fields
- Setter must detect and handle loops, and prevent stack overflow caused by excessively long chains (MC already does this when evaluating expressions)

It hardly matters, but FWIW, I removed that from the suggestions earlier by limiting the Write-To Field to only User-Data fields. The building chains idea was a bad one.

But I think we've gotten somewhere in the discussion which wasn't completely clear earlier around the edges of how the entire design could work better. The core of the thing is the Setter function, really, not the Getter. The existing Calculated Data functionality is, effectively, already letting you define a Getter function with an expression. It just can't reference itself (an underlying private variable).

Also, calculated fields would keep existing just as they are now, no change for them.

If this is done, and you convinced me it is better, then it would (IMHO) make more sense to rearrange that section of the Library Manager a bit though. The Disabling\Enabling of the Edit Type and Show a Link Checkbox options is already weird anyway, and this is a good opportunity to clean that up and make it all easier to understand.

I have an idea on that, but I'll put it in a different post to summarize and see if we all agree?

I know I write too much, so things get missed

Yup. I do too, and I get it. But yup, I missed that. Sorry. Realizing that was there already is what made it "click" for me, finally.

[RoderickGI]