INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2   Go Down

Author Topic: Listcombine mode request  (Read 9192 times)

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Listcombine mode request
« on: January 30, 2013, 08:09:06 am »

Would it be plausible to add a mode to list combine?

where list 1 would set the skeleton of the list, i.e all it's values and if values from list 2 were equal to any values in list 1 they would combine otherwise all other entries from list 2 wouldn't merge?

It could be quite useful. I could then finally make that pane I've been dreaming of doing, where I could combine Album Artists and remixers. So all work of an artist would appear under their name but the pane wouldn't be cluttered with remixers that don't have any albums..
Don't know how it would affect performance but if it were a similar hit to global variables it would be a hit I would be willing to take.


Here's hoping :)
Thanks
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #1 on: January 30, 2013, 12:55:57 pm »

Are you looking for a way to "add the two lists then remove duplicates", or a way to say "only output values that are in both lists"?
Logged
Matt Ashland, JRiver Media Center

syndromeofadown

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 812
Re: Listcombine mode request
« Reply #2 on: January 30, 2013, 01:15:48 pm »

It would be: Only output values in both lists

I agree it would be a great feature.
One could use [artist] as the "skeleton" of the list then as add [album artist].
Then when browsing artists the list would exclude artists on compilations that don't appear on non-compilation albums.
But you could still choose the artist/abumartist "Soundtrack" to see the full track listing of the album including the previously excluded artists.
If Artist Images were enabled for this field it would be the perfect default for theater view. All the music without a pile of one track artists.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #3 on: January 30, 2013, 01:51:46 pm »

This was one of the modes I suggested in our ListSort'ing thread:

DupsOnly.
Logged
The opinions I express represent my own folly.

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #4 on: January 30, 2013, 03:34:12 pm »

In a coming build:
NEW: Added an optional mode to ListCombine(...) expression function to allow outputting only values in common to both lists.
Logged
Matt Ashland, JRiver Media Center

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #5 on: January 30, 2013, 04:28:44 pm »

Whoa that was quick thanks Matt :)

Been racking my brain over how to solve that since I started using MC, I always thought it would need some sort of relational ability. And this just occurred to me this morning.. Quite a nifty workaround..

It will make it so much easier to view an artists work now. And much easier whilst in J Remote as well..

Thanks  ;D
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #6 on: January 31, 2013, 09:51:42 am »

Great thanks again.

How do I exactly invoke the mode? It doesn't say on the update thread.
 :)

Edit
Never mind found it :)
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #7 on: January 31, 2013, 10:09:05 am »

can you tell me what I'm doing wrong?

It's working in a sense.. All artists that appear in the pane also have entries in the Album Artist field. But only the tracks that have the artists name in the remixer field are appearing. I've tried mode 0, it combines everything and has all the entries but also has all the 1 track artists..

I must be doing something wrong with the expression here..
listcombine([Album Artist],[Remixer],\,;,1)&datatype=[list]

Edit

Think I get it now I don't think what I said will be possible  :'(
For mode 1, the selection exist must exist in both fields for the same file? Here I thought it would it would just regard both lists as text and if one entry were equal to any other entry on the other list no matter the field they would combine..
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #8 on: January 31, 2013, 10:35:33 am »

I'm having a hard time trying to figure out what input you have and output you'd like.  Can you give a couple of example inputs and desired outputs?

See the wiki Examples for ideas of what I'm looking for:

http://wiki.jriver.com/index.php/Media_Center_expression_language#ListCombine.28....29:_Combines_two_delimited_lists_into_a_single_delimited_list
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #9 on: January 31, 2013, 11:07:22 am »

Ok with the expression listcombine([Album Artist],[Remixer],\,;,1)&datatype=[list] and the artist Machinedrum

I get only two outputs

AlbumAlbum ArtistArtistNameRemixer
BidnezzMachinedrumSteve & RobWallis & Futuna (Machinedrum Remix)Machinedrum
Nastyfuckk EpMachinedrumMachinedrumNo Schnitzel (Machinedrum Remix)Machinedrum

But I also have these remixer tracks where Machinedrum is not the Album artist and they only appear when the mode is set to 0
AlbumAlbum ArtistArtistNameRemixer
Cool World EpAmerican MenAmerican MenTenzin Zopa (Machinedrum Remix)Machinedrum
RemixesBarker & BaumeckerBarker & BaumeckerNo Body (Machinedrum Remix)Machinedrum

I also have several other albums by machinedrum where there is no value in the remixer field and they don't appear when mode is set to 1..

While mode is set to 0, all remixes of machinedrum, and albums of machinedrum appear together regardless if he is the album artist or even remixer. while this is what I was looking for, with mode 0 the pane is still cluttered with artists with no albums
When set to 1, I only get those two tracks in the top table..

Within the pane I've got 98 entries and all of them the remixer and album artist are the same..
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #10 on: January 31, 2013, 12:17:33 pm »

It sounds like you want combine list 2 into list 1 with dups removed, only adding the values from list 2 that are also in list 1?

input:
  list 1: a b c
  list 2: b e
output:
  list final: a b c

input:
  list 1: a b c
  list 2: e f
output:
  list final: a b c

Is this correct?
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #11 on: January 31, 2013, 12:34:48 pm »

Yes that is correct..

I initially thought you could compare contents of both lists and remove entries that don't appear in both lists and combine the rest regardless of which file and field they came from.

sadly, I didn't think about it correctly before I made the request, I thought it could be a workaround to get the results I was after..
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #12 on: January 31, 2013, 12:47:57 pm »

listsort(listcombine(listcombine([Album Artist],[Remixer],;,;,1), [Album Artist],;,;,0))

This combines both lists, showing only dups.  Then, it takes those dups, and combines again with list 2 (remixer) removing dups.  You now have the list you want.
Logged
The opinions I express represent my own folly.

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #13 on: January 31, 2013, 12:51:57 pm »

It sounds like you want combine list 2 into list 1 with dups removed, only adding the values from list 2 that are also in list 1?

If you only add things from (list 2) that are in (list 1), what are you adding?

Could you show an example where (list 1) != (list final)?
Logged
Matt Ashland, JRiver Media Center

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #14 on: January 31, 2013, 12:56:51 pm »

Please don't ask me hard questions before 11am PST.

I didn't notice that.  I'm having a very hard time understanding the OPs goal for some reason.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #15 on: January 31, 2013, 01:01:18 pm »

I think the issue is context.  You and I are thinking in terms of list values from a single track.

I think the OP might be wanting the list of all Album Artists and the list of all Remixers across all tracks, and then don't show Remixers that are not also in at least one track's Album Artist.
Logged
The opinions I express represent my own folly.

vagskal

  • Citizen of the Universe
  • *****
  • Posts: 1227
Re: Listcombine mode request
« Reply #16 on: January 31, 2013, 01:09:20 pm »

I think the OP might be wanting the list of all Album Artists and the list of all Remixers across all tracks, and then don't show Remixers that are not also in at least one track's Album Artist.

Yes! Or at least that is what I want (with Album Artist and Artist in my case). It is not possible with the tools we have now in MC (as far as I can tell).
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #17 on: January 31, 2013, 01:48:10 pm »

You mean, you can't do this (see attached)?

Hey, wait, I just did!
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #18 on: January 31, 2013, 01:57:40 pm »

I think the issue is context.  You and I are thinking in terms of list values from a single track.

I think the OP might be wanting the list of all Album Artists and the list of all Remixers across all tracks, and then don't show Remixers that are not also in at least one track's Album Artist.

Yeah that's what I had in mind originally for some reason I thought this would allow me to do that...

Theoretically what would it take to get this kind of functionality without a relational database? Could a combine list mode be made to do the same but treat all the output from both lists as a text disregarding which files each piece of text came from, then applying the common values to output to some sort of ghost field? That still is relational though I think, Or is t just as cumbersome as having relational capabilities?
Sorry don't have that much knowledge regarding database stuff if I am waffling please just say lol
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #19 on: January 31, 2013, 02:01:21 pm »

You mean, you can't do this (see attached)?

Hey, wait, I just did!

did you do that with this?
listsort(listcombine(listcombine([Album Artist],[Remixer],;,;,1), [Album Artist],;,;,0))
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #20 on: January 31, 2013, 03:22:26 pm »

I used that expression, but had to do it in a different place.  It might run very slowly.

First, you need to modify Set rules for file display in your view (I'd recommend you make a copy of a view first, for testing).

Add the following code to your view, via the Import/Export button:

Code: [Select]
[=save(listcombine([Album Artist,0],load(list1),;,;,0),list1)save(listcombine([Remixer,0],load(list2),;,;,0),list2)1]=1
Now in an expression column for a pane, use the expression:

   listcombine(ListCombine([list1],[list2],;,;,1), [list1],;,;,0)&datatype=[list]
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #21 on: January 31, 2013, 04:15:24 pm »

Strange I can only get this to work if I omit [load(,list1)load(,list2)1]=1

What do this do exactly, the expression column as far as I can tell contains the value for every single artist and remixer in my library. pretty labour intensive.. Better for a maintenance view I think

Whats the best way forward? create a custom field with the edit type as large value, where I can copy this column over and use it with list combine to get the results somehow??

Thanks
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #22 on: January 31, 2013, 06:11:56 pm »

Sorry, not only was that initializing code wrong using the wrong function (load vs. save), it doesn't work anyway because you can't omit the first parameter to save() - I was trying to assign empty.  This is what happens when you copy/paste and change values before posting.  Sorry, corrected above.

See this thread for an explanation of how we're using the file display rules for building your list of artists/mixers that you want displayed:

   http://yabb.jriver.com/interact/index.php?topic=77826.0

The same concepts apply here.

A artist/mixer pane you create already has every single artist/mixer.  We're just building an identical list and using it instead of specifying artist/mixer in a column.  So unless you are seeing serious performance hits, don't worry about it.

You can save the calculated pane column into your own user field in MC.  So long as the file display rules have run, the calculated field will have a good value.

Edit: I see the slowdown I suspect you might be seeing.  I was working with a very small test library.  Now that I let it run amok in my main library, it takes forever.  Oh well.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #23 on: January 31, 2013, 06:49:18 pm »

Matt - if you happen to read this.

The Rules for file display above on my system take about 6 seconds to complete for just under 30k files.  Not unexpected.

But the expression ends up taking just under 2 minutes.

I realize I've abused your fine List*() functions, but maybe you'll see something standout that is an optimization candidate.

Otherwise, we'll just call this an exercise in the boundaries of MrC MC.
Logged
The opinions I express represent my own folly.

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #24 on: January 31, 2013, 06:55:08 pm »

Otherwise, we'll just call this an exercise in the boundaries of MC.

You need one of these:
http://yabb.jriver.com/interact/index.php?topic=77667.msg527438#msg527438


But seriously, could you be a little more specific about the slow part?  You're tossing a lot of expressions around, so I'm not certain which one takes two minutes.  You could send me a library backup if it's easier.
Logged
Matt Ashland, JRiver Media Center

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #25 on: January 31, 2013, 06:59:25 pm »

I do indeed.  It could double as a space heater too.

Sorry - here are the expressions that should  be sufficient.

Rules for file display:

Code: [Select]
[=save(listcombine([Artist,0],load(list1),;,;,0),list1)save(listcombine([Composer,0],load(list2),;,;,0),list2)1]=1
Panes column expression:

listcombine(ListCombine([list1],[list2],;,;,1), [list1],;,;,0)&datatype=[list]

Create a blank panes view and add the rules for file display above.  Time it.

Then add a single expression column with the expression above to a panes view.  Time that.  You probably don't even need values in Composer.

Logged
The opinions I express represent my own folly.

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #26 on: January 31, 2013, 07:27:28 pm »

You're a problem.  A really, really smart problem ;)

I admit that's a pretty neat trick.

Pane expressions get evaluated per matching file.  So in your case, it's doing the same thing 30k times in a row (once for each file).  I never imagined an expression like this, so it's just outside the bounds of what we're optimizing for.  I suppose if we knew there were no file variables in the expression, it could evaluate it a single time.
Logged
Matt Ashland, JRiver Media Center

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #27 on: January 31, 2013, 07:30:22 pm »

Thanks.  You're gonna send me nasty-grams after you have a chance to try the statistics example can-o-worms I opened.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #28 on: January 31, 2013, 07:36:36 pm »

Quote from: ?
Otherwise, we'll just call this an exercise in the boundaries of MrC.

That's funny.
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #29 on: January 31, 2013, 07:40:17 pm »

So there is a chance this might be workable in the future? Sounds promising :)

You are a guru of expressions MrC, i'd be lost without your help, time and time again. You must like the challenge, whereas I still like the challenge but don't know enough about the in's and out's of all the expressions like you..

Thanks
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #30 on: January 31, 2013, 07:59:25 pm »

Matt - if you happen to read this.

The Rules for file display above on my system take about 6 seconds to complete for just under 30k files.  Not unexpected.

But the expression ends up taking just under 2 minutes.

I realize I've abused your fine List*() functions, but maybe you'll see something standout that is an optimization candidate.

Otherwise, we'll just call this an exercise in the boundaries of MrC.

That's also strange, i'm sitting just under 30k files too lol, that part was very quick on my system too..

As for the expression, it never completed on my comp, I had to force close mc - but I do have lots of other stuff running at the same time.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #31 on: January 31, 2013, 08:00:16 pm »

This is definitely pushing MC's expression and search capabilities to their limits, but I've been operating under the assumption that it is Matt who is the real challenge lover.

I'm sure this type of a set operation would be must faster when done internally, but it is pretty esoteric.  I'm taking an educated guess that Matt has some string primitives used to create strings and lists (whch are really just very large strings), and operations on these like we're performing increase substantially as a list gets larger.  Fast insertion/deletion routines that avoid duplication require special algorithms and are probably overkill for almost all uses in MC's expressions.

Since MC works file-by-file, it would have to provide an aggregate list of either:

  - an enumeration of a pane's contents (e.g. all artists from an Artist pane)
  - any given field (e.g. [Artist_all]).

If it could provide either of these lists, then the operation we're performing here might be much snappier.  Since these lists don't exist directly, we've cheated to create and employ them (in a very expensive fashion).
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #32 on: January 31, 2013, 08:10:58 pm »

It does sound like some challenge, I just wish I could be of more assistance in a technical standpoint instead of firing shot in the dark statements and questions here and there. But hey they got this far :)

Quote
I suppose if we knew there were no file variables in the expression, it could evaluate it a single time.

Could an expression know if there are any file variables within it or would it have to be set manually with another mode?

Edit:
Would copying the cheated list into a custom field with the edit type set as large value, then periodically manually refreshing the values help any?? Just thinking out the box a little maybe some sort of global field, that applies to all files but only stores one set of the values instead of for every individual file. The database backups would get huge otherwise.. Those poor sidecar files.. lol
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #33 on: January 31, 2013, 08:24:01 pm »

Could an expression know if there are any file variables within it or would it have to be set manually with another mode?

Matt means that if the Search rule expression we've employed did not reference any properties (aka tags) for a file, it could be optimized.  But you require those, since that's where your Artist and Remixer values come from (each file has a set of fields/properties, and we're using two of them in the expression - this is multiplied by the number of files, hence 30k operations, minimally).
Logged
The opinions I express represent my own folly.

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72444
  • Where did I put my teeth?
Re: Listcombine mode request
« Reply #34 on: January 31, 2013, 08:25:59 pm »

Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42385
  • Shoes gone again!
Re: Listcombine mode request
« Reply #35 on: February 01, 2013, 01:36:58 pm »

In a coming build:
Faster: The ListCombine(...) and ListCount(...) expressions are much faster.

It's hard to quantify how much faster, but it would be interesting to have you list a before and after performance.

I thought about special case handling for constant expressions, but I think it's too fraught with danger (complexity to us, possible bugs for users) relative to the payoff.  The point of an expression is to evaluate something per file, so an expression that's not doing that is somewhat orthogonal to the design goals.
Logged
Matt Ashland, JRiver Media Center

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #36 on: February 01, 2013, 03:11:13 pm »

In a coming build:
Faster: The ListCombine(...) and ListCount(...) expressions are much faster.

so an expression that's not doing that is somewhat orthogonal to the design goals.

completely understand, hopefully the faster expressions this will make it more workable.
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #37 on: February 01, 2013, 04:59:48 pm »

I remember when I first started to use MC and was tying to do this, I did find a way to get all the remixers in a pane with album artists.

Can't quite remember how I did it, but it did require me to have a field where I could enter Yes or No if the remixer, artist or supporting artist had an album in my library... The only problem was that every track has to be assigned this value manually.

I did use these expressions in calculated fields
#List Album Artists & Artists
=listcombine([Album Artist],[Artist],\,;)&datatype=[list]

#List Supporting & Remixers
=listcombine([Supporting Artist],[Remixer],\,;)&datatype=[list]

#Calculate
=listbuild(1,;,[#List Album Artists & Artists],[#List Supporting & Remixers])&datatype=[list]

It might work, it might not, the only blip I can see is multiple, artist, remixers or supporting artist in one field using the ; delimiter.
If so I could potentially use your expression to make a maintenance view where I could occasionally semi-automatically update the Yes/No field if artist, remixer or supporting artist has an album.. Maybe this is something where regex might come in useful?

I'll have a dig around and see if I can find out what I did
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #38 on: February 01, 2013, 05:20:04 pm »

I'll test as per Matt's request later today to see how the old vs. new code runs.  It is not usable currently as I've written it, as the performance hit is far too great.  I also didn't spend any time thinking about how to optimize, so I'll do that too.

A suggestion about your remixers.  Your entire goal seems to be to simply exclude some remixers from your [Artist] / [Remixer] list.  A possibly easier option might be to create a new field called [Remixer Suppress] which is a boolean - you set this field to 1 when you don't want this remixer to be shown in lists.  Then your list in a panes view can simply be:

   [Artist];ifelse(!isequal([Remixer Suppress],1),[Remixer])&datatype=[list]

Now only the non-suppressed remixers will appear.
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #39 on: February 01, 2013, 05:39:55 pm »

That sounds like a great idea, much simpler..

The only problem I can see if a track has Artist A;Artist B in it's remixer field where Artist A has an album but B doesn't.. A rarity so something that I could live with as tracks like these wouldn't add too much clutter, or I could just buy an album by them or more probable just remove Artist B from the remixer field as it is still stored within the name tag anyway. Wouldn't be fully automatic but together with the other expressions in a maintenance view it will be relatively easy to keep on top of. It would be a small sacrifice to pay to get those results. As I said before when I first started using MC I got close to that but manually suppressing the remixer the way I was doing it was completely unworkable.

Thanks again MrC for your handy work, you certainly deserved those steaks :)
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #40 on: February 01, 2013, 05:56:27 pm »

Ok, I have another suggestion then...

If you enter your remixers in some form with an internal delimiter separating those remixers that should be listed vs. those that shouldn't, for example:

   Remixers: Bob,Sally,::,Joe,John

then you can output only remixers that precede the :: pseudo-remixer delimiter sequence.  Place any not-to-be-listed remixers after the ::.  When you want to list them all, just use Replace() to remove the :: sequence when necessary.  So in your tags, you'll see a :: as a remixer, but I suspect you're OK with that.
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #41 on: February 01, 2013, 06:21:13 pm »

Yeah I'm ok with that, that's really clever. Would I need to use an expression to remove the :'s then when building the pane? If so it doesn't seem too hard anyway..

As for the maintenance view.. When using your rules for file display expression when complete to calculate which remixers have albums. Could I use a pre-defined expression in Find and replace? I don't think you can use expressions there can you?
maybe it would just be better making a calculated field that would auto-construct the Non Suppressed remixers,::,Supressed where the calculated field generates the values when the maintenance view is activated and then use Move, Copy Fields
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #42 on: February 01, 2013, 07:29:43 pm »

Here's something to use to split the remixers.  See attached image.

For easy copy/paste:

   regex([Remixer], /#^(.*?)(?=,?::|$)#/, 1)

The says, grab as little as required from the beginning of the line (including nothing if necessary), followed by either ,:: or ::, or end of line.
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #43 on: February 01, 2013, 07:48:39 pm »

Cool thanks, working perfectly..

Can't wait to get this together, up & running to test it out.. Think I'll make a backup or my remixer field in-case I mess things up..
Logged

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #44 on: February 02, 2013, 05:32:56 pm »

Hey MrC just looking at your expression, since the idea now is just to suppress Remixers that don't have albums, would it be better just to use your expression along the lines of this?

I would put this in a custom field called [Album Artist All]
listcombine(listbuild(1,;,[list1]), [list1],;,;,0)&datatype=[list]

This would be the rules for file display for the maintenance view, so I would only experience slowdowns with the [Album Artist All] field when the maintanence view is activated
[=save(listcombine([Artist,0],load(list1),;,;,0),list1)1]=1

Not sure if I edited it correctly your expression correctly, but the idea is just to use your expression to generate a list of all album artists. There I could in my new [Remixer Suppressed] field (Which I will use to help build the views using your regex expression), I could then use some expression to check whether each artist in the Remixer field exists in the [Album Artist All] field..

As for the expression to calculate which remixers belong on which side of the internal delimiter, I could just keep a note of it and enter it in the tag window like so =[expression], I haven't got that far yet figuring out which expression to use here..

I did come up with another solution to this a while back, but I am unsure how outlandish it is, and whether or nor it is feasible. But in my mind it is really simple, the ins and outs may otherwise be complicated.. Imagine a special pane that could that could be identified as special just like the way we define a list (&datatype=[list]). So it would look something like [Album Artist]&datatype=[List]&[SpecialPane].. In my head it would pull all the values from album artist and split them into a list but when you select any album artist (or whichever field you use in the pane), instead of MC selecting that subset of files that contain that value in album artist, MC just passes that text parameter to the search functionality in MC that just searches all fields.

I'm not pushing for it but I am interested if it is possible or if it completely goes against the fundamentals of panes?
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #45 on: February 02, 2013, 09:37:47 pm »

Hey MrC just looking at your expression, since the idea now is just to suppress Remixers that don't have albums, would it be better just to use your expression along the lines of this?

I would put this in a custom field called [Album Artist All]
listcombine(listbuild(1,;,[list1]), [list1],;,;,0)&datatype=[list]

This seems inefficient and effectively a no-op.  First, it is building a list out of list1, but list1 is already a list (e.g. a semicolon-separated list of items).  Second, it combines list1 with list1 and then removes the duplicates.  Hence, nothing left.  So, perhaps there's a typo here.

Quote from: Cassangelo
This would be the rules for file display for the maintenance view, so I would only experience slowdowns with the [Album Artist All] field when the maintanence view is activated
[=save(listcombine([Artist,0],load(list1),;,;,0),list1)1]=1

So you're wanting to combine the list of all Artists with Album Artists, removing dups?

Quote from: Cassangelo
Not sure if I edited it correctly your expression correctly, but the idea is just to use your expression to generate a list of all album artists. There I could in my new [Remixer Suppressed] field (Which I will use to help build the views using your regex expression), I could then use some expression to check whether each artist in the Remixer field exists in the [Album Artist All] field..

There's no way to efficiently do this.  There are no functions that will iterate through a sequence to allow performing some action.  You're wanting this:

    Foreach artist in Remixer;  do {
        test if artist is in [Album Artist All] field
    }

The only way to accomplish it now, is very inefficient.  Combine R w AA, showing only dups.  Combine that dups list with R, and see if the results == R.  Very, very, slow.

Quote from: Cassangelo
As for the expression to calculate which remixers belong on which side of the internal delimiter, I could just keep a note of it and enter it in the tag window like so =[expression], I haven't got that far yet figuring out which expression to use here..

This didn't occur to me.  It does seem a bit of work.  I was thinking you'd exclude when noticed or when you cared to exclude.  To do them all at once might be a chore.

Quote from: Cassangelo
I did come up with another solution to this a while back, but I am unsure how outlandish it is, and whether or nor it is feasible. But in my mind it is really simple, the ins and outs may otherwise be complicated.. Imagine a special pane that could that could be identified as special just like the way we define a list (&datatype=[list]). So it would look something like [Album Artist]&datatype=[List]&[SpecialPane].. In my head it would pull all the values from album artist and split them into a list but when you select any album artist (or whichever field you use in the pane), instead of MC selecting that subset of files that contain that value in album artist, MC just passes that text parameter to the search functionality in MC that just searches all fields.

I'm not pushing for it but I am interested if it is possible or if it completely goes against the fundamentals of panes?

It sounds like you want essentially this feature:

   http://yabb.jriver.com/interact/index.php?topic=77222.0

A comment about the LIST datatype, as I sense some fogginess.

You can for all intents and purposes think of a list in MC as a string that happens to have some repeating character that you think of as a delimiter.  In a pane column, a string value "a; b; c; d" is represented as a single entry:

    a; b; c; d

But when the datatype is a list (either because you coerce it as a list in your expression, or because it is defined that way in a Field definition), MC splits the string apart at semicolons and represents each segment as an item (i.e. multiple entries):

   a
   b
   c
   d

A similar string "a, b, c, d" can also be thought of as a list, but MC won't auto-split on commas, so it would be represented as a single entry no matter how it was used.  If you Replace() commas with semicolons, the the string can be auto-converted to a list in its presentation like above.

So, you generally only need to perform those datatype=[list] casts when you want MC to actually present that data as a list somewhere.  It is unnecessary if you're just combining lists (again, these are just strings) or using other list functions.  For example, this is perfectly fine:

   listsort(a-b-c-d, 0, -)
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #46 on: February 02, 2013, 11:37:58 pm »

Quote
There's no way to efficiently do this.  There are no functions that will iterate through a sequence to allow performing some action.  You're wanting this:

    Foreach artist in Remixer;  do {
        test if artist is in [Album Artist All] field
    }

The only way to accomplish it now, is very inefficient.  Combine R w AA, showing only dups.  Combine that dups list with R, and see if the results == R.  Very, very, slow.

I never got as far as the pane generating a list with your expression, mc crashed first, your rules for file display were really quick though.. So I guess this is a no go.

Quote
This didn't occur to me.  It does seem a bit of work.  I was thinking you'd exclude when noticed or when you cared to exclude.  To do them all at once might be a chore.

Only problem there is the remixer count stands well over 1000 higher than the album artist. That's why I was kind of hoping the above would be semi workable. Then every few months I could run the maintenance view, let it run even though it's inefficient, and then use some sort of expression to mass copy over the values of which remixers should be suppressed.. But now that seems even more unworkable.

Quote
It sounds like you want essentially this feature:
Yeah that's basically then same, the pane could be populated by by any field, and each entry in the list would just be treated as a search string that when clicked on would just be passed to MCs search functions.. (but Album Artist A;B would be treated as different search strings to avoid two artists simultaneously getting searched

Quote
You can for all intents and purposes think of a list in MC as a string that happens to have some repeating character that you think of as a delimiter.  In a pane column, a string value "a; b; c; d" is represented as a single entry:

    a; b; c; d

But when the datatype is a list (either because you coerce it as a list in your expression, or because it is defined that way in a Field definition), MC splits the string apart at semicolons and represents each segment as an item (i.e. multiple entries):

   a
   b
   c
   d

Normally any custom field I use that has delimiters in them are just standard fields and say if for a track there are two remixers I just enter the ; manually, then when using it a pane I use [Remixer]&datatype=[List]

I think I get it, when combining any two standard fields with list combine, &datatype=[List] is assumed when using an expression like list combine in a pane? Does that mean if I change my remixer fields' edit type to List, I don't have to use &datatype=[List], if I use it in pane?

lol this getting mind boggling fast, well it is 5:30am, I'd better get off to bed..

Thanks
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #47 on: February 03, 2013, 05:47:47 pm »

In a coming build:
Faster: The ListCombine(...) and ListCount(...) expressions are much faster.

It's hard to quantify how much faster, but it would be interesting to have you list a before and after performance.

I thought about special case handling for constant expressions, but I think it's too fraught with danger (complexity to us, possible bugs for users) relative to the payoff.  The point of an expression is to evaluate something per file, so an expression that's not doing that is somewhat orthogonal to the design goals.

Nice work.

Here are the results of my testing using the rules for file display and calculated pane column from Reply #25 above.  I tested 18.0.126 vs. 18.0.128.  I used a sample size of 18055 files, and performed 4 test cases @ 3 trials each.  I created different views so that I could quickly select a view and test.  The views were:

 1. File display rules only
 2. Expression column only
 3. File display rules + expression column
 4. View 3 above + ListCount(resultant list) in the File display rules

For test 2, View 1 was selected first to ensure valid pre-calculated global list vars required for View 2.

See attached results image.

It was clear that ListCount() was entirely insignificant.  File display rules-only speedup was about 2.33x.  Expression column-only was about 5.59x.  The view is now semi-usable for a maintenance view.

One problem that occurs when these rules take too long - the delay inhibits view tab pull down menus and right click context menus.  They timeout before MC returns from the calculations and no menus appear.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Listcombine mode request
« Reply #48 on: February 03, 2013, 05:56:57 pm »

@Cassangelo - my tests above show that you can gain some advantage if you're going to use this method for a maintenance view.

You should create two views like I did for tests 1 and 2.  The first view only specifies the necessary Rules for file display for the view plus our listcombine() rules for calculating the lists.  You will select this view just prior to the second view.

The second view will contain only the panes expression column.  Only include Rules for file display that view 1 includes minus our listcombine() calculations.  This works because you don't need to re-calculate the global variables list1 and list2 constantly.  Only prior to using the view, or after changing any Artist (or Album Artist, I'm not sure which you are ultimately using) or Remixer values, do you need to select view 1.

So select view 1 to pre-calculate the lists.  And then select view 2 to see the results.

Btw. you can make a copy of a view by dragging the view to its parent group and selecting Copy.
Logged
The opinions I express represent my own folly.

locust

  • Citizen of the Universe
  • *****
  • Posts: 699
Re: Listcombine mode request
« Reply #49 on: February 04, 2013, 04:19:06 pm »

Thanks glad to hear it is possible :)
 
For my first view it just has the following expression for the rules for file display
[=save(listcombine([Album Artist,0],load(list1),;,;,0),list1)save(listcombine([Remixer,0],load(list2),;,;,0),list2)1]=1
 
Quote
Only include Rules for file display that view 1 includes minus our listcombine() calculations.

I'm not sure what you mean here
 
For the second view I am using the following expression (The expression is in a custom field called [Remixer.calc] so I can use an expression in the tag window to copy results over and is displayed in this view as a column so I can see the results.)
listcombine(ListCombine([list1],[list2],;,;,1), [list1],;,;,0)&datatype=[list]

When I activated the first view and then the second, I found a track where the remixer also has an album.. I previously created a another custom field called [Remixer has album] but upon trying this expression in the tag window for [Remixer has album]  just to test if it will work, it would never output any remixer. The remixer I chose did exist in the [Remixer.calc] field.
=if(isequal([Remixer],[Remixer.Calc],8),[Remixer],)

Any ideas what is going wrong?

As for the separating the remixers with the custom delimiters, I think it will be easier just having the [Remixer has album] just field to store the remixers that do have them and just ignore the rest. Then I can build views with that, I'll still have the original Remixer field anyway.. When there are multiple remixers where one does have an album and others don't, can I use regex to just capture the remixer that does, to use in the tag window??

Another bit of advice please. In this thread I have been talking about remixers just to try & keep it as simple as possible. But once this is working, I'd like to do it for [Artist], [Remixer] and [Supporting Artist]. it is possible just to make a calculated field, to store an expression like this
listbuild(1,;,[Artist],[Remixer],[Supporting Artist])

Then for the rules for file display call upon that calculated field instead of remixer?

Edit p.s.
Thanks for all your help and doing that testing
Logged
Pages: [1] 2   Go Up