Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[zybex]

I've published ZStats 1.0, a mini-tool to compile statistics from the [Play History] field and create a bunch of user-defined Top Plays playlists with the most played tracks for any given time range (daily/weekly/monthly/custom). It can also write the stats back to a field to be used in views/expressions/smartlists.
 
This tool is intended to be used with the [Play History] tracking field as described here:
https://yabb.jriver.com/interact/index.php/topic,130266.0.html

Please read the tool description and instructions in GitHub:
https://github.com/zybexXL/MCStats#zstats

This tool has no GUI - it's intended to be scheduled to run nightly to update all statistics in the library, though you can also just run it at any time to update immediately.

[zybex]

v1.0.1 published with some bugfixes - looks like I somehow missed the month of October on the month enumeration
https://github.com/zybexXL/MCStats/releases

[lepa]

Thanks zybex for adding another cool toy to the Z-family.

One note to users that first time you might want run UpdateStats=0 and then change created [Play Stats], [Album Stats] and  [Artist Stats] fields not to write to file tags as with big libraries it would take a long time to write tags into files. At least I prefer Stats to be written only to MC library field.

[zybex]

Thanks Lepa!
The default config has UpdateStats=0, and the [Album Stats] and [Artist Stats] sections are also disabled by default (so only the Playlists get created).
Note that the fields will NOT be created when UpdateStats=0, so the best is for the user to manually create those fields before activating UpdateStats. Alternatively, you can abort ZStats shortly after it starts with CTRL+C after it creates the fields (you can safely CTRL+C at any time). You can then fix the field definitions in MC and re-run ZStats to update the statistics.

The description text on the project page mentions the slow-write issue at the bottom. Thanks for pointing out the potential problem.

[JimH]

Thanks, zybex!

[zybex]

v1.0.2 published with a feature requested by Lepa:

Add Pre-History feature:
-- InferPreHistory and PreHistoryField config options
-- [PreHistory] token for number of plays before [Play History] started

This option is to have the tool generate older timestamps when the [Play History] field has fewer entries than [Number Plays]. For instance:
- file was Imported in July 2019
- [Number Plays] is 50
- [Play History] was started in May 2021 and only has 30 timestamps
=> In this scenario, ZStats will generate the missing 20 timestamps, spreading them out between July 2019 and May 2021. Playlists and Stats that cover this period will then include these Plays. This option is enabled by default.

Additionally, the tool can optionally write this auto-generated list of Timestamps back to MC by setting the PreHistoryField option in zstats.ini. This is disabled by default.
You can also use the new [PreHistory] token to output the count of Plays before History started, and to generate a Playlist of Top Pre-History played files.
 
The SampleConfig has been updated with the new options.

[lepa]

Thanks for the preHistory! Also if one wants to have song level real time you can always +1 the needed items int the [Play Stats] using MC's expression after playing.

I got some exceptions on both Album and Artist level stats

Code: [Select]

Updating statistics field [Artist Stats], grouped by [Artist]
An exception occurred during execution! Please report this issue to the developer:
System.NullReferenceException: Object reference not set to an instance of an object.
   at ZStats.Program.<>c__DisplayClass13_1.<UpdateMCStats>b__0(MCFile f)
   at System.Linq.Enumerable.WhereListIterator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at ZStats.Program.UpdateMCStats()
   at ZStats.Program.runStats()
   at ZStats.Program.Main(String[] args)


Code: [Select]

Updating statistics field [Album Stats], grouped by [Filename (Path)]
An exception occurred during execution! Please report this issue to the developer:
System.ArgumentNullException: Value cannot be null.
Parameter name: key
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.ContainsKey(TKey key)
   at ZStats.Program.UpdateMCStats()
   at ZStats.Program.runStats()
   at ZStats.Program.Main(String[] args)

[zybex]

The second exception was caused by case-sensitive field names. Please change "Filename (Path)" to "Filename (path)" in the zstats.ini, or just upgrade to ZStats v1.0.3 which is no longer case-sensitive.

The first one seems similar but I couldn't replicate it. Please let me know if you still see it.
The first one was triggered by empty [Artist] field values and is also fixed in v1.0.3.

[lepa]

I can confirm that it is now a bug free program

[zybex]

I can confirm that it is now a bug free program

AH! Famous last words... there's no such thing!

[zybex]

Published v1.0.4 to improve pre-history timestamp distribution.
https://github.com/zybexXL/MCStats/releases/tag/v1.0.4

[marko]

Hi zybex, I got a short amount of time to mess with this today... long enough to figure some stuff out and get the stats fields and playlists populated. ~140,000 records processed in 6 minutes.

Now... I've not had time to (and am out of time now) digest the data inserted into the stats fields, but have a question regarding this:

Code: [Select]

## History field to read - must be a list containing the play timestamps for the track
## numeric date format and the current PC date format are recognized automatically.
My system date format is "dd/MM/yyyy HH:mm:ss"

However, for sorting and readability, I store(d) the history data as "yyyy/MM/dd HH:mm:ss".
Will ZStats automatically read these dates?
When I set that date format in the ini file at "HistoryFormat", the program would not run, so I took it out, left the rest at default, and then it appears to have done its job, but, has it?

-marko

[zybex]

You found a bug! Looks like I introduced a few when I reworked the Config file in v0.94.

Published ZStats v1.0.5:
- Fix HistoryFormat config option
- Fix MidnightOffset config option

However, for sorting and readability, I store(d) the history data as "yyyy/MM/dd HH:mm:ss".
Will ZStats automatically read these dates?
When I set that date format in the ini file at "HistoryFormat", the program would not run, so I took it out, left the rest at default, and then it appears to have done its job, but, has it?

If it can't parse your dates it will print an error message saying so. So if there's no error, it worked... though I find it strange that it could parse dates in a different format than what you have setup in windows.

Please re-run using the new version where you can setup the correct HistoryFormat, just to be sure.

[lepa]

I guess it is not possible to use multiple fields for grouping? Say [Artist][Name] to get actual numbers for particular song which exist in multiple albums. Could of course make combined field inside MC to achieve this now that i think of it

[zybex]

Could of course make combined field inside MC to achieve this now that i think of it

Indeed, that's why I didn't add that as a feature and decided to keep it simple.

[lepa]

Hi zybex! Do I need to calculate stats or can I run ONLY RunExpressions?

My use case for example is that:
BEFORE: Collect every movie name  (movies I have) to a list
AFTER: Check collected list against recommendations list and if match then enter item to field which tells me that I have that title which is recommended for this movie

(expressions are just sketch ups at this point so there may be errors)

Code: [Select]

####################################################################################################
## ZStats config file
## Instructions: Read all comments in this file to learn about what each setting does
##
## [JRiver] section:
##   - check the MC connection info (MANDATORY)
##   - enable or disable ZStats functionality flags
##   - check if HistoryField matches your MC field name for Play History
##   - change other settings if needed on this section (defaults should be OK for most cases)
## [Stats] sections (if UpdateStats=1):
##   - check the config for each section; you can add more sections if needed
##   - set enabled=1 to activate a section. ZStats will update the [UpdateField] with the statistics
## [Playlists] section (if UpdatePlaylists=1):
##   - add or remove playlist definitions. ZStats will create/update the listed playlists
## [RunBefore] and [RunAfter] sections (if RunExpressions=1):
##   - Enter ExpressionLanguage code to be executed before/after processing the statistics

####################################################################################################
## MC server connection and global settings
####################################################################################################
[JRiver]
MCserver=https://localhost:52200      # alternative syntax
MCuser=user
MCpass=pw
MCFilter=[Media Sub Type]=[Movies]             # filter to limit processing to a given set of files

UpdateStats=0                           # 1 to update stats as defined in [Stats] sections, 0 to disable
UpdatePlaylists=0                       # 1 to enable playlist processing, 0 to disable
RunExpressions=1                        # 1 to execute [RunBefore] and [RunAfter] expression, 0 to disable
CreateFields=0                          # 1 to create missing output fields, 0 to disable (allow only update)

####################################################################################################
## expressions to execute on each file BEFORE statistics and playlists are updated
####################################################################################################
[RunBefore]
SaveAdd(;[Original Title]), v_MoviesList)

####################################################################################################
## expressions to execute on each file AFTER statistics and playlists are updated
####################################################################################################
[RunAfter]
#setField(Stats Updated, formatdate(now(),%c))
ListMix(If(ListContains([v_MoviesList],RemoveRight(ListItem([L1],1,/ /[),1)),SetField(RecInLibrary, [RecInLibrary];RemoveRight(ListItem([L1],1,/ /[),1),),,[Recommended Titles])


[zybex]

You can run only expressions. You may need to leave the other sections blank instead of removing them, not sure.

Though... you can get all your titles with this:

Code: [Select]

fieldquery(media sub type, movies, name, 0, 1)

[lepa]

I re-added [Stats] and [Playlists] sections as blank

Code: [Select]

[Stats]
enabled=0

[Playlists]I still get

Reading file list
  0 files read

Finished in 00:00:02

Though... you can get all your titles with this:

Code: [Select]

fieldquery(media sub type, movies, name, 0, 1)

I though it could be painfully slow to do fieldquery for each file

[zybex]

Try

Code: [Select]

MCFilter=[Media Sub Type]=[Movie]
You have an extra 's' there

Isn't fieldquery() cached? I think it only re-runs every five 60 seconds or something like that.

[lepa]

You have an extra 's' there

 
Thanks

[DangerJP]

Hi,

I really appreciate this plug-in, great work!

On my setup (Windows 10), I can run ZStats.exe on demand without any problems, by manually clicking on the file.

But when I'm trying to use Windows Task Scheduler to run ZStats.exe overnight, but I get the following error message: "Invalid MCWS credentials - please check username and password" in command window.

Any idea how to fix this?

[JimH]

Is MC connected to an MC server?  If so, read about authentication.

[DangerJP]

Is MC connected to an MC server?  If so, read about authentication.

It's on the server itself, not on a client.

[zybex]

Hi DangerJP,

You need to set the current folder in the Task, as it defaults to c:\windows\system32 (a new zstats.ini was likely created there).

Solution 1: set the "Start in (optional):" field in the task to the folder where your zstats.ini is (same as the EXE).
Solution 2: add the zstats.ini path+filename as an argument on "Add arguments (optional):"

[DangerJP]

Hi DangerJP,

You need to set the current folder in the Task, as it defaults to c:\windows\system32 (a new zstats.ini was likely created there).

Solution 1: set the "Start in (optional):" field in the task to the folder where your zstats.ini is (same as the EXE).
Solution 2: add the zstats.ini path+filename as an argument on "Add arguments (optional):"

This dit it
Thanks Zybex!

[EnglishTiger]

Thanks zybex - ZStats is super efficient and I love the level of detail on the progress report. 

The is one thing I would like to see, if possible/easy and that is for it to produce Playlists for Album and Artist stats similar to the ones it creates for Play Stats

[zybex]

Hi ET,
Not sure what the Artist/Album playlists would contain... we can only add tracks to a playlist. So if a given artist is the Top Played artist, would it add ALL tracks from that artist? It doesn't make much sense to me. Can you please elaborate?

[EnglishTiger]

Hi zybex

In lots of ways my request was based on the fact that ZStats provides the additional Album and Artists Stats but doesn't provide any thing, like playlists, that utilises that data.

After playing around with the Playlists created for Play Stats I reckon I could probably feed those Playlists into 1 or more "Most Popular Album" and "Most Popular Artist" views or smartlists.

Before using ZStats I cleared out the  Number Plays, Last Played and Last Played(Album) Fields and then set up the Play History field.
Hopefully this is down to me not fully understanding the data in the Artist Stats and Album Stats fields; in the attached image the highlighted text has values in those 2 fields that look "wrong" to me.

That Track has only been played once, it is the only track that has been played from that album and it is the only track by the artist "Status Quo" that has been played in the period since I installed Zstats

ET

[zybex]

Yeah, the "25" doesn't look right.

ZStats by default groups files by [filename (path)] to calculate the Album stats. This is because at the time I assumed that people would always have a separate folder for each album - a bad assumption which is of course not always true. I guess you have more files/albums in the same folder as that Status Quo track? Or perhaps it's a Compilation album.

Try this - find this line in the ZStats.ini:
GroupByField=Filename (path)

And replace it with:
GroupByField=Artist - Album (Year)

Regarding the artist stats... strange. I would need to see your DB, if you care to share it.

[EnglishTiger]

I always keep all the tracks for every individual album  in the same folder, regardless of how many discs make up the album.

So for Studio Albums, which can include Greatest Hits labelled albums, the filename structure is Artists/Album Artist/Album/disc#-track#-track name
For Multi-Artist Compilation Albums the filename structure is Compilations/Collection Name/Album//disc#-track#-track name
For Classical Music Boxsets the filename structure is Classical/Composer/Album//disc#-track#-track name

Where the "Artists", "Compilations" and "Classical" folder names are designated by me to reflect the type of music that is in those folders, whilst the "Collection Name" is from a field/tag I added to MC to differentiate between the differing sources of Compilations and also the different Collection/Series names from people like "Now That's What I Call Music".

In addition there can be up to 6 discs in each "Compilation" album and up to  160 discs in each "Classical" album and yes I do consider a Boxset to be an Album.

I agree the "25" in that Status Quo track does look wrong, probably even more so when you look at this set of numbers:-

I have 448 Status Quo Track spread across 34 Studio Albums and 17 tracks spread across 16 Compilation Albums.

The smartlist I use to feed Playing Now selects one track that has never been played with a genre that is not Classical from each disk, removes any Artist & Name duplicates and provides a list that is 100 tracks long sorted by Artist.

Since Friday morning  MC on the win pc has probably played about 1,000 tracks and according to the Number Plays field/album that is the only Status Quo track that has been played;
but it has has played more than 1 track by the same Artist and more than 1 track on the same album.

I have once again cleared down the Number Plays, Last Played, Last Played(Album), Play History, Play Stats, Artist Stats and Album Stats fields and changed zstats.ini to not collect/generate the Artists Stats, Album Stats or PreHistory data.

[zybex]

Would you be willing to send me yesterday's backup of your DB, that would still have this data?

[EnglishTiger]

I'll upload it to my Website tomorrow morning and PM you a link that will allow you to download it.

N.B the Album & Artist Starts data contained in the library may not be the same as in the screenie as I'd run ZStats today before I saw that odd looking data.

ET

[zybex]

Thank you for the backup.

It looks like ZStats numbers are correct. This backup from last Sunday has a total of 1460 recorded plays, 23 of which from Status Quo, all with History timestamps. Presumably you played an additional 2 Status Quo tracks before running ZStats for your screenshot.

The highlighted track on the screenshot below has the following stats:
- [Number Plays] = 1
- [Play History] has 1 value (45253 = 23.11.2023)
- [Play Stats] shows 1 total play for this track (first number), matching [Number Plays]
- [Album Stats] shows 1 play for "Piledriver" (first number) - on your screenshot, you have another track played on the Piledriver Album ("Paper Plane"), thus making this stat=2 on your run
- [Artist Stats] shows 23 plays for "Status Quo" (first number) - you presumably played 2 more, raising this to 25 when you ran ZStats.

Note that your [Play History] seems to have data since 19.11.2023, so ZStats is counting all plays since that date. It shows 1460 plays, where your screenshot is already at 1626.

[EnglishTiger]

zybex thanks for all your work - I must have not made sure that I cleared every relevant field when setting the back to never being  played before ruuning ZStata witth the creation/generation of PreHistory enabled.

Now that I'm restarting with all the relevant fields being empty and I've told ZStats not to calculate/generate any PreHistory it looks like I can now use the Album & Artist Stats to create some "Most Popular Artist" and "Most Popular Album" views or smartlists.

[EnglishTiger]

zybex - I've recently encountered one or two problems that were being caused by MC being in Error Free Mode, i.e. Find & Replace not reporting how many changes it had made. So I've been keeping a careful eye on MC to see if I could spot what is switching MC into Error Free Mode despite me not wanting it to happen and unfortunately it looks like running ZStats is causing MC to switch into Error Free Mode.

[zybex]

Confirmed, good find!

This is a bug/change in MC, not ZStats. I've reported it here:
https://yabb.jriver.com/interact/index.php/topic,138725.0.html

This was likely introduced recently as there were MCWS API changes to enable/disable ErrorFree mode. I'm not using those APIs.

[zybex]

Issue fixed in MC 32.0.40.