INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Storing "Date (Release)" Info  (Read 4085 times)

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Storing "Date (Release)" Info
« on: October 26, 2020, 02:09:43 am »

Whenever I move a track or folder I lose the date stored in "Date (Release)". Isn't that a field that gets stored in the music file itself? It says it is in the "Manage Library Fields" menu. Why is it disappearing?
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #1 on: October 26, 2020, 02:43:57 am »

Move a track or folder how?  If you're moving them in Windows, outside of MC, that's a bad idea. Use MC's Rename Move and Copy Files function instead.  Moving files outside of MC is bad because oftentimes the effect is the same as deleting the file from MC and re-importing. And in that case, if the metadata is not in the file, you lose it. Sounds like maybe you have been familiarizing yourself with this effect.

MC needs to be able to keep track of items in its library, and know where they are.  To move them around behind MC's back is rather rude. :)  If you move things using the RMCF tool, you will not lose metadata.

If you've ensured that the option "Save in file tags (when possible)" is checked for [Date (release)] then the other requirement is that you're actually allowing MC to write tags to files.  There is an option "Update tags when file info changes" that must be checked to allow MC to write tags to files.

MC will only write info to the tags when the info changes, or when you do a "Update tags from Library" via a right-click.  If you edited the [Date (release)] field when "Update tags when file info changes" was turned off, then MC never tried to write the data in the first place. Force the write to occur by manually doing it via the right click menu on the files, as I just mentioned.

Finally from an MC perspective, either the file in question must actually support tags (some file formats don't) or you must use sidecar files.  What file format are you having problems with?

And of course, the file must be writable at the filesystem level. If you have the file flagged as Read Only (or the share, if it's on a network share) then the tags cannot be written to the file regardless. That's not an MC problem, it's a problem of the computer administrator.

So I'd say you're losing metadata because at least one of these criteria is not satisfied.  Those are the things you need to check.

So check all these things.  Which do you think is your problem?

I hope this helps...
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #2 on: October 26, 2020, 06:27:30 am »

Look at the bottom of the Tag Editor. It has a section called Tag Dump. That shows the data that is stored in file itself, rather than in the library.
Logged

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #3 on: October 26, 2020, 01:31:10 pm »

The problem occurs for all file types I use: flac, mp3, m4a. I have options "Save in file tags (when possible)" & "Update tags when file info changes" selected. All other tags are saving properly.

Looking at Tag Editor, it has four listings for [Date (release)] listed under ID3v2.3 Tag but the value is a numerical code instead of the date as entered. But I would assume that means it's part of the ID3v2.3 standard and should be able to be stored in the files internal tag.

It's all well and good to just move files within MC (even though that seems more labor intensive, as implemented, then simply cutting and pasting in file explorer, at least to me). But that's not always feasible. I could potentially lose my MC Library database through HD errors/failure and then completely lose any info not stored in my files. Or if I wanted to share a track with a friend or with one of my other devices then that info wouldn't carry over there either. I use the field [Date (release)] mainly for tracks on compilation albums to differentiate the date of release of the compilation and the original release/recording date of the individual track. Tracking down those dates can be labor intensive at times. I'd hate to lose all that work if I'm going to put the effort into it. If I can't rely on the Date tags sticking in the files then I need to find another way to retain that info. But for sorting purposes within MC using [Date (release)] and [Date] is the most ideal, if possible. I've tried to find a comprehensive list of tags stored by file type but that information strangely doesn't seem to be listed anywhere.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #4 on: October 26, 2020, 01:57:46 pm »

It's all well and good to just move files within MC (even though that seems more labor intensive, as implemented, then simply cutting and pasting in file explorer, at least to me). But that's not always feasible. I could potentially lose my MC Library database through HD errors/failure and then completely lose any info not stored in my files.

You're conflating two different things. I didn't say "don't write the info to the files".  You SHOULD write the info to the files.  I said don't move files outside of MC, so that MC doesn't know. As in don't move media folders around using windows explorer.

MC is a database. It needs to know where the files in its library are. Period. You need to accept that. If you persist in moving media files around without using MC to do it, you will have to turn off auto import, or disable the fix broken links setting, or take other steps, that would seem to be above your current level of knowledge, to prevent you damaging your data or library.

It is normal for dates to be stored as a numeric code, instead of as "12/25/1972".

You never answered the question about how you're moving things.  You also never described how you reached the conclusion you're losing data.

If you want effective help, I suggest you explain those things in detail, and describe exactly what you're doing and what you see.
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #5 on: October 26, 2020, 03:14:43 pm »

It is normal for dates to be stored as a numeric code, instead of as "12/25/1972".

The dates are stored as a numerical code, but they should be displayed to the user as actual dates. The code is not useful to most users.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #6 on: October 26, 2020, 03:29:28 pm »

... but they should be displayed to the user as actual dates.
Depends on the app and how you look at it.

In MC, using the tag dump window, if a date field contains a simple value, like 1988, it displays as 1988.  But if it contains a complex date, like 6/1/1988, then it displays as 32295 in the tag dump window, but as 6/1/1988 in the normal edit field.
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #7 on: October 26, 2020, 03:30:45 pm »

Just moving the files outside of MC will not typically change a tag value in the file.  If you do not see the date after you move the file, it was probably never in the file or it is in the file but did not make it into the database after the move. I would focus on checking to see if the dates are actually stored in the files and, if not, figure out why not.  And, if they are in the file and not in the database, then why not. That is why I suggested looking at tag dump. And that is why knowing exactly how you are moving the files is important. If you simply move the files with Explorer, then MC will not typically know they are moved and will report the files as unknown. That is why it is necessary to know if auto import is finding the files or if fix missing links is doing something that is not working for you.

So, look at one of the files were you think the date is missing and see what values are in the database and what values are in the file.  Then look at the values in a file before you move and do whatever file move you are making and check the same values. That should help track down why, after your process, you are not seeing the values you expect.  And, check your auto import settings and your fix broken links settings.
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #8 on: October 26, 2020, 03:50:38 pm »

Depends on the app and how you look at it.

In MC, using the tag dump window, if a date field contains a simple value, like 1988, it displays as 1988.  But if it contains a complex date, like 6/1/1988, then it displays as 32295 in the tag dump window, but as 6/1/1988 in the normal edit field.

I would suggest in tag dump, that MC should display the appropriate date. The conversion is known, so why not do it?  Certainly, floating point numbers are converted from internal format to normal readable format, so why not dates?
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #9 on: October 26, 2020, 03:51:46 pm »

The dates are stored as a numerical code, but they should be displayed to the user as actual dates. The code is not useful to most users.

Unless I'm  wildly mis-remembering, MC uses the same exact date format as Excel. Which makes it convenient when you copy/paste values into Excel (or export CSVs, etc) because they "just work" so long as you change the formatting on the cell to Date in Excel.

It is also a common format, so lots of applications know how to handle it.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #10 on: October 26, 2020, 03:55:46 pm »

Unless I'm  wildly mis-remembering, MC uses the same exact date format as Excel. Which makes it convenient when you copy/paste values into Excel (or export CSVs, etc) because they "just work" so long as you change the formatting on the cell to Date in Excel.

It is also a common format, so lots of applications know how to handle it.

Yes, I agree it is a common format for computer programs. But most users do know know how to convert it to standard form. My point is that displaying the internal code is not very useful for most human beings.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #11 on: October 26, 2020, 04:05:53 pm »

Again, MC displays the code in tag dump, and displays the human readable value in the editable field.

The argument could be made that tag tump isn't useful for most users: they should just use the regular tag fields.

Perhaps the intention with tag dump is to display what's actually in the file, for debugging purposes. And what's in the file is the integer.
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #12 on: October 26, 2020, 04:13:27 pm »

All I am suggesting is that a human readable form would be more usable to most users in the tag dump.  That way they can compare what is in the library and what is in the file. In the current form, the user cannot do that without doing the conversion, which most users do not know how to do.  If, in fact, the library value and the file value were different, now would the average user know?  How is this poster supposed to know?

I understand the current model. I am just suggesting that a change might make the data more useful to the average user and probably even for the sophisticated debugger.

EDIT: The simplest was to convert an internal date to a normal date format is to enter than value into a cell in Excel and then set the format for that cell to the desired date format. The internal format is just the number of days since 1/1/1900, so you need to do all the leap year calculations to do the calculation yourself, remembering that leap year does not come every 4 years.  Better to let Excel do it. To convert an internal format to normal format use the DATEVALUE function as in DATEVALUE("10/26/2020")  For example, that function gives today as 44130 in  internal format.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #13 on: October 26, 2020, 04:21:22 pm »

Ok, well we still need to hear from the OP exactly what he's doing and seeing, and where. The story is a little sparse.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #14 on: October 26, 2020, 07:45:27 pm »

Perhaps the intention with tag dump is to display what's actually in the file, for debugging purposes. And what's in the file is the integer.

That is exactly the purpose of the Tag Dump, as I've always understood it. Otherwise, it would just be a duplicate of the entry in the regular field, and then what would be the point?

I don't have a strong opinion on this particular thing. But... Anyway, that's the way it makes sense to me.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #15 on: October 26, 2020, 08:14:55 pm »

That is exactly the purpose of the Tag Dump, as I've always understood it. Otherwise, it would just be a duplicate of the entry in the regular field, and then what would be the point?

I don't have a strong opinion on this particular thing. But... Anyway, that's the way it makes sense to me.

The point is that the library and the file are different locations and they can have different values. The purpose of the tag dump is to see what is in the file, rather than what is in the library. Who really wants to know the internal date format? Why not put the other tags out in binary and make people do the conversions?  I really do not see any reason someone would want to see the internal format. What would you ever use it for? And, if that is what they really want, then they can do the conversion.  In this case, we are trying to figure out whether the library and the file have the exact same value after some file moving. Wouldn't that be easier with the actual date, rather than the number of days since 1/1/1900 in one place and the converted value in another?
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #16 on: October 26, 2020, 09:53:43 pm »

I don't have a strong opinion on this particular thing. But... Anyway, that's the way it makes sense to me.

I do.

The Tag Dump is exactly what the name says it is; a dump of the text information stored in the tag area in a file. It is exactly for debugging.

If a user wants to see the formatted version, as above, view the field in the Tag Window above the Tag Dump area, or in a View. If the Tag Dump didn't show exactly the text in the file, then users would have to resort to using Hex editors to check all tags. I still do that sometimes, but that would be bad for most users.

ALL metadata editing applications manipulate what they read from file Tags, which is how many MC users have been tripped up, comparing MP3Tag display with MC, and so on.

There is a lot of manipulation between what is stored in tags and what appears in MC, particularly for dates. That manipulation is also different for different file formats and different tag standards, the latter mostly for IDv3 in MP3s.


AlreadyFree, if you want to move files in an Explorer type interface, you should use the MC Explorer function (Left Navigation Tree > Drives & Devices > Explorer) rather than using Windows Explorer, and you should use Drag & Drop. Using Cut and Paste in Windows Explorer is about the best way to lose metadata.

But you are correct, if MC is set to save the tags, then if the file format supports it they should be in the file. When those files are imported into MC, if MC has never seen those files, tags should show up correctly again. If MC has seen the files before, there are some situations where it won't use the tags in the file. What is the hierarchy for metadata on import again Glynor? I know you have documented it in many posts.


PS: Also note that the "DATE" tag is stored in human readable format, even by MC. For some files it is stored in the YEAR field, even though it may hold a specific date in the format DD/MM/YYYY or similar. The "DATE (RELEASE)" tag is stored in numeric format.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #17 on: October 26, 2020, 11:12:24 pm »

The Tag Dump is exactly what the name says it is; a dump of the text information stored in the tag area in a file. It is exactly for debugging.

If a user wants to see the formatted version, as above, view the field in the Tag Window above the Tag Dump area, or in a View. If the Tag Dump didn't show exactly the text in the file, then users would have to resort to using Hex editors to check all tags. I still do that sometimes, but that would be bad for most users.

That's my view.

Formatting the date also loses precision, which is why they're stored numerically with floating point numbers in the first place. You can always convert the raw numeric value to a formatted value (MC's expression language will do it for you, so you don't even have to resort to an online Excel date converter). You can't always convert a formatted date to the original numeric value (if you don't show it to the same level of precision in the display).

In this case, we are trying to figure out whether the library and the file have the exact same value after some file moving. Wouldn't that be easier with the actual date, rather than the number of days since 1/1/1900 in one place and the converted value in another?

Not to me, because you're depending on a lossy formatting conversion. If I was paranoid, and wanted to confirm, I'd want to compare the raw tag values.

It depends what you want it for. The numeric value stored in the Library is not just a count of days since 1/1/1900. First of all, there's the weird thing with 2/29/1900. More importantly though, it also encodes time. The numeric date value is a floating point number with at least 32 bits of precision (Excel's are 64-bit, but I don't know for sure if MC actually stores them in FP64 or not). Formatting that to any "human readable display" value, even including seconds, is throwing away data. (Adding to that, floats are also inherently fiddly when it comes to precision, adding an extra dollop of complexity on top.)

But, as I said, I don't really have a strong opinion on it. Maybe they could make it so it shows the formatted dates when you hoover over the raw value display or something?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #18 on: October 26, 2020, 11:20:07 pm »

What is the hierarchy for metadata on import again Glynor? I know you have documented it in many posts.

There's a lot of "it depends" in there.

For example, if you previously imported the file and then removed it, but didn't also remove it from the Removed Items Database, then MC doesn't actually fully re-import the files from scratch, it "un-deletes" them from the Library. It also depends on if the file is something that gets handled by Carnac (which overrides embedded tags in some cases), so videos are handled differently than audio. The long-and-short of it is: It is complex.

I don't think there's a definitive description of exactly what it does and in what order other than the source code.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #19 on: October 27, 2020, 10:39:25 am »

We are way off the original topic, but I wanted to clarify some of these comments.


It is exactly for debugging
Yes, the question is for by whom. The average user can use it to see if the data in the library and the data in the file are the same, which is what was suggested here.  But since the library value displays as a normal date form and the tag dump displays as a numerical value, the only way to compare this is to convert one or the other. Most users do not know how to do that.
Quote
If a user wants to see the formatted version, as above, view the field in the Tag Window above the Tag Dump area, or in a View.
Viewing the field in the Tag Window does not necessarily show the value that is in the file or in Tag Dump. It will if they are the same value, but if there is a difference between the library field and the file tag, then the Tag Windows display does not show the value in the file.

More importantly though, it also encodes time. The numeric date value is a floating point number with at least 32 bits of precision (Excel's are 64-bit, but I don't know for sure if MC actually stores them in FP64 or not)
The Wiki says that date and time are stored as 32 bit floats, with the integer part as the date and the decimal part as the time.  However, with the 64 bit version, I get 16 digits after the decimal point, so I assume it is now 64 bit float. However, the format for Date (Release) is listed in MC as a Date format, whereas the format for fields like Date Last Opened, which also contains time,  is listed as (unknown). Not sure why. In any case, there should be no round off error in a pure date value, like the field we are discussing here.

I looked at a time + time field for Date Last Opened and it was displayed as 12/9/2019 12:08 pm from the library and, after writing to the file, as 1575911300 in Tag Dump, which is very different from the MC internal value of 43808.5055555555591127.   So you need yet another conversion routine to check what value is in the file.  I am not sure what format that Tag Dump value is. Writing Date (Release) with a date and time in the field to the file gives the same numerical value in Tag Dump and for the converted field value. So that works as expected. Confusing, except one data type is Date and the other is (unknown).

Again, we are way off topic. I just want to suggest that there should be a way to view the normal date + time in the Tag Dump without having to resort to using conversion tools.


EDIT: An expression column containing    formatdate(tag(Date /(Release/)),DateTime)  will show the normal date and time format for the Release Date stored in the file.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #20 on: October 27, 2020, 01:23:10 pm »

Only the first 5 posts in this thread are on topic and relevant to the OP.

Perhaps a mod could split all the rest off to a new thread.
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #21 on: October 27, 2020, 02:32:32 pm »

Unfortunately, when  you split off a discussion like this it completely loses it  context and people have a hard time figuring out why the discussion even happened.  And, in the future if anyone has a similar date problem in a file, the discussion may be useful to them.   I would suggest the OP can skip the posts if they are not relevant.

Sorry for the detour.
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #22 on: October 27, 2020, 06:27:29 pm »

Yes, the question is for by whom.

For the average user, as you suggest.

Information on exactly what happens with tags in a file, with respect to MC or other applications, is a bit hard to find and often not precisely correct. Because there are no real standards for tagging, only software implementations, some of which use agreed formats, mostly. To try to work out what a particular tagging application has done is often difficult. Many users who ask for help on the forum have used some other tagging applications, which they believe is using some standard MC is or should be using, and that their application is correct. That has been proven incorrect time and again.

There are many examples of tag information being manipulated before being presented to users. Date tags are the most obvious.

The Tag Dump is sort of an easy step between knowing almost nothing about tags, and understanding how they work generally. Yes, users need to learn something in order to understand what is going on. But that is true for all tagging software.


PS: I have no idea where that 1575911300 came from, but [Date Last Opened] is an automatically managed field in MC, and all such fields have a Data Type of "Unknown". I've never seen a reason for that. When I look at the raw value of that field in MC it is shown as a Floating Point number with 16 decimal places in MC, 44132.3903703703690553, which is 28/10/2020 9:22 am. But if I write it to a tag in the file I get a similar value to what you saw, 1603837328, in the Tag Dump, which I confirmed with the Hex editor. I can't see any obvious relationship between the numbers. I guess as an automatically managed field it isn't intended to be written to a tag and it uses some other format. Only JRiver could clarify that.

Regardless, the "DATE (RELEASE)" tag is written in the Floating Point format, as noted above, which uses the 30 December 1899 Epoch Date system the same as Excel.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #23 on: October 27, 2020, 06:55:23 pm »

Yeah I didn't know I was getting into anything so controversial. ;D I was essentially trying to find out if [Date (Release)] was stored locally in mp3/m4a/flac files or not. I thought I would get either one of two answers:

"No that field is not part of the ID3 tag standard and is thus not stored internally in the files and must be stored externally in the MC sidecar. So moving the file outside of MC will cause that info to be lost."

or:

"Yes that field is part of the ID3 tag standard and should be getting stored internally within the file. There must be a bug in are storing code for that particular field. Thanks for pointing this out."

Just to limit confusion let's stick to mp3. "Save in file tags (when possible)" is checked. "Update tags when file info changes" is checked. MP3's are not "Read Only".

I had a mp3 with a user visible [Date (Release)] field entry of "1952". It was stored in the Tag Dump as "18994". I copied and pasted the file into another folder in Windows 7 and then doubled clicked it so it would load up in MC so I could check the tags. After being moved the user visible [Date (Release)] field entry is now blank but the tag dump [Date (Release)] field entry is still "18994". So if "18994" corresponds to "1952" then it looks like the date is being stored in the file but is not then being read properly by MC, if it's being read as a new file. Otherwise "18994" must just be a dummy entry of some kind and the entry I type into the [Date (Release)] field is being stored in the sidecar, which of course can't follow the file once it's moved from the library. But if that were true, why would the Tag Dump show that the file has an internal [Date (Release)] field with a dummy entry that can't be written to?
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #24 on: October 27, 2020, 07:56:01 pm »

I just tested this out.  Something is broken with MP3 importing.

The tag is preserved and imported correctly if the file is FLAC. If the file is MP3, the tag is present in the file but not imported by MC.
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #25 on: October 27, 2020, 08:01:18 pm »

Not so controversial, but maybe a little.  ;D

Good to stick to discussion on MP3s for the moment, as they do use the ID3 tag system. I have been looking at flac files which use the Vorbis Comment tag system.

Anyway, you are correct. Something is broken for MP3s.

The first thing I noticed, and confirmed, was the MC was writing multiple copies of the "TXXX (Date (release)):" tag to a test MP3 file in the ID3v2.3 section when I edited the [Date (Released)] field in MC, or ran the "Update tags (from library)" function. It fact, just running the "Update tags (from library)" function repeatedly resulted in the same date being written to the file multiple times, and editing the [Date (Released)] field to a new value added yet another tag. Using a Hex editor I also confirmed that all of these duplicate tags were being written to the file. So far, there has been no limit to the number of tags written, and they are maintained through a MC restart. I'm up to 15 "TXXX (Date (release)):" tags so far!

Deleting the value in the [Date (Released)] field in MC didn't delete any of the tags in the file, even when I ran the "Update tags (from library)" function. There is no way within MC to remove these additional tags.

When I copied that file to a new location and imported it, MC did not pick up the value in the "TXXX (Date (release)):" tags and put it into the [Date (Released)] field, which it should have done, and is what you are reporting.
Suspecting that the multiple  "TXXX (Date (release)):" tags may be causing MC to import none, since it wouldn't know which was correct, I took a clean file that had no value in the "TXXX (Date (release)):" tag or the [Date (Released)] field and added a value to the [Date (Released)] field, which added a "TXXX (Date (release)):" tag, then copied it to a new location and imported it. MC did not pick up the value and put it into the [Date (Released)] field, but the single "TXXX (Date (release)):" tag remained in place.


@JRiver, this seems broken to me.

I never really noticed this because I generally don't use the [Date (Released)] field. Looking at some files though, this seems to have been the situation for some time. Hopefully this can get fixed for you AlreadyFree.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #26 on: October 27, 2020, 08:12:16 pm »

It sounds like you are on your way to getting to the bottom of the problem. It sounds like the data in the file has not changed, but for some reason when moving the file, MC has lost track of the value.     MC does can do some weird things when it thinks files are deleted. Will and Rod are experts at figuring these types of things out.

To see the date stored in the file, go into a standard view, click on an album and right click on the display header. It should give you an option to Create an Expression field. Do that and enter  formatdate(tag(Date /(Release/)),DateTime)  That will create a column that shows you the date that is stored in the tag in the file.   And, yes, 18992 is 1952.

And don't worry about the discussion above. I was just suggesting that the tag shows the actual date rather than having to do this expression field. But, as always with MC, there is a way to see whatever you want to see. No big deal.

EDIT :Will and Rod responded when I was typing. Sounds like you are getting the help you need. Good luck.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #27 on: October 27, 2020, 08:22:27 pm »

"No that field is not part of the ID3 tag standard and is thus not stored internally in the files and must be stored externally in the MC sidecar. So moving the file outside of MC will cause that info to be lost."

or:

"Yes that field is part of the ID3 tag standard and should be getting stored internally within the file. There must be a bug in are storing code for that particular field. Thanks for pointing this out."

One thing your examples point out one thing that you may not understand about the core functionality of MC: It never uses the in-file tags (or sidecar metadata) after importing the files.

Never.

Media Center is a database (they call the "database" the "Library"). The Library is all that matters during regular operation of MC. You can set any field (at least the vast majority of them) to also save the data out to tags in the files, but in general once you've imported the tracks, the only times this would be used is if you then import the files into a different copy of MC (or, at least, a different Library).

That's simplifying a bit because if you set Auto-Import up to do so, you can have it "detect" external changes to the files and update the Library to correspond. But the general idea holds: The Library is the source of "truth" and MC always stores the data it uses there, and then sometimes, optionally, syncs this data to the tags (or sidecars, which are the same thing for "untaggable file types"). But those tags are really only used for "interchange" with other applications.

Note: That's also why you don't want to move files outside of MC if you can avoid it, because then the database in MC is left with a "broken" file (that points to no file on disk anymore) and when it imports the file in the new location, it will (in some cases at least) re-import it from scratch. Auto-Import does try to "detect" moved files and "fix" them (if you have this option enabled), but this only works if you leave them relatively unchanged (moving whole directories and not changing the file names inside the directory). Anything it can't auto-match, it will remove the old broken link and re-import found files as "new".

You may have understood that, maybe not, but it was worth making clear.

The other thing is about the ID3 tag standard. MC will store ANY field into tags (for supported file types) if you tell it to, including custom fields. So, even if you make up your own custom fields with names you created, you can have these saved to the ID3 tags in a MP3 file (or the comparable tags in APE or FLAC files). It doesn't matter if the field is one of the original, predefined tags supported in the ID3 spec.

Of course, custom tags like that are basically only going to be usable in another copy of MC, because no other program will know about them. But, it'll still save them if you tell it to save to tags when you define the custom field.

Moving on, Dates are pretty weird. MC uses the floating point number internally to store date values. ID3 (and, really, any text-based tag system) doesn't handle floats. So, the way dates and times, at least for many of the standard fields, are defined in the ID3v2.4 spec is to use two separate tags (one for the date and one for the time). The Time format for the stored strings in the tags is specified in this document: https://id3.org/id3v2.4.0-structure

I'm honestly not sure how MC maps and stores its various date-related fields in the ID3 tags. I know it can interchange Last Played and other similar "standard" fields with other applications (like iTunes), but I've never bothered to look at how they're actually stored in the files. And, it sounds like, from above, maybe the special Date (Release) field is broken in some way at how it does map it to the in-file tags.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #28 on: October 27, 2020, 08:23:54 pm »

That copied file that you imported, and said the [Date (release)] field is blank: do an "Update library from tags" on it and see what happens.

Doing that didn't make any changes to the [Date (Release)] field. Although the folder I moved the file to is outside of my folders set to "Auto-Import" and I have "Automatically import files when played" unchecked so using "Update library from tags" shouldn't have any effect on a file that's not in the library. So I think that's functioning as it should. Still, if the [Date (Release)] is stored in the tag MC should be able to read it from a brand new file upon playback just like it does with all the other fields (name, artist, album, etc) that are stored in the file.

I did try moving the file back within my Auto-Import folder, but not back into it's original folder that I moved it from, and once the file was auto-imported the [Date (Release)] field was correctly repopulated. I think MC just recognized the filename and compared that to it's library (which must have still contained a reference for it even though it had been removed). That's actually impressive and I wouldn't have expected that. But it still isn't reading the [Date (Release)] field from the file so if that file had never been in the library MC wouldn't have populated that field.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #29 on: October 27, 2020, 08:25:44 pm »

I see that Rod was typing his much longer post while I did my nice concise post. :)

I'm fairly certain it's an import problem.  MC claims that when importing, it will read file tags where the tag name is spelled exactly the same as a library field.  It obviously is in this case, because MC just wrote it to the file itself. And yet, MC does not parse the value.  But as I said, it does with FLAC.  I think the mp3 issue is a bug.  Like Rod, I don't normally use this field; I don't normally use MP3 either.

The reason why MC only seems to sometimes lose the data is that MC caches metadata for files deleted from the database.  When you surreptitiously move files (naughty naughty!) if you have fix broken links turned on, and MC can match the moved file as the old one, it reuses the cached metadata.  But if it can't make the match, then it considers it a NEW file, imports it, does not use cached metadata, and then bungles the import of your tag.

Update: To clarify what glynor said about "never": MC doesn't use file tags AFTER import (except if you tell it to) but it DOES read them DURING import.  At least it's supposed to. :)
Logged

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #30 on: October 27, 2020, 08:39:02 pm »

The first thing I noticed, and confirmed, was the MC was writing multiple copies of the "TXXX (Date (release)):" tag to a test MP3 file in the ID3v2.3 section when I edited the [Date (Released)] field in MC, or ran the "Update tags (from library)" function. It fact, just running the "Update tags (from library)" function repeatedly resulted in the same date being written to the file multiple times, and editing the [Date (Released)] field to a new value added yet another tag. Using a Hex editor I also confirmed that all of these duplicate tags were being written to the file. So far, there has been no limit to the number of tags written, and they are maintained through a MC restart. I'm up to 15 "TXXX (Date (release)):" tags so far!

I noticed that I had multiple "TXXX (Date (release)):" fields listed in Tag Dump too but I wasn't familiar enough with it to know that wasn't just the way it was supposed to be, for whatever reason.

But it's awesome that we all finally got on the same page and the problem is being looked into. That means I can continue tagging files the way I've been doing it without worrying about losing info, since I know it's really being stored in the file. And I'm sure the bug in MC will be worked out now that the root of the problem has been found.

It's also good to know that custom fields are stored in the file. I also thought custom fields were always stored in the sidecar, that's why I've never used them. But's that's good to know! Thanks for all the help and quick responses!

Update Although looking now at the Tag Dump for m4a & flac files, it doesn't appear that they support internally storing the [Date (release)] field. So I might have to figure out an alternative after all for non-mp3's
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3125
Re: Storing "Date (Release)" Info
« Reply #31 on: October 27, 2020, 09:18:40 pm »



Update Although looking now at the Tag Dump for m4a & flac files, it doesn't appear that they support internally storing the [Date (release)] field. So I might have to figure out an alternative after all for non-mp3's

I wrote (Update Tags from Library)  Date (Release) data to flac files successfully and it is stored in the standard days since 1/1/1900 (Epoch) format.  Tag dump shows the correct numerical value of the tag. Note sure why you are not able to write to the flac files.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #32 on: October 27, 2020, 09:27:55 pm »

Update Although looking now at the Tag Dump for m4a & flac files, it doesn't appear that they support internally storing the [Date (release)] field. So I might have to figure out an alternative after all for non-mp3's

FLAC definitely works.  I just tested it for my prior post.

I just tested this out.  Something is broken with MP3 importing.

The tag is preserved and imported correctly if the file is FLAC. If the file is MP3, the tag is present in the file but not imported by MC.
Logged

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #33 on: October 27, 2020, 09:42:15 pm »

OK. Yeah you're right. Both flac and m4a work. I needed to select a different file after inputing the info into the [Date (release)] field and then reselect the track I'd updated for the new info to show up in Tag Dump. Very cool.
Logged

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #34 on: October 27, 2020, 10:04:23 pm »

I've mentioned this in the bug report thread, so hopefully it will get a response.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Storing "Date (Release)" Info
« Reply #35 on: October 27, 2020, 10:59:26 pm »

Update: To clarify what glynor said about "never": MC doesn't use file tags AFTER import (except if you tell it to) but it DOES read them DURING import.  At least it's supposed to. :)

For sure. I tried to make that clear, but perhaps I wasn't. It reads tags at import and maps them to the Library Fields, and Auto-Import can, optionally, scan them and update the relevant Library fields if changes are made externally, but otherwise MC doesn't care about the in-file tags at all.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #36 on: October 27, 2020, 11:15:30 pm »

I see that Rod was typing his much longer post while I did my nice concise post. :)

It was my turn.  ;)

Plus I found what I think is a bug, so I wanted to document it properly for the developers. Hopefully, someone will look at the issue.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #37 on: October 27, 2020, 11:23:29 pm »

Except that I had already found that bug, and didn't need an entire page of text to announce it.  ;D
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #38 on: October 27, 2020, 11:40:23 pm »

I expanded the issue to include writing the tag to the file multiple times.  :P
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

wer

  • Citizen of the Universe
  • *****
  • Posts: 2640
Re: Storing "Date (Release)" Info
« Reply #39 on: October 27, 2020, 11:42:46 pm »

Pffft.
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #40 on: October 28, 2020, 01:21:01 am »

 ;D ;D ;D
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42441
  • Shoes gone again!
Re: Storing "Date (Release)" Info
« Reply #41 on: October 28, 2020, 09:40:48 am »

Next build will have this fix:
Fixed: The Date (released) MP3 tag could get written over and over.

Thanks everyone.
Logged
Matt Ashland, JRiver Media Center

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #42 on: October 30, 2020, 06:27:48 pm »

Your wish was Matt's command, and from MC27.0.28 MC should read and write tags to MP3 files correctly. You will want to test that though, because I've not tested importing new files with values in the tags, or moving files outside MC and re-importing them.

Also note, very importantly, that Matt has changed the tag that is written to the file, to conform to the ID3v2.3/2.4 standard.

[Date (release)] is now written to the correct "TDRL (Date (release))" tag, rather than the old custom "TXXX (Date (release))" tag. I've not attempted to work through the implications of that, but it should mean that new files imported into MC that have a date in the "TDRL (Date (release))" tag will have that date loaded into the MC [Date (release)] field. Hence, AlreadyFree, your original issue will be fixed.

Test.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #43 on: November 21, 2020, 12:06:55 am »

I'd been patiently waiting for MC27.0.28 to come out and just realized that it had already been out in the beta forum release. So I installed it and after updating the [Date (release)] field in mp3's they do now indeed maintain the value even after being moved. However I assumed that updating the field would also erase the multiple, multiple erroneous entries for that field that had been stored in the files by previous versions of MC, but they are all still there. Is there any way to erase these from the file?
Logged

RoderickGI

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 8186
Re: Storing "Date (Release)" Info
« Reply #44 on: November 21, 2020, 02:58:34 am »

I didn't find any way using MC AlreadyFree.

Probably the easiest way, since they are MP3 file, is just to use the free MP3Tag. I haven't checked, but it can probably do it.

EDIT: Yes, MP3Tag can remove the TXXX tags. Just open the Extended Tag dialogue (Alt+T) and delete the tag with incorrect values in there. You may want to check the "TDRL (Date (release))" tag in MC after doing that, and possibly run "Update tags (from Library)" on the files.
Logged
What specific version of MC you are running:MC27.0.27 @ Oct 27, 2020 and updating regularly Jim!                        MC Release Notes: https://wiki.jriver.com/index.php/Release_Notes
What OS(s) and Version you are running:     Windows 10 Pro 64bit Version 2004 (OS Build 19041.572).
The JRMark score of the PC with an issue:    JRMark (version 26.0.52 64 bit): 3419
Important relevant info about your environment:     
  Using the HTPC as a MC Server & a Workstation as a MC Client plus some DLNA clients.
  Running JRiver for Android, JRemote2, Gizmo, & MO 4Media on a Sony Xperia XZ Premium Android 9.
  Playing video out to a Sony 65" TV connected via HDMI, playing digital audio out via motherboard sound card, PCIe TV tuner

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9162
Re: Storing "Date (Release)" Info
« Reply #45 on: November 21, 2020, 04:52:59 am »

In MC, I think your only option is to use the "remove tags" command in library tools, followed by an update tags from library, in a kind of sledgehammer meets nut way 😀

AlreadyFree

  • Junior Woodchuck
  • **
  • Posts: 56
Re: Storing "Date (Release)" Info
« Reply #46 on: November 21, 2020, 07:34:53 pm »

Yep both are kinda clunky solutions but they got the job done. Thanks!
Logged
Pages: [1]   Go Up