INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: ID3v2.3 <=> ID3v2.4 - no forward and backward compatibility with TYER/TDRC, TDRL  (Read 37162 times)

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

I'm new to MC, just installed 19.0.135 and testing around.

Until yet I'm using Tag & Rename (TR) (3.8 RC 1), Keyfinder (1.25), Serato Scratch Live (SSL) (2.5) for tagging (all support ID3v2 in WAV files) and have no problem using the column "year" (id3frame TYER and TDRC) and the column "release time" (TDRL).

But with MC ... I think. ?

At first, the specification of 2.4 says: "TYER - Year: This frame is replaced by the TDRC frame, 'Recording time'".

E.g. the track "ABBA_Summer Night City_(Original Single Mix)_[4549419].wav" itself has been released in the year 1978 - which is stored in TYER or TDRC. The compilation (discogs release id 4549419) has been released in 1984 which will be stored in TDRL.

T&R writes ID3v2.3 and when reading ID3v2.4 before, TDRC will be moved "back" to TYER. So TYER=1978, TDRL=1984 (TR supports writing this v2.4 frame into v2.3). T&R shows in column "year" always the correct value 1978 - from TYER or TRDC.

Keyfinder writes back v2.4, TDRC will be taken from TYER (if reading v2.3). TDRC=1978, TDRL=1984. T&R shows the correct value 1978 in column "year".

SSL writes back v2.4 or v.2.3, depending on what is stored before. In column "year" the correct value 1978 will be also shown, taken from TYER or TDRC.

So whatever I use, the values for TYER/TDRC are always correct and moved forward or backwards between the standard. Unknown tags (TDRL in v2.3) are written back with v2.3. with T&R and SSL.  :)


After the first start of MC I saw the column "Date" which shows me "1984" (TDRL). I've tried to find (right click on the columns) another "date" column which shows me TYER / TDRC - but I wasn't able to.

In Options / Library & Folders / Manage library fields I found the drop-down "show all fields (incl. hidden fields)" (btw: what does "hidden" mean in MC?). I found "Date Recorded" - that's all - but I wasn't able to make it visible in the view - maybe I found out it's a video tag and not an audio tag.

I've updated a value in MC and MC writes back ID3v2.3 when ID3v2.3 was inside. TYER has been set to 1984 from TDRL. The original value of TYER "1978" has been lost (checked the WAV file with hex editor) :-( - of course Tag & Rename shows now 1984 in "Year" and an empty value in "Release time".  >:(

That's completely wrong and against the standard ID3v2.3 <=> ID3v2.4: http://id3.org/id3v2.4.0-changes

During preparing the file for updating a ID3v2.4 file (after Keyfinder has updated), and re-scanned the files from file with the MC library tool, under "Date" is now 1978. Let's say MC shows TDRC before TDRL? (btw: physically TDRC is before TDRL in the WAV file what I've seen with the hex editor).

I've updated a value in MC and MC writes back ID3v2.4 when ID3v2.4 was inside. TDRC is now 1978. 1984 is lost. Finally T&R shows 1978 in "year" and "release time" is empty of course. >:(


So how could this sad situation be fixed and be persuaded to buy JRiver MC?

Maybe it's possible but I didn't find out how-to-do:


Finally:

1. How can I see the values TYER/TDRC and TDRL?
2. How can I get the ID3 frame TKEY (musical key) - didn't find the column.
3. When I update the files with MC in ID3v2.3 or ID3v2.4 not losing any year?

Thanks a lot!

btw. I also found this very useful tag overview: http://wiki.jriver.com/index.php/Tags ...
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I'll reply when I return from a very late lunch.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

Welcome.

For others, Lefisu63 is a user over at the directory Directory Opus forums, and was interested in creating something to pull Discogs information to update fields in media files.  I suggested the nice combination of Media Center and my discogs.pl script.  To quote part of the private message reply I received:

     I've just installed JRiver and look around. Impressive. I don't know how I have worked without it until today. ;-)
  

After the first start of MC I saw the column "Date" which shows me "1984" (TDRL). I've tried to find (right click on the columns) another "date" column which shows me TYER / TDRC - but I wasn't able to.

The best way to figure out any required tag mappings is to select a single file, and open the Tag Action Window, and there, click the very first heading line that shows the format, duration and size.  That will open a physical tag dump window where you can see the raw file tags.

Once you have the exact tag name, you can create a new user field so that this can be imported.  I mentioned in a private reply how you can do this.  If the Library Tools > Update Library (from tags) tool is not importing the value, then it might be that the tag is already being pulled/mapped via the importer, or worst case you can use an expression to pull the raw tag value from the file.  This operation is slow, so you only want to do it when necessary.  You can mass assign the value to your new tag field by selecting your tracks, and in the Tag Action Window, edit the field (or edit the file in the file list column) and enter the expression:

   =tag(TYER)

where TYER is the tag you want pulled.  You can learn more about the expression language here:

   http://wiki.jriver.com/index.php/Expression_Language


In Options / Library & Folders / Manage library fields I found the drop-down "show all fields (incl. hidden fields)" (btw: what does "hidden" mean in MC?). I found "Date Recorded" - that's all - but I wasn't able to make it visible in the view - maybe I found out it's a video tag and not an audio tag.

Search for "hidden" at:

   http://wiki.jriver.com/index.php/Tags#Field_Specifications

I've updated a value in MC and MC writes back ID3v2.3 when ID3v2.3 was inside. TYER has been set to 1984 from TDRL. The original value of TYER "1978" has been lost (checked the WAV file with hex editor) :-( - of course Tag & Rename shows now 1984 in "Year" and an empty value in "Release time".  >:(

That's completely wrong and against the standard ID3v2.3 <=> ID3v2.4: http://id3.org/id3v2.4.0-changes

During preparing the file for updating a ID3v2.4 file (after Keyfinder has updated), and re-scanned the files from file with the MC library tool, under "Date" is now 1978. Let's say MC shows TDRC before TDRL? (btw: physically TDRC is before TDRL in the WAV file what I've seen with the hex editor).

I've updated a value in MC and MC writes back ID3v2.4 when ID3v2.4 was inside. TDRC is now 1978. 1984 is lost. Finally T&R shows 1978 in "year" and "release time" is empty of course. >:(

So how could this sad situation be fixed and be persuaded to buy JRiver MC?

I can't comment on this aspect.  Perhaps someone at JRiver will be able to provide more comment.  Please note, their lead developer Matt has been for a while recovering from a bump on the head.  Maybe its worth searching the forum here for TYER to see any comments on it in the past.  MC has a long history, predating some of these now de-facto standard.  Here are a couple of threads:

   http://yabb.jriver.com/interact/index.php?topic=73518.msg498769#msg498769
   http://yabb.jriver.com/interact/index.php?topic=63985.msg428256#msg428256


1. How can I see the values TYER/TDRC and TDRL?
2. How can I get the ID3 frame TKEY (musical key) - didn't find the column.

I think this should be addressed above.

btw. I also found this very useful tag overview: http://wiki.jriver.com/index.php/Tags ...

The guy who wrote that is a doof.  I'd ignore him. :-)
Logged
The opinions I express represent my own folly.

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

The best way to figure out any required tag mappings is to select a single file, and open the Tag Action Window, and there, click the very first heading line that shows the format, duration and size.  That will open a physical tag dump window where you can see the raw file tags.

Once you have the exact tag name, you can create a new user field so that this can be imported.  I mentioned in a private reply how you can do this.  If the Library Tools > Update Library (from tags) tool is not importing the value, then it might be that the tag is already being pulled/mapped via the importer, or worst case you can use an expression to pull the raw tag value from the file.  This operation is slow, so you only want to do it when necessary.  You can mass assign the value to your new tag field by selecting your tracks, and in the Tag Action Window, edit the field (or edit the file in the file list column) and enter the expression:

   =tag(TYER)

where TYER is the tag you want pulled.

Thanks a lot again MrC,

I tried (and hopefully correct) added the column "Track Release Year" which is based on TYER - but without success. I also reloaded the library from files of course. I tried it 2 times but the value 1978 won't be shown under the column "Track Release Year".

Please see the enclosed screen shot.

What I am doing wrong?
Logged

AndrewFG

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3392

^

Try putting an equals sign in your expression =tag()

Logged
Author of Whitebear Digital Media Renderer Analyser - http://www.whitebear.ch/dmra.htm
Author of Whitebear - http://www.whitebear.ch/mediaserver.htm

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

^

Try putting an equals sign in your expression =tag()



Ok - sorry, did it now. After pressing ok, reloaded the library from files ... column/cell "Track Release Year" is still empty.

Screen shot enclosed.

I also tried another column =tag(TKEY) .. without success - also empty.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

The leading = character is only required to force evaluation of an expression in the edit areas such as the Tag Action Window, in-cell or in-pane editing.  It is the lead-in character that lets MC know the string you entered is not verbatim, but should instead be evaluated and its output used as the value.

In the Calculated data area of a field's definition, MC already knows an expression will be entered, so no need for the = character.

@Lefisu63 - If you can make a file available as a test case, I'll take a look.  Maybe use dropbox or similar, and PM me the link.

In general, it is not a good idea to define a user field to include the Tag() function.  It causes access to the physical files when evaluated, and this will cause performance problems for you.  Instead, if you need to use it, you can use the method mentioned above; that is, by doing an in-cell or Tag Action Window edit of the tag value for the selected files and entering the expression =Tag(xxx).  This pulls the values once, and stores them in your static user field.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

I received your PM and file.  Thanks.

The Tag() function won't allow reading in these Txxx tags that already have a mapping, and the mapping is shown in the Tag Action Window (see attached).

From your PM:

Quote
Note: I still need the values I change in MC in the correct tags TYER (2.3) and TDRC (2.4) and TDRL - otherwise MC does not work as the standard describes it and is not compatible to other software I need: Serato Scratch Live and Recordbox for djing, Keyfinder to find the key and Tag & Rename - and also WAVELAB ... which converts ID3 perfectly when converting them to MP3 (which I do for selected files to have a backup on USB when I'm djing and the laptop does not work - I put the USB into the CD player).

Although the problem has been around a while, I'm hoping someone from JRiver has a chance to comment on this.  I'm bummed that I suggested you give MC a shot, but that it might not workout for you.  I do have a tool that will allow you to read those values into a chosen field in MC (but it can't control how MC maps some tag values)  Still, you might be able to use MC and disable tag writing for the Date and Year fields (see item 9: http://wiki.jriver.com/index.php/Tags#Field_Specifications).

Quote
I don't know what the column "Date" should be - but it is a "data type" and not a "column name". I know it can be customized - but it shouldn't be there as a default. It's like a bad name as "Year" in ID3v2.3 - which has been changed to TDRC in 2.4 - with a better logical name.

The Date column in MC is a flexible date field column.  It can be a year-only value, or a fuller date.  And there are automatically generated fields that both are calculated from this value (e.g. Date (year) aka "Year"), and can be used to store values (e.g. writing to Year sets the year value in Date).  It is generally used for an albums date... and you get to define what that date means.  It is primary used in MC for view grouping and sorting.  The Date field in MC is defined as mentioned in the above link under "Field Data Types".  I don't disagree that "Date" and "Year" (again, this is really just the year components of Date) are vague and suboptimal today.  Keep in mind these fields were probably designed 10 or more years ago back in the Media Jukebox days.  And its a big job to update all the input plug-ins, views, mapping rules, etc.  JRiver has generally been very good at taking on jobs like this when someone has fully mapped out the specs, including how popular software handles the fields, and how values should be read, written, upgraded and downgraded when necessary.  JRiver often takes on no more than one large job like this per release, and I think they are already shorthanded and knee-deep.
Logged
The opinions I express represent my own folly.

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10925

MC doesn't have any alternative Date field that could be mapped differently between TDRC and TDRL - all we have is "Date" and the calculated fields that depend on it like "Year".
Without a stock field to even present these values, I'm not sure what you would suggest we do (except adding a new field)?

From what I can see, MC doesn't write TDRL though, it just writes TDRC for 2.4 and TYER for 2.3 - it only reads TDRL to fill "Date". It also favors TDRC when reading, although when TYER and TDRL are set (and TDRC is not), TDRL is preferred, because its a full "Date" mapping, and TYER is only "Year" (that file would also be out of spec, technically, since you list spec compliance)

PS:
"Date Recorded" is an internal field used by the TV engine to mark its own recordings.
I could remove the filtering of that field if you want, but that doesn't really solve the issue of which field maps to which, and with "Date" being a rather generic field that most users expect to just get filled by whatever data is present in the file, and don't care for the differences.
Logged
~ nevcairiel
~ Author of LAV Filters

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

Hi,

MC doesn't have any alternative Date field that could be mapped differently between TDRC and TDRL

Yes - that is the "big" problem - and finally that makes MC incompatible against other software which tags ID3v2.x because these dates have a different meaning!

Read the ID3v2.4 and ID3v2.3 specifications

http://id3.org/d3v2.3.0
http://id3.org/id3v2.4.0-changes
http://id3.org/id3v2.4.0-frames
http://id3.org/id3v2.4.0-structure

and see these columns:

"Recording time", (2.4: TDRC, 2.3: TYER/TDAT/TIME).
"Release time", 2.4: TDRL, 2.3: not supported
"Original release time", 2.4: TDOR, 2.3: TORY
"Tagging time", 2.4: TDTG, 2.3: not supported

TYER = Year, TDAT = DDMM, TIME = time - of the recording time (see v2.3 spec). Finally TDRC = TYER+TDAT+TIME.

The ID3v2.3 column TRDA is only a complement of TYER/TDAT/TIME and needn't be supported in my opinion.

When MC is reading the files, MC merges the columns TDRC/TYER and TDRL into one column. :-( => Data loss - different meanings! (I didn't check if TDAT+TIME will be used by MC).

If MC should (and it must) be compatible with the standard ID3v2.x and finally with other software, this must be done in MC:

1.

a) The column "Date" should be renamed into "Recording Time", "Recording Date" or "Recording Timestamp" - whatever you prefer by default. But THAT is the correct name and meaning of this column!

b) The columns "Release Time" and "Original release time" must be added to the database.

c) Adding "Tagging time" to the database is optional (for me).

So columns which are defined in the standard needn't be created in custom TXXX frames.


2. An option in the preferences which the customer can choose:

a) When MC is saving ID3v2 in files

  1. save the following ID3v2.x version:

  (single choice):
  x1 always save ID3v2.4 (default)
  x2 always save ID3v2.3 (data can be lost)
  x3 keep the same ID3v2 version which is stored in the file.

  2. Additional options
  o1 when saving ID3v2.3 stream also save ID3v2.4 frames (only available when x2 or x3 selected) (enabled by default)
  o2 when saving ID3v2.3 stream write the full timestamp into year column (TORY).

When ID3v2.4 is written into the file, MC writes TDRC, TDRL, TDOR and TDTG into the file. As ID3v2.4 "Timestamp" data type.

When ID3v2.3 is written into the file, MC writes TYER/TDAT/TIME (for TDRC), TORY (for TDOR). When o1 is also selected, TDRL and TDTG will be also written (but only new ID3v2.4 columns - not replacement-columns like TDRC ...).

When o2 will not be implemented then only save the year from the timestamp. The customer should know that when saving ID3v2.3 that data might be lost!


I would know some more options and recommendations. I can help with the analyzes, etc. - it's my main job as programmer and database designer for 30 years beside djing for 30 years - when you implement it.


Thanks a lot!
Logged

AndrewFG

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3392

^

It might not be so easy as what you suggest. What you say might all be fine on source file formats that use ID3v2 tags. But MC must also compatibly handle those formats that use other tagging schemes e.g. m4a, flac tags etc.
Logged
Author of Whitebear Digital Media Renderer Analyser - http://www.whitebear.ch/dmra.htm
Author of Whitebear - http://www.whitebear.ch/mediaserver.htm

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

@Lefisu63,

Any change proposal needs to address how most users deal with MC.  They simply don't care about a distinction between the album's original release date and any other dates.  They want a Date.

There is nothing wrong with using a simple catch-all Date column label in MC, as it addresses the needs of most users.  You are in a special segment of the user population, that is, one who does care about these matters.  So any proposal that is going to fly must address first the needs of typical users, and second then needs of the specialized population(s).

I'm very confident that the Date field will not go away or be renamed.  It is a legacy field, and a change here will break many 3rd party apps, backup libraries, etc.

The Date field has the capability of encoding both date and time data.
Logged
The opinions I express represent my own folly.

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

It might not be so easy as what you suggest. What you say might all be fine on source file formats that use ID3v2 tags. But MC must also compatibly handle those formats that use other tagging schemes e.g. m4a, flac tags etc.

I know.

You only need the correct logical data fields and map it to the physical fields depending what is supported & possible - and options might have to be prepared in the preferences.

A mapping table from MC's logical data field into the physical columns in the twiki (or does it exist?) will help the user what is possible with the file format the user prefers.

Here is an example page from MP3tag.de: http://help.mp3tag.de/main_tags.html#TCAT

Of course - new columns arrive from time to time. Also ID3v2 does not support all - but TXXX helps to create missing columns. When e.g. ID3v2.5 will arrive with additional columns then everyone who want to swap from v2.4 to v2.5 has to copy his TXXX to the new T???. And maybe store this one value into both tags for some time.

If ID3v2 in WAV wouldn't be possible I would have a problem with WAV because the LIST INFO chunk doesn't support so much columns I want and I would have to find another solution.

They simply don't care about a distinction between the album's original release date and any other dates.  They want a Date.

You are forcing the people to use ONE date column?

What would you say if I would say - "here you have ONE text field - store whatever you want". I am sure you wouldn't be happy to have artist, album name, track title, etc in one text field. ;-)

At first it is a great & big difference if you have an album bought in the 80's on CD or a "remastered" later. Compilations arrive usual later and have a newer release date as the track itself.

The logic of the dates is different:

Recording Time = track based
Release Time = release based (single, album, compilation, etc.)

I know a lot of people from discogs.com over the years who are updating any bad data and correct them and store both date columns in their files as I do:

Track "A Taste Of Honey_Boogie Oogie Oogie_(Original Mix)_[987752].wav" ...

http://www.discogs.com/release/987752

Recording Time = track based => TDRC = 1978
Release Time = release based (single, album, compilation, etc.) = 2007
Original Release Time = release based (single, album, compilation, etc.) = also 2007 in this case because this compilation has not been release before. It's not a reissue.

Finally: If the user want ONE date only then he only needs to use ONE date and not 2 or 3 dates.

I (and others) want 2 or 3 dates as supported by the standard ID3v2.3/4 and when the software (MC) is not able to handle it - then it is the wrong software for me because the minimum requirements for me are not there.

But I am sure - if you have 2 or 3 dates as above - and users are insterested in to have this information and correct data they would fill it in. Currently the MC users can't (except creating own TXXX columns - not always portable to other software).

E.g. I have not bought the car SKODA because their brand new cars do not have an USB to connect - although it is a standard today! I have bought a HYUNDAI i30 instead. The USB was one of my minimum requirements for a car. Others maybe only want a cassette. ;-)

You are in a special segment of the user population, that is, one who does care about these matters.

I am sure there are more people who want 2 or 3 dates instead of one - as people who needs Dynamic Range values stored - and there are 2 (!) columns for it ... only one example.

I'm very confident that the Date field will not go away or be renamed

The physical value shouldn't go away. If MC wants to name it "Date" or changes the name to  it's meaning (this always happens with various things on newer software, etc) - however - MC supports a display name. Ok for me and others. As long as I know how it works and where it is visible in other programs - ok.

It is a legacy field, and a change here will break many 3rd party apps, backup libraries, etc.

I don't think it will break many 3rd party apps. Currently MC breaks the standard (ID3) and a lot of big software which uses the standard correctly. Backup libraries? Don't understand this why it should break my backup? I backup my current files ...

The Date field has the capability of encoding both date and time data.

That's good! I don't want anything else. I will store my 1500 japanese CDs with the full release date from discogs.com (printed on the release).

btw. you might not have noticed it above: ID3v2.4 "TDRC" stores date and time in one column. In ID3v2.3 the column TDRC was splitted into 3 parts: TYER (year), TDAT (day+month), TIME (time).
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.

First off, as I've mentioned, I'm not affiliated with JRiver, so I don't speak for them.

Regarding people being "forced" to use one field - those are you words, not mine.  I claimed that the large majority of users don't care about more than a single date value.  JRiver tries to not over-complicate things, so this is just an area where simplicity and legacy have thus far been the winner.

I get what you want.  And I also get what it takes to get things implemented. And I'm trying to make the case that your chances of getting something implemented increase if and only if you address appropriately both legacy and ease-of-use concerns.

I don't agree with your implication that a majority of users want more than a single date value.

You are absolutely wrong about how changing Date will affect 3rd party apps, and MC itself.  But I'll leave this to your discovery to understand why.
Logged
The opinions I express represent my own folly.

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72413
  • Where did I put my teeth?
Logged

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

Thanks a lot for the help. I'll wait for a reply from JRiver people.

Summary for JRiver people (before you read the postings above):

MC should be extended with 1 standard ID3v2.4 date columns (TDRL) in the database (incl. writing it back to files). The other 2 (TDOR, TDTG) are possible (personally I don't need them).

The initial merge of TDRC and TDRL from the files into one database column "Date" should be removed because data will be lost (in my case currently well tagged 47.000 !! WAV files).

Those who have 2 different dates stored know why and don't want to lose data when starting using MC and to be compatible with other programs which support the ID3v2 standard.

All current customers do not lose any information and are not affected - also not when writing back the information into files of course.

Thanks a lot!
Logged

Lefisu63

  • Junior Woodchuck
  • **
  • Posts: 95

TDRL is preferred, because its a full "Date" mapping, and TYER is only "Year"

Dear Hendrik,

I write this again here - maybe you didn't see it due the amount of text:

in ID3v2.3, TYER+TDAT+TIME represent the ID3v2.4 frame TDRC:

TYER = Year, TDAT = DDMM, TIME = time - of the recording time.

Of course, a lot of older programs only allowed to enter a year (maybe due the compatibility to ID3v1).

TDRL = release time (not the recording time!).

The precision itself is not the problem (TRDC + TRDL are timestamp fields):

The difference is that the recording time is track based, the release time is released based.

E.g. the track "A Taste Of Honey_Boogie Oogie Oogie_(Original Mix)" itself has been recorded in 1978 (=TRDC), the compilation from where I have ripped the track (discogs release id 987752) has been released in 2007 (=TDRL). (Updated to this example because for the ABBA compilation no release date info was stored on discogs.com anymore.)

Beside the problem that MC merges these 2 different meanings into 1 column - MC prefers TDRL over TRDC when reading, but writes it back as TRDC when updating the file (TRDL itself is not written anymore). So the new value of the recording time is now the release time - 2007 in this example and the recording time 1978 is lost.

When I e.g. go djing and play with Serato Scratch Live and want to play 70's ... the smart crates (rule based ... TRDC/year < 1980) wouldn't contain this track anymore.

Please let me know if this will be fixed soon.
Logged
Pages: [1]   Go Up