INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2]   Go Down

Author Topic: Converting SHN files to FLAC files  (Read 40036 times)

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #50 on: March 21, 2013, 05:02:23 pm »

So, let me see if I understand the options:

If you use -f , then the process will internally tag the flac files with the metadata.   So -cf (as in your example) means convert the shn files to flac, and then tag them.

Alternately, you can create an mpl file, and use that to import the metadata to MC18.  So, if you want to not actually convert the shn file, you could create mpl files, import them, and then see all the fields filled out in MC18 for the shn files.

Is that correct?

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #51 on: March 21, 2013, 05:21:03 pm »

No, -f means Force Overwrite (of existing FLAC files).  Only meaningful with -c.

No file tagging is done - that is accomplished entirely via importing the MPL.

Correct about creating an MPL to import, which will set the tags for the corresponding SHN files.  Don't use -c in this case (-c means Convert); only the MPL file is created for the existing files.  SHN's are favored over FLACs when both exist.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #52 on: March 21, 2013, 05:35:16 pm »

Okay, so the -f is just for cases where you rerun the same conversion twice (hopefully won't need that very often).

I'm trying it right now...

ActivePerl installs into a folder that it creates, and then it adds the path of that folder to PATH.   I put the three programs into System32.

However, it is looking like PATH doesn't work for the script, so you have to specify the absolute path if your command window is not run in the folder of the script.   And, in fact, the easiest way to convert a single folder is to use "Run Command Window Here", and then specify the absolute path of the script.

But going back to glynor's original concept - converting all the legacy shn file sets as a one time operation - it looks like we will want a batch file that traverses sub-folders and looks for .shn files and .txt files...

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #53 on: March 21, 2013, 05:41:07 pm »

I picked one folder randomly and here is what happened:

M:\Temp\gd73-02-26.sbd.kaplan.1208.sbeok.shnf>perl.exe c:\perl\parse-etree.pl -c
f  gd1973-02-26.1208.txt
Converting [gd73-02-26d3t01.shn] (25:18.39) --> [gd73-02-26d3t01.flac] : 100% OK

Converting [gd73-02-26d3t02.shn] (17:52.19) --> [gd73-02-26d3t02.flac] : 100% OK

Converting [gd73-02-26d3t03.shn] (7:18.07) --> [gd73-02-26d3t03.flac] : 100% OK
Converting [gd73-02-26d3t04.shn] (3:14.19) --> [gd73-02-26d3t04.flac] : 100% OK
Converting [gd73-02-26d3t05.shn] (6:35.40) --> [gd73-02-26d3t05.flac] : 100% OK
Converting [gd73-02-26d3t06.shn] (7:45.23) --> [gd73-02-26d3t06.flac] : 100% OK
Converting [gd73-02-26d3t07.shn] (3:00.07) --> [gd73-02-26d3t07.flac] : 100% OK
Converting [gd73-02-26d1t08.shn] (4:59.40) --> [gd73-02-26d1t08.flac] : 100% OK
Converting [gd73-02-26d1t09.shn] (7:06.16) --> [gd73-02-26d1t09.flac] : 100% OK
Converting [gd73-02-26d1t10.shn] (6:56.33) --> [gd73-02-26d1t10.flac] : 100% OK
Converting [gd73-02-26d1t11.shn] (3:22.04) --> [gd73-02-26d1t11.flac] : 100% OK
Converting [gd73-02-26d1t12.shn] (7:39.21) --> [gd73-02-26d1t12.flac] : 100% OK
Converting [gd73-02-26d1t13.shn] (4:15.38) --> [gd73-02-26d1t13.flac] : 100% OK
Converting [gd73-02-26d1t14.shn] (5:40.00) --> [gd73-02-26d1t14.flac] : 100% OK

So, it does seven tracks from Disk 3 and then seven tracks from Disk 1.

Here is the .txt:

Grateful Dead
February 26, 1973
Pershing Municipal Auditorium, Lincoln, NB

Source:  S:MR > cassette > DAT > CDR > EAC > SHN
Extraction and SHN encoding by Seth Kaplan
Thanks Seth for this show!

Info file and notes by G.S. Hamilton

Disc 1:
Set 1
1.  The Promised Land              3:05
2.  Loser                          6:47
3.  Jack Straw                      5:03
4.  Don't Ease Me In                3:34
5.  Box Of Rain                    4:53
6.  China Cat Sunflower >           6:47
7.  Jam                            0:42
8.  I Know You Rider                4:59
9.  Looks Like Rain                 7:06
10. Loose Lucy                      6:56
11. Beat It On Down The Line            3:22
12.  Row Jimmy                     7:39
13.  El Paso                       4:15
14.  Big Railroad Blues            5:39
Total time:                                 70:53


Disc 2:
Set 1, cont.
1.  Playing In The Band                     17:59
Set 2
2.  They Love Each Other                     5:38
3.  Big River                                4:23
4.  Tennessee Jed                            7:58
5.  Greatest Story Ever Told                 5:30
Total time:                                 41:30

Disc 3:
Set 2, cont.
1.  Dark Star >                             25:18
2.  Eyes Of The World >                     17:52
3.  Mississippi Half Step                    7:18
4.  Me And My Uncle                          3:14
5.  Not Fade Away >                          6:35
6.  Going Down The Road Feeling Bad >        7:45
7.  Not Fade Away                            3:00
Total time:                                 71:04

Notes:
D3t02 (Eyes): splice @ 3:25

==

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #54 on: March 21, 2013, 06:08:56 pm »

The FLAC files converted from SHN seem to play fine, and the tags look good - except that as shown above, the titles and durations are from the wrong tracks.   (Interesting that duration in the tags is a written text, rather than an audio duration, but that's an unrelated issue.)

I checked LiveShowTagger, and it has the Artist, Date, Venue and Setlist correct - except that it did not know to stop at "Notes:" and it lists " (Eyes):splice " as the title for 3/02, even though it had previously listed 3/02 as "Eyes Of The World >"...

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #55 on: March 21, 2013, 06:28:44 pm »

Okay, so the -f is just for cases where you rerun the same conversion twice (hopefully won't need that very often).

I'm trying it right now...

ActivePerl installs into a folder that it creates, and then it adds the path of that folder to PATH.   I put the three programs into System32.

However, it is looking like PATH doesn't work for the script, so you have to specify the absolute path if your command window is not run in the folder of the script.   And, in fact, the easiest way to convert a single folder is to use "Run Command Window Here", and then specify the absolute path of the script.

But going back to glynor's original concept - converting all the legacy shn file sets as a one time operation - it looks like we will want a batch file that traverses sub-folders and looks for .shn files and .txt files...


Perl won't know where the script is, and perl won't search your path for it, so you have to specify it.  ActivePerl can be invoked by calling a script directly, but I think that requires a simple registry change.  Then, just placing the script in the path makes it executable just like any program.  Cygwin won't have this problem, since you can make the script directly executable.

Regarding converting an entire folder tree, I'd use the *nix find tool to do the recursion - it's a one-liner.

   find samples -name "*.txt" -type f -exec  parse-etree.pl -cf '{}'  \;

This is where *nix tools shine (so the advantage goes to Cygwin here).  (Actually, I did all the development on Mac OS X, and the same script just runs on the various platforms).

The difficulty with the above is that your info files are not a) well named, b) don't all end with .txt, and c) there may be multiple possible info files, so you'll have to know ahead of time how to specify the collection of info files so that find will find them.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #56 on: March 21, 2013, 06:32:32 pm »

The FLAC files converted from SHN seem to play fine, and the tags look good - except that as shown above, the titles and durations are from the wrong tracks.   (Interesting that duration in the tags is a written text, rather than an audio duration, but that's an unrelated issue.)

I checked LiveShowTagger, and it has the Artist, Date, Venue and Setlist correct - except that it did not know to stop at "Notes:" and it lists " (Eyes):splice " as the title for 3/02, even though it had previously listed 3/02 as "Eyes Of The World >"...

Are you saying the script got it wrong?  If so, I'll need to see the info file.

The durations aren't something the script is dealing with.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #57 on: March 21, 2013, 06:41:45 pm »

Are you saying the script got it wrong?  If so, I'll need to see the info file.

The durations aren't something the script is dealing with.
See:

http://yabb.jriver.com/interact/index.php?topic=78216.msg539019#msg539019

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #58 on: March 21, 2013, 06:52:31 pm »

See:

http://yabb.jriver.com/interact/index.php?topic=78216.msg539019#msg539019


I saw that.  Generally, pasted text does not work.  I think I've mentioned this before - it changes the line endings, and whitespace is destroyed.  I need the raw file(s).  The script relies on certain line patterns, and if something doesn't work, I don't want to chase my tail.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #59 on: March 21, 2013, 06:55:18 pm »

I saw that.  Generally, pasted text does not work.  I think I've mentioned this before - it changes the line endings, and whitespace is destroyed.  I need the raw file(s).  The script relies on certain line patterns, and if something doesn't work, I don't want to chase my tail.
Okay, I'm happy to send you the files.... I was just assuming that there might be something about the visible syntax, irregardless of line endings...

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #60 on: March 21, 2013, 07:44:20 pm »

Ok, I see the problem.  I didn't update the code in some sections to deal with multiple discs.  I'm working on a fix now...
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #61 on: March 21, 2013, 10:47:53 pm »

Let's give version 1.1 a try (attached).  It runs against all your sample data.

Changes:
  - Multiple Discs are handled correctly, matching names with filenames, even when files are sparse
  - Automatically switches shntool executable name based on OS
  - Removes some track durations at the end of the track name
  - Miscellaneous bug fixes

Edit: attachment removed - see latest thread.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #62 on: March 21, 2013, 11:13:26 pm »

You should remove the old version (or update the attachment)...

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #63 on: March 21, 2013, 11:26:30 pm »

So far so good on V1.1 run on the same files...

... I forgot to mention that Album came out as "gd1973"... the last mention in the thread was:

" Since I don't know the city with certainty, I can give you the venue portion for Album:

 Live in {venue} November 6, 1967 "

(Of course, gd1973 is not unique.)

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #64 on: March 22, 2013, 04:34:40 am »

I've fixed this and a made venue and date parsing more robust.  I'll have a new version shortly...
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #65 on: March 22, 2013, 12:00:11 pm »

Two Qs on MPL files:

1) If you import an MPL file, and then change the Field Properties within MC18, and then do "Update Tags from Library", will it write the tags to a) the individual FLAC files, b) the MPL file, or c) both ?

2) Can an MPL file list "tracks" that are all within a single FLAC file (as can be done by cue files) ?

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #66 on: March 22, 2013, 12:08:17 pm »

Two Qs on MPL files:

1) If you import an MPL file, and then change the Field Properties within MC18, and then do "Update Tags from Library", will it write the tags to a) the individual FLAC files, b) the MPL file, or c) both ?

2) Can an MPL file list "tracks" that are all within a single FLAC file (as can be done by cue files) ?


1. Individual files, not MPL.

2. I don't know how to do this, and have not tried.  My understanding is the the Filename field is what references the entry in MC.  So I imagine that importing an MPL with multiple records, all the same Filename, will just update the same entry over and over on a last-seen-wins basis.

I believe things like particles and cue entries are virtual in MC, and I don't know of any way to address them via MPL.
Logged
The opinions I express represent my own folly.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #67 on: March 22, 2013, 04:56:17 pm »

Here's version 1.2.

Changes:
  - Assigns Album tag as: Live in <first line of venue>, <formatted date>.  The formatted date can be configured.
  - More robust parsing of venue and dates with stricter filename vs. in-venue date correlation
  - Better debugging
  - Miscellaneous fixes and cleanups


Edit: attachment removed - see latest thread.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #68 on: March 22, 2013, 05:40:17 pm »

Cool.  So what is the option for formatting the date ?

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #69 on: March 22, 2013, 05:52:06 pm »

The formatting templates are described here:

   http://www.cplusplus.com/reference/ctime/strftime/

See the code:

    # format of date shown in Album as part of venue (strftime format)
    my $venue_date_fmt = '%B %d, %Y';

and change the $venue_date_fmt string using the codes from strftime.  Other characters can be inserted too:

    my $venue_date_fmt = 'On the fine %d day of %B in %Y';
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #70 on: March 22, 2013, 06:17:23 pm »

Aha, so you are just suggesting that the user just edit the script (probably many European users will want to reformat the date).   That makes sense, considering that one has to have a little tech savvy to run a commandline program.

Anyway, Version 1.2 worked fine on my test case, the Album, Date, Comment and Description are all great.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #71 on: March 22, 2013, 06:34:55 pm »

For now, that's the way you'll have to edit.  If there is more need, it can be a config file, command line option, or other.  Mileage first, tune-up next.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #72 on: March 22, 2013, 06:38:25 pm »

Okay, just to clarify the use of the MPL file.

Since you mentioned that tag writing in MC18 does not write to the MPL file, this would indicate that it is being used here as an import mechanism (although I do see the Export Playlist command should be able to update the MPL file if desired).

So, to further automate the process, I think a chain of MCC commands need to be figured out, that would write the field information to the tags in the FLAC files (since this is part of the desired outcome).  I do see:

MCC_UPDATE_TAGS_FROM_DB

I guess that a batch file could use MCC to select the files listed in the MPL file, and do the above MCC command.  Is there a better way?

==

Also, assuming there are no bugs reported over the weekend, do you have any problem if I posted a link to this thread in a live show trading Forum or two, next week ?

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #73 on: March 22, 2013, 07:02:46 pm »

Auto-Import has a "Write file tags when analyzing audio, getting cover art and applying folder-based tags".  What we need here is "Write file tags when applying playlist-based tags".

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #74 on: March 22, 2013, 07:45:03 pm »

I have found that just placing an MPL playlist in an auto-import folder that is configured to include the proper audio types and MPL playlists, the files will get automatically imported.  There's nothing you need to do.

Let's hold off on sharing the tool until we've worked out a few more kinks.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #75 on: March 22, 2013, 08:52:50 pm »

I'm not referring to importing, I'm referring to writing the tags into the flac files themselves.   There are some easy workarounds, but it would be nice to have an elegant "solution". ;)

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #76 on: March 22, 2013, 09:41:22 pm »

Here's a CNBC Financial Reporter who will likely be interested in converting SHN files:

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #77 on: March 22, 2013, 10:22:34 pm »

It is curious that an MPL will drag files in, as per auto-import rules, but not perform the configured analysis, etc.

Anyway, it is easy enough to use a smartlist or view that shows files with no tags or analysis info.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #78 on: March 23, 2013, 12:49:50 pm »

Auto-Import has a "Write file tags when analyzing audio, getting cover art and applying folder-based tags".  I am assuming that if one does not analyze audio, get cover art, or apply any folder-based tags, then MC18 will not write file tags when importing.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #79 on: March 24, 2013, 01:30:47 am »

Here's version 1.3, which fixes the problem you mentioned in your email.  It also supports the non-standard track names.

Changes:
  - Fixed error that occurred when found filename date could not be matched against the in-venue date.
  - Support track names of the form [Tt]rack##.shn
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #80 on: September 21, 2013, 09:15:12 pm »

It is curious that an MPL will drag files in, as per auto-import rules, but not perform the configured analysis, etc.

Anyway, it is easy enough to use a smartlist or view that shows files with no tags or analysis info.
Okay, so I ran one of these conversions today, and it a) made FLAC files & b) made an MPL file.

Then I set Import to include MPL files, and then ran an Import, but the process did not apply the fields from the MPL file to the FLAC files.

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Converting SHN files to FLAC files
« Reply #81 on: September 21, 2013, 09:54:03 pm »

I think you have to use File > Import Playlist.
Logged
The opinions I express represent my own folly.

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Re: Converting SHN files to FLAC files
« Reply #82 on: September 21, 2013, 10:39:34 pm »

I think you have to use File > Import Playlist.
Ah, that worked perfectly, thanks much !
Pages: 1 [2]   Go Up