INTERACT FORUM
More => Old Versions => JRiver Media Center 18 for Windows => Topic started by: Fred1 on April 15, 2013, 07:43:34 am
-
I have a special problem, that could probably mastered by a little Perl Script.
I ripped a bunch of CDs lately, that i would like to tag with data from All Music Guide.
If you select the CD titles with the mouse and copy them to the clipboard and later drop them unformatted to a text editor, you get something like this:
.
.
.
2 <- Track #
Walk in My Shadows <- Name
<- Composer
Joe Bonamassa <- Performer (Album Artist)
3:27 <- Duration
<- Link 1
<- Link 2
<- Link 3
<- Empty Line
3 <- Track #
A New Day Yesterday<- Name
J. Anderson <- Composer
Joe Bonamassa <- Performer (Album Artist)
4:45 <- Duration
<- Link 1
<- Link 2
<- Link 3
<- Empty Line
4 <- Track #
.
.
.
I would like to have it in that format, however ...
1 - Cradle Rock - Rory Gallagher - Joe Bonamassa
2 - Walk in My Shadows - - Joe Bonamassa
3 - A New Day Yesterday - J. Anderson - Joe Bonamassa
.
.
.
... to copy it via AppleScript (with MC Mac) into the MC database.
The number of empty lines may vary due to the links for the actual CD.
An AppleScript can call a Perl Script and use the result.
It would be nice to have such a transformation.
I assume MrC could be the right man for this :-).
-
This isn't the best way to do this. If one is going to write a script to pull the data, using the clipboard involves extra steps (select, copy, paste clip data into a file, parse the less-than-ideally structured file contents, etc...).
The better method is to connect to the URL and grab the key segments of the data, which is well-defined, and easy to do (esp. with the great Perl modules that parse HTML / XML data).
If I were to want this for myself, I'd pull the data from the URL, grab the important segments, and output that data as an MC MPL file that MC can consume directly.
-
Hi MrC,
i'm sure, there are many better ways to solve my little "problem".
There is only one disadvantage with them: they demand a steep learning curve for me, because i'm not familiar with any of the methods you mentioned.
I thought, it would be a 2-liner Perl Script to parse this structure - but apparently i'm wrong.
A little example of an AppleScript calling a Perl Script, to demonstrate the principle:
set str to do shell script "perl -e '
use Data::Dumper;
print Dumper(\\%ENV);
'"
display dialog str buttons ["Ok"] default button 1
The clipboard -> AppleScript -> Perl Script -> AppleScript -> MC method has advantages, too, and it runs in memory only.
You can select one or two lines (titles) that are missing for you and paste them into MC with a keyboard shortcut very quickly.
(http://foto.arcor-online.net/palb/alben/08/1895808/6165323862633661.jpg)
Nevertheless, thank you for your attention to my problem.
-
The script as you asked for it is easy. I'll post one tomorrow, but will use Tab characters rather than dashes to separate the fields so that there is no need to escape any of these from fields that contain a dash.
-
That's great, MrC!
Which separators are used is not the problem.
My problem is: i don't speak Perl - but i should learn.
It is a really elegant tool for parsing text.
Thank you.
-
Which browser are you using to copy the AMG data? I'm finding that the formatting is different that you mention, and each browser copies the data differently.
-
I'm using Safari on a Mac.
But the comments (<- Track #, <- Name etc.) are not originally in the data.
Looks like this:
1
Cradle Rock
Rory Gallagher
Joe Bonamassa
3:50
2
Walk in My Shadows
Joe Bonamassa
3:27
3
A New Day Yesterday
J. Anderson
Joe Bonamassa
4:45
Pasting it here has removed the empty lines.
-
Copy and Paste isn't going to work. The fields are not consistent when copied, as missing fields are collapsed so it is impossible to determine which line belongs to which field.
The data will have to be parsed from the HTML.
Here's an example, where ^I are tabs and $ means end of line:
Sample ^I^ITitle/Composer ^IPerformer ^ITime$
^I1 ^I$
Walk a Fine Line$
Paul Anka / Michael McDonald$
^I$
Paul Anka$
feat. George Benson, Michael McDonald, Frank Sinatra ^I5:32$
^I2 ^I$
Find My Way Back to Your Heart$
Paul Anka / Michael Thompson$
^I$
Paul Anka$
feat. Tita Hutchison ^I4:39$
^I3 ^I$
Do I Love You (Yes, In Every Way)$
Paul Anka / Alain Joseph Yves Le Govic / Michel Albert Louis Pelay / Maxime Piolot$
^I$
Paul Anka$
feat. Dolly Parton ^I4:53$
^I4 ^I$
This Is It$
Paul Anka / Michael Jackson$
^I$
Paul Anka$
feat. Michael Jackson ^I
Taken from: http://www.allmusic.com/album/duets-mw0002487037 (http://www.allmusic.com/album/duets-mw0002487037)
-
Here's a script to try. Unzip it, and locate it somewhere convenient.
Call it as follows:
perl amg.pl URL
where URL is the AMG page URL you want parsed. It will output tab-separated fields of:
Track # Name Composer Artist Featuring
$ perl amg.pl http://www.allmusic.com/album/a-new-day-yesterday-mw0000104226
1 Cradle Rock Rory Gallagher Joe Bonamassa
2 Walk in My Shadows Joe Bonamassa
3 A New Day Yesterday J. Anderson Joe Bonamassa
4 I Know Where I Belong Joe Bonamassa Joe Bonamassa
5 Miss You, Hate You Joe Bonamassa Joe Bonamassa
6 Nuthin' I Wouldn't Do (For a Woman Like You) Al Kooper Joe Bonamassa
7 Colour and Shape Joe Bonamassa Joe Bonamassa
8 Headaches to Heartbreaks Joe Bonamassa Joe Bonamassa
9 Trouble Waiting Joe Bonamassa; Stephanie Tyrell; Steve Tyrell Joe Bonamassa
10 If Heartaches Were Nickels W. Hayes Joe Bonamassa
11 Current Situation Joe Bonamassa Joe Bonamassa
12 Don't Burn Down That Bridge Joe Bonamassa
13 Miss You, Hate You Joe Bonamassa Joe Bonamassa
$ perl amg.pl http://www.allmusic.com/album/duets-mw0002487037
1 Walk a Fine Line Paul Anka; Michael McDonald Paul Anka George Benson, Michael McDonald, Frank Sinatra
2 Find My Way Back to Your Heart Paul Anka; Michael Thompson Paul Anka Tita Hutchison
3 Do I Love You (Yes, In Every Way) Paul Anka; Alain Joseph Yves Le Govic; Michel Albert Louis Pelay; Maxime Piolot Paul Anka Dolly Parton
4 This Is It Paul Anka; Michael Jackson Paul Anka Michael Jackson
5 I Really Miss You Paul Anka Paul Anka Leon Russell
6 Think I'm in Love Again Paul Anka; Bob McDill Paul Anka Gloria Estefan
7 Pennies from Heaven Johnny Burke; Arthur Johnston Paul Anka Michael BublC)
8 Crazy Willie Nelson Paul Anka Willie Nelson
9 You Are My Destiny Paul Anka Paul Anka Patti LaBelle
10 Les Filles de Paris Paul Anka Paul Anka Chris Botti
11 It's Hard to Say Goodbye Paul Anka; Mark Spiro; Jack White Paul Anka Celine Dion
12 She's a Lady Paul Anka Paul Anka Tom Jones
13 Hold Me 'til the Morning Comes Paul Anka; David Foster Paul Anka Peter Cetera
14 My Way Paul Anka; Claude FranC'ois; Jacques Revaux; Gilles Thibault Paul Anka Frank Sinatra
Its quick and dirty and might need some tune-ups.
-
Wow, it works like a charm!
Thank you very, very much for your work.
The handling is very easy.
You can drag 'n' drop the URL to the open terminal window and press Return to let the script run.
I will have a hard time to fully understand what your script does exactly - but i will try.
I've purchased a Perl book yesterday and begin to study.
-
Great news.
If you make the script executable (and possibly place it somewhere in your path), you don't have to call perl directly:
chmod a+x amg.pl
Execute it using any of the following (depending upon its location and your path):
./amg.pl URL
path/to/amg.pl URL
amg.pl URL
-
FYI. I've rewritten this script to be more comprehensive. It works like this:
1. You define a user field in MC called Allmusic URL, and enter the AMG URL for the album into the field for all the tracks.
2. You select all the tracks for the album (these must contain the AMG URL) and Ctrl-C to copy. This places the track data on the clipboard.
3. You run the script, capturing the output into an MPL file.
4. You import the MPL file (File > Import Playlist) and the AMG data is imported. Currently it imports Name, Composer, Artist, Date, Genre, Styles, Moods and Themes. Styles, Moods and Themes need to be defined (and should be list fields) in MC to be imported.
The script works by pulling the track info from the clipboard, matching the track # from your list with the track # from AMG. This is required, because to import the items and associate them with your existing MC entries, the Filename field must be grabbed and used. Track # seems like the most reliable method to match tracks on the clipboard with pulled AMG tracks. Obviously the Filename and Track # fields must be visible in the view.
Let me know if you want to test it out. I'll provide more error and sanity checking later.
btw. I tried to post the data directly to the clipboard in an attempt to coerce MC to allow Paste Tags to work. It didn't work - MC doesn't enable Paste Tags. I'd have to load the clipboard with MC's special clipboard tokens, and I can't do that easily. So, its the old MC MPL import method.
-
Let me know if you want to test it out. I'll provide more error and sanity checking later.
btw. I tried to post the data directly to the clipboard in an attempt to coerce MC to allow Paste Tags to work. It didn't work - MC doesn't enable Paste Tags. I'd have to load the clipboard with MC's special clipboard tokens, and I can't do that easily. So, its the old MC MPL import method.
OMG I missed this. Leaving on vacation for 10 days but would LOVE to try this Mr. C. Great idea/work guys 8)
Sure do wish we could paste tags for multiple files/multiple fields, btw I use chrome ;)
-
This. Looks amazing.
Unfortunately I have no knowledge of Perl... how easy would it be to get this usable assuming zero knowledge of Perl? :)
-
That's a great idea, MrC,
i'm looking forward to it!
btw. I tried to post the data directly to the clipboard in an attempt to coerce MC to allow Paste Tags to work. It didn't work - MC doesn't enable Paste Tags. I'd have to load the clipboard with MC's special clipboard tokens, and I can't do that easily. So, its the old MC MPL import method.
This is an effect, i experienced in the Mac version already.
You can't paste from the clipboard to MC from an other application.
I tried Keyboard Maestro, Quicksilver and some others to do the job to no avail.
MC seems to be the only application, that doesn't accept this. Strange.
Perhaps a side effect from the self written libraries?
-
At least one person wants to also capture the Credits page as well. Anyone else think this is useful?
@darichman: No knowledge of Perl is required.
@Fred1: This works on a Mac too.
@Arindelle: Have a good vacation.
-
At least one person wants to also capture the Credits page as well. Anyone else think this is useful?
Yes, this is highly appreciated!
-
At least one person wants to also capture the Credits page as well. Anyone else think this is useful?
Yes, I would love to have that data. But how is this going to be handled in a sensible way in a non relational db like MC? That issue is what has been preventing me from adding this data (using another method that I am more familiar with),
-
One question. The AMG Rating values are 0 to 9, which allow representation of half-star increments. I can cut those values in half, and round up or down. Or would you prefer the actual AMG ratings? Or both? How should I handle this?
Yes, I would love to have that data. But how is this going to be handled in a sensible way in a non relational db like MC? That issue is what has been preventing me from adding this data (using another method that I am more familiar with),
I'm just making the data available in an MPL. For relational fields, I believe MC will use one value (even though it is repeated verbatim / track in the MPL). So if you define AMG Review to be an album-relational field, there's only 1 value for the album. So I suppose if you're going to use the data, the fields should be per-album relational.
-
So here are the results (see attached).
-
I need to add a little code to handle multi-disc albums. I'll do that shortly.
Just as soon as I get your feedback on how you want the AMG Rating to be handled (see questions above), I'll post the script and instructions.
-
One question. The AMG Rating values are 0 to 9, which allow representation of half-star increments. I can cut those values in half, and round up or down. Or would you prefer the actual AMG ratings? Or both? How should I handle this?
I'd rather have the ratings 0 to 9 if possible.
Would adding a soloist field be possible? - very nice for jazz -- lot of info I grab manually from the "credits" tab (What/where is the "People" field being populated with?)
What happens when you pull from an album like this http://www.allmusic.com/album/high-fidelity-original-soundtrack-mw0000605666 (http://www.allmusic.com/album/high-fidelity-original-soundtrack-mw0000605666) do they use an "AlbumArtist" tag like Various Artists?
-
One question. The AMG Rating values are 0 to 9, which allow representation of half-star increments. I can cut those values in half, and round up or down. Or would you prefer the actual AMG ratings? Or both? How should I handle this?
0 to 9 would be great.
-
OK, so the 0 to 9 rating seems good (and easy).
I'm grabbing all the Credits as they are. There is nothing track-related. The credits (People) look like this:
David Angell [Violin]; Kai Blankenberg [Mastering]; BOY [Composer, Primary Artist]; ...
The Artist field is the value set from a scraped artist. MC's Album Artist (auto) features will override these (if set). Example tracks from the URL above:
$ perl amg.pl | egrep '"Track #"|"Artist"|"Composer"'
<Field Name="Track #">2</Field>
<Field Name="Composer">Ray Davies</Field>
<Field Name="Artist">The Kinks</Field>
<Field Name="Track #">13</Field>
<Field Name="Composer">Neil Michael Hagerty;Jennifer Herrema</Field>
<Field Name="Artist">Royal Trux</Field>
<Field Name="Track #">9</Field>
<Field Name="Composer">Elvis Costello;Clive Langer</Field>
<Field Name="Artist">Elvis Costello & the Attractions</Field>
<Field Name="Track #">7</Field>
<Field Name="Composer">Sheila Nicholls</Field>
<Field Name="Artist">Sheila Nicholls</Field>
When an MPL is imported, all fields with values overwrite existing values. So the program is going to have to know which fields you want written. There are a several ways I can think of doing this:
1. Populate only the fields that are in the clipboard (i.e. your view)
2. Populate all fields except those specifically excluded via some special MC field
3. Populate all fields except those specifically excluded via command line argument
4. Ignore any fields pulled from the clipboard that have a value (i.e. fill empty cells)
5. Overwrite all cells
Thoughts?
-
Dumb question?
How to run that script :)
Im a noob when come to this type of things.
And thanks MrC for the time spent on this :P
-
I'm just making the data available in an MPL. For relational fields, I believe MC will use one value (even though it is repeated verbatim / track in the MPL). So if you define AMG Review to be an album-relational field, there's only 1 value for the album. So I suppose if you're going to use the data, the fields should be per-album relational.
How would I in MC in a structured way handle the relationship that guy X played Y on album Z by album artist Q? I could not figure that out. That is what I meant.
Before I settled for MC I looked at other music organisation software and some had a relational db that wold handle this very well, but MC was already then superior in its file tag reading and writing capabilities and that determined my choice. I do not regret that at all, but if you have a clever workaround I would be interested. (Perhaps I am spoiled by the Squeezebox MySql and then SqlLite db although I could never handle the queries in a professional manner of course.)
-
I don't know of any way to do that. Sorry.
-
Ok, here we go...
[edit: see next post for a new version and changes]
I'm sure there are errors below, and I'll correct them as you report them.
Attached is the script to scrape some data for the selected album tracks. It works like this:
1. You supply an album's AMG URL either through an MC field of your choosing (default: AMG URL), or when you run the script.
2. You select all the tracks for an album and copy them to the clipboard (Ctrl-C). This places the track data on the clipboard which will be used by the script. The two key components are Track # and Filename, so those columns must be available in the file list.
3. You run the script, capturing the output into an MPL file. If you have not defined an AMG URL field and filled it with a valid value, you can pass the URL on the command line.
4. You import the MPL file (File > Import Playlist) and the AMG data is imported. Below is a list of the values available, and the default MC field names (which you can change to import to other fields). Any fields not defined will be silently ignored (not imported).
The script works by pulling the track info from the clipboard, matching the Track # from your selected file list with the Track # from AMG. Valid Track #'s are required, because to import the items and associate them with your existing MC entries, the track's Filename field must be used. Track # seems like the most reliable method to match tracks on the clipboard with pulled AMG tracks. Again, the Filename and Track # fields must be visible in the view.
Below is the list mapping the available AMG data to MC fields. These are located at the top of the amg.pl script. Search for %MCFieldMappings. You may change the names in the MC Field Name column to match your data. Some already match MC's default field names. Other fields you must create, or MC will not import data for this field. Review the comments in the script to learn more.
| AMG data | --> | MC Field Name |
| url | --> | AMG URL |
| rating | --> | AMG Rating |
| pick | --> | AMG Track Pick |
| styles | --> | Styles |
| moods | --> | Moods |
| themes | --> | Themes |
| name | --> | Name |
| composer | --> | Composer |
| artist | --> | Artist |
| credits | --> | People |
| date | --> | Date |
| album | --> | Album |
| genre | --> | Genre |
| ratingmc | --> | Rating |
| tracknum | --> | Track # |
Currently any item defined in the %MCFieldMappings table will be imported if the field exists in MC, and will overwrite any existing values. If you do not want certain fields to be imported, you can comment out the line in the table with a # character. This field will then be omitted from the MPL file. Later I wlll add additional ways to disable certain fields (either by using the view's columns, or via command line).
So let's get started with
Installation and Setup:
1. Download and install ActivePerl for your version of Windows:
http://www.activestate.com/activeperl/downloads
2. Download the attached amg_1.0.zip file and unzip it. Place it somewhere convenient, like your Desktop. Later you can move it once you become familiar with how to use it.
3. Open a Windows command shell (Start and type cmd in the Search box hit Enter) to install a required perl addon module, and enter:
ppm install Win32-Clipboard
Keep this command shell open for later use.
4. Define any fields from above that don't already exist in MC if you want those fields imported. If you want to change the field names, edit the amg.pl script and read the instructions at the top of and within the %MCFieldMappings table. Take care to leave the quotes in place. And don't rename 'Track #'. You'll probably want to define a user field in MC called AMG URL so that the AMG URL will be retained for later use (make it Relational: Store one value for each album).
Usage:
1. In your browser, go to allmusic.com and search for and navigate to the album's page. It will have the track listing, ratings, etc. Copy this final URL.
2. In your still open Command Shell window, type:
cd Desktop
and hit Enter. Now partially enter the next command - don't hit Enter yet:
perl amg.pl -u URL > import.mpl
replacing URL with the allmusic URL you copied in step 1. You can paste the URL you copied by right-clicking in the Command Window anywhere, and use your left and right arrows to move around the command line.
3. Now go to MC, and select an album's tracks (you might want to use a scratch album for testing purposes). Make sure the Filename and Track # columns are shown in the file list view.
4. Edit > Copy (or Ctrl-C on Windows, ⌘C on a Mac) to place the track data on the clipboard.
5. Now you can hit Enter in the Command Window to run the script. It will output the results into the file import.mpl on your Desktop. If there was an error, correct the problems as indicated by the error message.
6. In MC, File > Import Playlist and select the import.mpl file.
If all was successful, the album data should now be updated with the AMG data.
In Step 2, you passed the URL on the command line. You can instead enter the URL into your AMG URL field in MC, and the script can use that instead. This is useful if you want to re-scrape the data in the future (or to use as the target of an MC link). The command would then just be:
perl amg.pl > import.mpl
Current Limitations:
- Works with only a single album at a time
- Diacritics aren't functional yet
- Doesn't yet handle multi-disc entries
Finally, I'm probably not going to extend the script to allow for scraping more than a single album at a time. I think the reasons should be obvious. Also note, as I was testing the script, AMG was changing their HTML, so the script might break at any time. Watch for any error messages.
-
[edit: see post 42 for the latest version and changes]
Here's a new version 1.1:
Changes:
- Fixed: problem with pulling credits
- New: supports Unicode
- New: handles multi-disc albums
- New: new include or exclude options to include or exclude a list of fields via command line
- New: new sequence option to ignore Track # and Disc #, uses the track order in the view
- New: Long or short options are supported (for example, -e or --exclude)
- Change: credits are now mapped to AMG Credits (create your own AMG Field, type List, Album relational)
- Change: output file goes to "import.mpl" on the Desktop by default, or use -o to specify an output file name
- Improved: error detection, bad clipboard data, parsing problems, track-mismatches, etc.
Usage Examples
Note the new syntax - redirection into a mpl file is no longer required (or supported). By default, import.mpl will be created on the Desktop, or use the -o option to specify an alternate file.
perl amg.pl
Runs using the AMG URL value from the copied tracks; outputs to import.mpl on the Destkop.
perl amg.pl -o my.mpl -u http://www.allmusic.com/album/revolver-mw0000189174
Runs using the specified URL, and outputs to my.mpl in the current directory.
perl amg.pl -e name,genre,moods
Runs like the first example, but excludes the Name, Genre, and Moods metadata.
perl amg.pl -i credits
Runs like the first example, but includes only credits (excludes all other metadata). The -i option includes ONLY the specified items.
perl amg.pl -s
Runs like the first example, but ignores Track # and instead uses the sequence order of the file list. This is useful for special issues, for example, AMG lists 12 tracks across 2 discs on the main page, but 13 tracks on the supplementary releases page.
It is a good idea to configure album-related fields such as Moods, AMG Rating, AMG Credits, etc. as Album-relational. This way, any change to a single track in an album affects all tracks. This is very useful when the number of track listed on AMG does not match your album (maybe due to special pressings). By selecting and copying just one track to the clipboard, you can update the album-relational metadata for all tracks.
-
Thanks for your work, MrC!
When i come home tonight, i will try your script.
-
Looks great Mr C. Great! That you thought about commenting out certain fields, too.
Can't wait to give it a go when I get back from vacation.
Thanks much, very considerate!
-
Thanks.
You no longer have to comment out fields. Just use the --exclude or --include options to pick the pieces you don't want, or just want.
-
Question: Do i have to install additional libraries for Mac usage (for clipboard handling for example, as in Windows)?
-
Mac::Pasteboard
You can use macports.
-
Thx MrC for the time u put in this to help me and others, what a nice gentleman :)
-
Anything for a guy named "sirgeo" - it just rolls off the tongue. I feel like having some sangria and paella now.
-
I completed adding a new feature which I'll release with 1.2. It is a new option, --fileregex which takes a regular expression as an argument, and matches it against the filename. The regular expression should match the track number portion of the file name. This is useful if you have no tags, but the files contain the track numbers, such as "1.mp3", "2.mp3", ... "10.mp3". As long as your tracks are numbered, you can grab all track data by calling the script with an AMG url and a regular expression pattern:
perl amg.pl --fileregex '^(\d+)' --url <amg url>
So now there are three ways to match your tracks against the AMG track list:
1. Track # column values
2. Sequence order with --sequence
3. Regular expression captured track numbers from the file's name
-
AMG changed the way they are displaying their mood and theme data so my old way of manually copying the data is no longer working well. Would it be possible to change your method to collect (only) Style, Theme, Mood and AMG staff rating data?
-
It already does this. Try:
perl amg.pl -i moods,styles,themes,rating
You might need to change the field names to match yours, and either supply the URL or have the correct URL in the AMG URL field.
-
Thanks!
I gave it a try. First, the script did not see the AMG URL field (it was in a visible column in the view I copied from). When I included the url in the command line I got a lot of errors and an empty mpl. I attach a file with the errors that the cmd window did not eat up.
Is there any chance of making the procedure easier to use, like having a file with the command line you use over and over to just double click while the track data are in the clipboard?
An additional issue I ran into was that the AMG page for my 2000 remaster of a 1970 album did not contain the mood data that the page for the 1970 album had, and the remaster of course had bonus tracks.
-
I worried about your case. I'm guessing its due to different locales and your use of commas and periods in data.
Can you send me the output of the view.
Select the files in question, and use File > Export Playlist. Select:
Format: Text File (CSV delimited)
Included Fields: All Fields
Output Range: Selection
Save it to the Desktop, zip it, and attach it in a reply here.
-
Do not put too much effort into this.
Here is the file.
-
[edit: see post 47 for the latest version and changes]
Attached is version 1.2.
Changes:
- Fixed: problem handling missing Composer
- Fixed: problem pulling Review section due to AMG html changes
- New: option --fileregex added, which allows pulling track numbers from filenames. See thread above.
@vagskal - besides the fixes above, the main AMG page for the album you attached above describes the original release album, not yours which is a later, enhanced release including additional tracks. So you'll have to use the -s option (sequence) to force using the main page despite track number mismatches. Also, you have Disc # assigned to 1 for this single-disc album, and amg.pl currently assumes this would be for a multi-disc entry. So the -s option also ignores the disc # data, and instead just uses the tracks sequence from amg.pl. The command you would need is:
perl amg.pl -s -i moods,themes,styles,rating
Alternatively, you could also use the new --fileregex option to grab the track numbers and ignore the disc #:
perl -d amg.pl -f '^(\d+)' -i moods,themes,styles,rating
Either way, this will only create MPL entries for the tracks listed in AMG. The additional tracks won't be updated. I suppose I could make an option to force updating data for all tracks when -s or -f is used, or just always assume this. Your thoughts?
As to making the process easier, certainly you can make a Windows batch file to do this. To just double-click an ActivePerl script and have it run requires updating some registry values, and I didn't want to get that aspect in the instructions above. But, yes, this can be done.
The missing component is that there is no way with MC to ask it to import an MPL via command line! You have to use the UI, so I've already put in a request to get a command line option to do this. With that in place, you could then just double-click some script to have it pull pieces and do the import. Still, since the parsing of data is fragile, there will be times when running it via command line will be required to diagnose when something fails.
See if the new script above, with the required options, works for you now.
-
Thanks!
I still could not get the script to recognize my AMG URL field.
Using the -u method and the first of your proposed command lines, data were collected with some error messages but they were not connected to the file copied to the clipboard so nothing was imported. I commented out the disc field in your script (and other fields but not the tracknum field).
-
Can you provide me with one more thing? Do the clipboard copy, and paste it into a Notepad document, and send that my way. You can reply via email directly.
-
Would it be useful for anyone to have an option to populate non-track-specific values from AMG when there are more tracks in MC than the main AMG entry?
For example, some main entries have, say 10 tracks, but your edition of the album might have 14. Currently, non-track-specific values (e.g. Genre, Moods, Review, etc.) won't be pushed out when you have more tracks than does AMG.
-
Would it be useful for anyone to have an option to populate non-track-specific values from AMG when there are more tracks in MC than the main AMG entry?
Yes, I think I can handle such an option (which could be very destructive if by mistake tracks from different albums were selected).
-
[edit: see post 48 for the latest version and changes]
Attached is Version 1.3
Changes:
- New: option --clipfile <file> allows using a file as a clipboard (for testing).
- New: ability to send MPL to STDOUT (use - as the output file name with --outfile).
- New: option --all allows populating non-track-specific values when selection has more tracks than AMG entry. Useful for special album releases.
- Change: add space after semicolons in list items.
-
[edit: see post 52 for the latest version and changes]
Attached is Version 1.4.
Changes:
- New: option --append will append to a possibly existing MPL file
- New: option --help provides usage / options text
- New: added descriptions for each option for help/usage text
- Change: --all option shortcut letter is now -A (was -a).
-
Thank you, MrC! It works very well now.
-
having a bit of a problem running perl .. I downloaded the 64bit version -- would this pose any issues?
-
Nope, not if you have a 64-bit Windows. I have the 64-bit version too. What's the issue?
-
[edit: see post 60 for the latest version and changes]
Attached is Version 1.5.
Changes:
- New: option --rating allows selecting format of AMG rating (d=0-9, f=0.0-5.0)
- New: localized numeric output for rating value (i.e. when --rating=f)
- Fix: problem when using --excludes
If you want ratings to range from 0.0 to 5.0 (i.e. includes half points), use --rating f. Otherwise, --rating d is the default.
-
I am getting these error messages now:
*** Use of uninitialized value $tracklisting in substitution (s///) at amg.pl li
ne 234.
*** Use of uninitialized value $tracklisting in split at amg.pl line 235.
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 280.
Tryck ned valfri tangent för att fortsätta...
http://www.allmusic.com/album/dig-thy-savage-soul-mw0002557546
Is there anything that can be done?
-
What is the full command line you are using (so I can replicate your usage)?
That page only includes a review, so you'll need to specify one of --fileregex or --sequence.
-
It is a .bat file with this, as you taught me:
@echo off
perl amg.pl perl amg.pl -s -i moods,themes,styles,credits,review,rating -A --append --rating f
pause
-
Ok, this is a case I hadn't encountered before. There are no tracks listed on the main page. I'll have to work around this. Be back later...
-
I sent you a private update in email. Let me know how it works out.
-
[edit: see post 60 for the latest version and changes]
Attached is Version 1.6.
Changes:
- Fix: resolved problem with options being case-insensitve
- Fix: handle entries where no track listings exist, to allow grabbing review, etc. only
-
Thanks! All is good.
-
[edit: see post 62 for the latest version and changes]
Attached is Version 1.7:
Changes:
- Fix: support fields w/embedded newlines (e.g. lyrics)
- New: output command line w/debug enabled
-
MC 19.0.36 changed the format of the data on the clipboard. I've updated the script to accommodate this, but want to wait for tonight's build as there may be one additional change. If anyone needs it, let me know.
-
[edit: see post 71 for the latest version and changes]
Attached is Version 1.8:
Changes:
- New: support MC 19.0.36's RFC 4180 clipboard text
Note: this version requires installing an additional perl module (used to handle the new RFC 4180 format of the clipboard). Install it from a Windows command shell by entering:
ppm install Text-CSV
Version 1.8 will auto-detect the format of the clipboard, so it is safe to use with any MC version.
-
Thanks. It is working great.
I suppose there is no way to display in a pane the persons in the AMG Credits without their role on the respective albums, so that I can get all albums credited to Artist X irrespective of if he is playing guitar on one album and harmonica on another (until we get global RegEx search)?
-
Would you like an option to not include their roles?
-
Would you like an option to not include their roles?
Thanks for the offer, but I do not think so. It is of course of importance if an artist plays guitar or harmonica on an album, and I like to gather all metadata available and not cripple what is there. I would like the ability to not _display_ their role in a pane, but I guess this is impossible until a global RegEx option is introduced.
-
You could replace the " [" with \ and remove the ] characters to create a hierarchical list (effectively hiding the roles):
removecharacters(replace([AMG Credits], / /[, \), /])&datatype=[list]
We could also create a separate mapping of AMG Credits (no roles), so you could get both.
-
Thanks for the tip. I forgot about that. I do not use nested lists in panes because I need scroll by typing to work reliably there (which it is not).
Maybe a second custom field ([AMG Credits (only name)]) with just the artist name for the pane would be ideal for the time being.
-
I've sent you a private release.
-
[edit: see post 71 for the latest version and changes]
Attached is Version 1.9:
Changes:
- Fix: problem with RE used to detect and split rows of track data.
- New: include new field mapping: credits_nr ==> "AMG Credits (only name)" which is the same as 'credits',
sans the per-person bracketed credits.
- New: Awards page parsing. Keys 'awards_bba', 'awards_bbs', and 'awards_gra' map to
Fields: "Billboard Albums", "Billboard Singles", and "Grammy Awards", respectively.
Use "-e awards_bba,awards_bbs,awards_gra" if you don't want these fields (avoids extra page pulling).
- New: Releases page parsing. Key 'releases' maps to Field: "Releases".
Use "-e releases" if you don't want this field (avoids extra page pulling).
- Fix: some reviews don't have author attributions.
- Fix: handle existing HTML entities better in Credits.
- Change: add space after list delimiter in Credits.
- Change: -c is now -C (to specify a clipfile)
- New: program will read some values from a config file specified with -c or --configfile. See the sample
config file that accompanies this release.
- New: Credits mapping to selectively map credits to other MC fields, or ignore certain credits. See the
sample config file.
Thanks vagskal for your testing.
I'm considering adding an option to use MCWS to update the tracks, so that the manual MPL import won't be necessary. The code is working, but it makes a lot of calls as each field, for each track, requires a connection, and I was hoping to use a method that does bulk updates for a given track. If this would be of interest, chime in.
Also, I've written code to pull data from Discogs and may incorporate this as well.
-
Although I've been way too busy lately to test this, just wanted to let you know, Mr. C, that there are more of us out there that appreciate your effort on this
Thanks much for taking so much time on this script ...
-
[edit: see post 80 for the latest version and changes]
Attached is Version 1.9.1:
Changes:
- Fix: the same track data was being output for each track
- Fix: remove extraneous newlines / line feeds from clipboard data
-
I have a little problem with the script
"C:\Users\...\Desktop>perl amg.pl -u http://www.allmusic.com/album/once-i-was-a
n-eagle-mw0002524684 > import.mpl
Failed to open \Users\Spiva\Desktop\import.mpl: Permission denied
Stopped at amg.pl line 795."
Any idea?
-
With the current version, you don't need to do the redirection. Just use:
perl amg.pl -u http://www.allmusic.com/album/once-i-was-an-eagle-mw0002524684
The file will be created for you. Otherwise, use the -o option to specify an output file:
perl amg.pl -u http://www.allmusic.com/album/once-i-was-an-eagle-mw0002524684 -o import.mpl
-
Thank for the reply MrC, and i have a few question to add
My album review/credits are large value and if i edit this type the script will not working, any idea how to fix it?
And i see that album rating is 0-10 and i use 0 to 5 (with .5) can this be change it on this way?
And thanks again for the support
-
My mistake, i did change the wrong field in the script, now is working as intended but the date is geting only year and not the exact date, ofc if it have it.
-
Which version of MC are you using?
Prior to 19, you can't have the large value (newline embedded) columns in the view and use Copy to clipboard. MC19 allows supports what the script needs to correctly interpret the values on the clipboard. So if you're using MC18, remove those columns from the view.
There are two ratings supported by the script: MC's 0-5 value, as key 'ratingsmc' and 'rating'.
Since version 1.5:
- New: option --rating allows selecting format of AMG rating (d=0-9, f=0.0-5.0)
so you can get 0-9 or 0.0 - 5.0 ratings for the AMG rating (keyword 'rating'). Use --rating f to get floating point 0.0 - 5.0.
The Rating field will be scaled to fit MC's 0-5 integer scale (keyword ratingmc).
-
My mistake, i did change the wrong field in the script, now is working as intended but the date is geting only year and not the exact date, ofc if it have it.
See the new config file. You can set your field names there, so you don't need to modify the script any longer.
I'm only setting the Date as the Year value. I can probably change it to accommodate the full date. Would that be helpful?
-
I tried today this script, (i know you did a great job) don't have the time to test it so today was good to start.
So far from what i read and tested works splendid, but i have some other things to fix it to be perfect or almost
I use ver 19 and it will be great if you get as date full not only year (when is possible)
And btw can i add more things to get like, review of songs or album release (label) or if an album is a pick.
Thanks.
-
I'll take a look at the full date pulling.
As far as album review, etc., please take a look at the sample config file. The script can pull more than the original posting indicates, and you can see from the config which key/MC Fields are available.
I really should start a new post in the 3rd party tools, and keep the main posting updated with the full description.
-
[edit: see post 91 for the latest information]
Attached is Version 1.10:
Changes:
- Fix: problem reading MC19's new clipboard format on Mac OS X.
- New: support full dates in Date value.
- New: option --yearonly or -y specifies Date values should be year-only
-
The script work nice and thank you for the help.
But i have a few question about this script.
Is possible to add some fields to fill it with info or do i need to code it myself?! (if so i'm zero skill there) why i ask this? is because i desire to get a few info from album page (like album pick, song review, performer and reviewer of song and album) when is possible ofc. .
And for releases field is possible to get only the label and nothing more? there you get year/album name and label.
-
Am I correct that you have not yet looked at the config file, because, as I've mentioned the script already pulls:
Album Pick
Album Review (there is no song review)
Album Reviewer
Track Composer
Track Performer
I grab everything on the main page (Overview), and also the Credits, Awards, and Releases pages, and make it available to you.
As for the Releases, you just want a list of labels, from all formats (that seems ambiguous)? See attached:
-
I looked on the config file and searched on the script a way of what i asked, and i catch my eyes with all code and i don't know how to use it (yet, maybe one day i will).
As for the second thing, yes some album have on the track review (usual older one) and for the release, yes only the title of label, because when i get info from album i take where it is album discography.
The review link move you to another page and there you find what i talk about.
http://www.allmusic.com/album/play-mw0000243891
-
I've never seen that!
I'll look into the song review, seems easy enough. Should this override the album review field (e.g. AMG Review) for just that track, or should it be a separate field (e.g. AMG Track Review)?
For releases, I can provide a way to get just the list of the labels. Maybe it will be like credits_nr, which are the credits without the role included (i.e. just people).
-
Separate if it is possible.
And one more thing! Some albums have album pick like track pick, can be this also added?
Thank you!
-
Ah, I see that also on the Moby album. So much to learn.
I'll add it as AMG Album Pick.
-
On the album view is a field called "performer" can this be added also? (only the names)
-
Performer is Artist, which is captured.
btw. AMG Album Pick is done.
-
Performer is Artist, which is captured.
btw. AMG Album Pick is done.
Performer as artist don't do nothing on that field, remain empty after the import. Can this performer be different from artist field?
Ex: Artist Album Moby; Artist Moby (feat Moby) as picture show uphead
Performer field: Moby/Moby ?
My mistake, i did change the artist to my field (performer) and is working, but without anything else, not additional info.
-
Please send me a PM with your email so that I can send you an update to test out.
Here's what's coming up (already done):
Version 1.11
- New: AMG Album Pick field: key 'apick'.
- Change: AMG Track Pick key 'pick' is now 'tpick'.
- New: AMG Track Review: key 'treview'.
- New: AMG Track URL field: key 'turl'.
- New: format and values included in Releases is controlled by a template. See the sample config file.
- New: Lyrics included with key 'lyrics'.
-
Let's consider this current thread closed, and henceforth use the new thread in the link below.
Version 1.11
- New: AMG Album Pick field: key 'apick'.
- Change: AMG Track Pick key 'pick' is now 'tpick'.
- New: AMG Track Review: key 'treview'.
- New: AMG Track URL field: key 'turl'.
- New: format and values included in Releases is controlled by a template. See the sample config file.
- New: field Lyrics included with key 'lyrics'.
- New: add version number to mpl attribute.
- New: field Featuring (featured track artists) included with key 'featuring'.
- Change: removed --htmlfile option
- New: -h available now as shortcut for --help
- Change: config file rename to amg-config.txt (for easier file editing on Windows).
- New: tags can be updated via MCWS instead of importing an MPL file. Use the
option --mcws. See MCWS in the config file for settings and requirements. Other
related options are --password to specify username:password and --server for server:port.
Version 1.11 is now posted here:
http://yabb.jriver.com/interact/index.php?topic=84268.0 (http://yabb.jriver.com/interact/index.php?topic=84268.0)