Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[jmone]

Last Updated: 27 Sept 2015

Background

MC introduced "Particles" in V17 that enabled us to create and manage "parts" of a video as separate entries in the data base. The current version lets you auto generate particles based on the various playlists and this works well for discs with different TV Eps, Alternative Endings etc.  One use is also to break up a Music Blu-ray by Track (just like you can with a CD). Up until now, this is a manual process.  More in the Wiki

Purpose
Thanks to the work done by MrC with MCUtils (a general purpose scriptable MC field manipulator + utility) we have the ability to:
- Generate individual particles based on Blu-ray chapters
- Populate the Names of the chapters from the Web

Here is a short Video showing what this script can do including:
1) Check what Chatper Information is available
2) Create Particles based Chapters in MC
3) Submit Chatper information from MC to ChatperDB

The script will in order
1) automatically use the details stored at http://chapterdb.org/ if it is available or
2) fall back to use MediaInfo to create blank particles that you can then tag in MC by hand

Prerequisites
You will need to download, install, and configure and have working the following components:
Core Prerequisites
- MCUtils:  This is the main script that does all the work, and all you need if the details of your Blu-ray has already been uploaded by other users to http://chapterdb.org/

Extensions
If the details of your disk has not been uploaded to http://chapterdb.org/, you have three choices
- MediaInfo: Many of us already use MediaInfo, and there is also a CLI version that you can download that MakeBDChapters can use to read the chapter information from most (but not all) Blu-ray disks.  MakeBDChapters can use this information to create the particles, though they will be titled "Chapter 1, Chapter 2 etc, so you will then need to manually update the names.  Note: There are many ways to author Blu-ray discs and MediaInfo does not always return the correct number of chapters.  A bug report has been logged with them.
- ChapterGrabber:  ChapterGrabber is the accompanying Windows Application for ChapterDB that that allows you to scan a Blu-ray, add Chapter Names and a Disc Title that are then uploaded to ChapterDB.  You will need to both install the application and establish an account
- Manual:  You can always do it all by hand!  See the wiki

Limitations
- Only works with Blu-ray Disks
- Blu-rays must already be ripped and imported into MC as unencrypted disc structures

Result

This is what you can expect to see:

[jmone]

Setting Up MC

Creating a Library View
While there is no longer any need to add a custom view scheme , it can be useful to create one similar to that shown in the first post to display your Music Videos in a similar layout to Audio Albums including "Track #", "Name", "Filename", "Media Type", "Media Sub Type", "Artist", Album", "Playback Range" etc.  

[jmone]

Auto Create BD Title Particles

As per the cp.png pic, select the BD you want to process and create the initial particles in MC as usual.  As you can see, MC has created a particle for each of the Playlists on this BD.  You will now need to play each one and determine which one is the main playlist showing all the chapters to process.  The other playlists may also have content you would like to keep so you can either keep them (eg it may be a bonus song, behind the sceens doco) or delete them (such as warnings, ads etc).

In this example (see cp2.png):
00000.mpls = 1 chapter for the entire concert (I deleted this entry)
00001.mpls = Behind the Scenes (kept and renamed)
00002.mpls = Used for the menu (I deleted this entry)
00003.mpls = Warning (I deleted this entry)
01002.mpls = same as 00001.mpls Behind the Scenes (I deleted this entry)
01002.mpls = same as 00002.mpls Used for the Menu (I deleted this entry)
02000.mpls = This is the concert with all the Menu Items

[jmone]

Running MakeBDChapters

So by now you have installed, read all the Readme files and configured pscriptor and are ready to MakeBDChapters!

Check Preview of the information available for your MPLS.
- Select the MPLS in MC
- Open a Windows CMD window and follow the pscriptor README.usage.txt instructions for running the pscriptor command:

     perl pscriptor.pl -E MakeBDChapters -v -a check

Create Create Chapter Based Particles in MC
- Select the MPLS in MC
- Open a Windows CMD window and follow the pscriptor README.usage.txt instructions for running the pscriptor command:

     perl pscriptor.pl -E MakeBDChapters -v


If the item was not found in the ChapterDB database you will get a message.  If you have MediaInfo Installed (and configured in pscriptor) it will automatically use this to create the chapter based particles but you will then need to enter the track names manually.

You should now have your particles correctly created as shown in ps2.png

Submit This allows you to upload the Chapter Based particle information back to Chatper DB.  This is very useful if your BD did not exist in ChatpterDB and you had to enter the name information manually.  Once done, this information will then be available to other users.
- Select all the Chapter Based particles in MC
- Open a Windows CMD window and follow the pscriptor README.usage.txt instructions for running the pscriptor command:

     perl pscriptor.pl -E MakeBDChapters -v -a submit

AutoHotKey Script To save some typing in a CMD box for Windows users, I have created a simple AutoHotKey script that you can run that lets you select these function (see pic).  Just unzip the files in the attached MCUtilsGUI.zip to your MCUtils folder, run MCUtilsGUI.exe then right click on it's icon in the System Tray to access the menu of commands.  I have also included the pscriptor.ahk script file so you can see what it does and also make changes (you would need AutoHotKey installed)

[jmone]

Options

In our testing most of the Music BDs where fine with the correct number of particles created and populated with the correct names.  If you find:

No Chapters were created at all  This happens when the disc was not in ChapterDB and you don't have the CLI Version of MediaInfo installed (or MCUtils is not configured to use it)
- You can create an entry in ChapterDB using the ChapterGrabber tool, or
- You can install the CLI Version of MediaInfo and configure MCUtils to use this (note: you will need to enter in the Chapter Names by hand)

Correct Number of Chapters but the names are Blank, Wrong, or Just "Chapter 1, 2, etc"  This can happen if the user data entered on ChapterDB is poor, or if MediaInfo was used to create the chapter particles (as it can not read the Chapter Names)
- You can manually edit the Name in MC (note: we have seen some instances where MC then renames the filepath breaking the particles)
- Locate a "better" example in ChapterDB, eg There were multiple entries in ChapterDB for some of my discs and the one being automatically returned did not have chapter names, yet the other entries for the same disc were correct.  If you take the Disc ID (the last part of the url), in this case "36873" and enter it into the custom "Chapter DB" field for the main particle, MCUtils will use this value to pull the data and create the chapter based particles. See ps3.png

The wrong Number of Chapters were created This happens when the disc was not in ChapterDB and MediaInfo retuned an incomplete list
- You can create an entry in ChapterDB using the ChapterGrabber tool

[jmone]

Chapter Grabber

Here is an example of using Chapter Grabber to first read the chapters from the playlist (00001.mpls) from which I entered the Names for the Chapters, the Title of the disk and on saving it uploads this information to ChapterDB.  I then ran MakeBDChapters over the entry in MC and it created the chapter particles correctly.

[jmone]

Change Log

2014/06/06:  First release
2014/06/10:  Updated the guide post MrC feedback (including a link to a video overview in the OP & copy of a View Scheme).  Also added a Batch File to run the pscriptor commands
2014/06/14:  Updated the guide to reflect 2 new options "- check" to preview what information is available and "- submit" which will push your Media Center information back to ChapterDB.  Replaced the batchfile with a AutoHotKey script.
2014/06/15: Updated AutoHotKey script (submit function was not working correctly)
2015/09/27: Updated to reflect change in the core script to MCUtils.  This new script engine no longer requires multiple passes and dummy fields

[jmone]

Thanks to MrC for his work over the last few weeks. 

We have looked at various on-line sources to find and extract accurate chapter times and names.  ChapterDB is the "best" for this but as with all user generated data it suffers from both missing entries and inaccurate entered Titles.  As a fallback for discs without an entry, pscriptor can use MediaInfo but it also has issues reliably reporting chapter times on some discs and it can not read the chapter names.

Potential Enhancements / Changes in the future (note: many of these would require MC changes)
- MC already "knows" these chapter times but at present can not create chapter based particles
- The ability to submit Chapter Info from MC to ChapterDB
- Ability to adjust the Chapter Times (some discs are authored where the first bar of the next song is heard on the last bar of the current song)
- Use the Chapter Information in MC on movies (so you can see Chapter Names instead of just "Chapter 1, 2 etc)
- Use the Chapter Information in MC on movies when converting format (so you can see Chapter Names instead of just "Chapter 1, 2 etc)
- Expand the use to DVD (at present MC does not support Time based particles from DVD)
- Use this information to backup particle information (aka Sidecar files)
-

FYI - One benefit, is BD's can carry some of the higher quality audio encoding commercially available (normally both a 2ch PCM and a DTS-MA 5.1 Mix).  Once you have chapter based particles, you can use MC's Convert Format to extract the Audio portion to FLAC (or other format) files. 

Please let us know how it goes and any other ideas that makes sence

[Frobozz]

FYI - One benefit, is BD's can carry some of the higher quality audio encoding commercially available (normally both a 2ch PCM and a DTS-MA 5.1 Mix).  Once you have chapter based particles, you can use MC's Convert Format to extract the Audio portion to FLAC (or other format) files. 

If you extract the audio this way will it perfectly maintain any gapless track or chapter transitions?  For example, if you extracted the 2ch PCM from a classical music concert where the transition between movement 1 and movement 2 is gapless, will it maintain that gapless transition when you extract the 2ch PCM to flac tracks?

[jmone]

While transitions with the video chapter particles are gapped, It "should" be gapless for extracted audio...but I would have to double check.  I'm away for a couple of days and will do so then.  Is there a particular BD you would like to check and I'll see if it avail and post the results?

Edit, you of course get gapless if you play the original whole MPLS or it's extracted audio that would be one big audio file

[Frobozz]

It "should" be gapless but I would have to double check.  I'm away for a couple of days and will do so then.  Is there a particular BD you would like to check and I'll see if it avail and post the results?

I don't have any particular BD title in mind to check.  I don't own any BD video or audio titles yet.

I'm trying to figure out if I want to subject myself to the issues of playing BD video and audio on a computer.  My interest in BD titles is primarily for audio purposes.  Concert video and BD audio titles and others that have quality lossless 2ch audio.  Gapless is an important feature for me since I like classical, jam bands, Pink Floyd, and other music that can have gapless transitions.

If this process is able to do gapless transtions when converting chapters or tracks to flac then I'll be able to consider getting a BD drive for the computer and the necessary software to make it all work.

[jmone]

Just tested and can confirm the converted files to FLAC play perfectly Gapless!  Couple of tips when converting from the original Video to Audio Conversion Thread:
- In MC select the Audio track you want to check (eg 2ch or 5.1 etc) in the particles to be converted
- In the Convert Format box apply any DSP functions you want (eg Mix 5.1 to 2.0 etc)
- In the Convert Format box options make sure you only have 1 track to convert at a time (this is important!!!)
- The converted FLAC tracks will be named in the format 00001.flac, 00001 (1).flac, 00001 (2).flac etc so when finished, use MC's "Rename, Move, & Copy Files" function to put them in the right place and name them correctly (see pic).
- Apply Coverart
- Enjoy!

Note: With one Music BD you may have a multiple tracks, say a PCM 2.0, DTSMA 5.1 mix and you could then from each make multiple Audio Albums for different purposes such as a:
- 2.0 --> MP3
- 2.0 --> FLAC 2.0
- 5.1 --> FLAC 5.1
etc

[jmone]

FYI - Amazon lists over 1,000 Music BDs and most are the same sort of price as a CD.  Why not get the BD, and then you would have both the Music Video and the ability to create your own Audio only tracks from some of the best quality mixes available.

[Frobozz]

Way cool!  MC continues to amaze me in what it is capable of doing.

There are some BD titles I want.  The reasons I don't have any BD titles is more philosophical than technical.  The reasons being more appropriate in the "Music, Movies, Politics, and Other Cheap Thrills" section of the forum than here.

Seeing that MC can extract gapless tracks from BD rips has me considering getting some BD titles I've had in my wish list.  What I want to be able to do is technically possible and is supported by MC.

[MrC]

There is a bug in pscriptor which causes chapter titles that are double-quoted in the chapterdb.org database to end up with generic "Chapter #" names on pass 2 when the chapters are being particlized.  I've fixed and have sent to jmone for his quick verification.  I'll make the update available shortly.

Edit: Update posted.

[jmone]

Updated guide.

Also particalised 3 More BD's.  2 of which worked perfectly and one needed it's details to be uploaded to ChapterDB.

[jmone]

Updated guide with 2 new MakeBDChapter options:
"- check" : This function will preview what Chapter Information is available for your BD Title Particle
"- update" : This function will submit to ChatperDB your updated Chapter Information from MC.

[jmone]

In Post #3 : Running MakeBDChapters
- Replaced the Batchfile with a AutoHotKey script so you can run the pscriptor commands from Windows Tray itself

[Ekpen]

Jmone,
I thank you and others MC "Die Hard" evengelists that contributed to this utility.
I have the autohotkey scripts in my tray now.
my BDs are in mkv rips. Will this utility work with rips?
Remember, I ripped to ISO  with your help. Realizing my Directv stbs do not recognize ISO, and then my Samsung Galaxy note 1 did not see any ISO files, I have no choice , but to start ripping my collections as mkv.
Utility installed, database field updated within MC, but not yet tested the program. Will do eventually.
When we get Linux platform fully fuctional, how are we going to get these Windows utility to Linux ?

Thanks to all of you.

George

[jmone]

This utility is aimed at processing decrypted Blu-rays themselves (eg those that have been ripped and stored by Structure, not ISO or repackaged to MKV).  MrC's pscriptor code works on Windows, iOS, and Unix so you will be able to use it once MC supports BD on these platforms (but the AHK script is Windows only)

The same "concept" could work with MKV files as you can create time based particles that work now by hand (but some of the auto lookup features for names would be harder).  You would need to ask MrC (the man behind the script).

[jmone]

2014/06/15: Updated AutoHotKey script (submit function was not working correctly)

[Ekpen]

This utility is aimed at processing decrypted Blu-rays themselves (eg those that have been ripped and stored by Structure, not ISO or repackaged to MKV).  MrC's pscriptor code works on Windows, iOS, and Unix so you will be able to use it once MC supports BD on these platforms (but the AHK script is Windows only)

The same "concept" could work with MKV files as you can create time based particles that work now by hand (but some of the auto lookup features for names would be harder).  You would need to ask MrC (the man behind the script).

I forgot the pscript  from MrC works for Linux, and Mac also.
It is autohotkey that is my concern.
Maybe MrC will see my  question that I have POSTED. i will give him time to read my concerns.
Thanks again, for the update.

George.

[jmone]

Hi George, you don't need the AHK script to run the commands, it is just an option I made for Windows to make it "easier" for those that prefer a GUI over opening up a CMD windows and typing them out (or you could run from batch files etc).  note: I don't know what the equivalent is on Linux/OSX but at this stage you can't play Video anyway on these versions.

Also MrC reads pretty much every post so I'm sure he will reply.
Thanks
Nathan

[MrC]

Maybe MrC will see my  question that I have POSTED. i will give him time to read my concerns.

Is your primary question about how you can get chapter info to your mkv files?

[Ekpen]

Hi George, you don't need the AHK script to run the commands, it is just an option I made for Windows to make it "easier" for those that prefer a GUI over opening up a CMD windows and typing them out (or you could run from batch files etc).  note: I don't know what the equivalent is on Linux/OSX but at this stage you can't play Video anyway on these versions.

Also MrC reads pretty much every post so I'm sure he will reply.
Thanks
Nathan

I am gearing up for future Linux platform. Whenever the video is available, I will move to Linux. My server  that is 99.99% will have about 80TB + of HD space.  You got me  in to ripping ? <grin>.
It looks like MrC has read my concerns. I will be replying him.

Thanks for all the contribution.

George

[Ekpen]

Is your primary question about how you can get chapter info to your mkv files?

Greetings:

Based on the topic of this thread.. "aka Rip Music BDs like CDs", I ask if it is possible to rip music  tracks off BD/DVDs that have been converted to Mkv. These mkv files are going to be in separate folders.  It looks like  the script you wrote will scan  optical drives as source.
Can the source also be folders, multiple folders that we can select(a little bit of batch) operation?

I found out I got lots of DVDs and BDs that are concerts. At times, I just want to listen to audio, the same applies to my wife.

My wife just sent an email from work a few minutes ago, to get Gino Vanelli concerts dvds or BDs.

Note: (an excuse to spend her money] < grin>

I hope to be able to put the audio in the Pono music hardware, when we get to the time of purchasing Pono.
All my audio is in Flac.
Note: I have configured the pscriptor and the autohotkey. I hope it is going to make my MC server really hot and a fine showcase for my friends to see and hear.
It is all about making MC the best.

George

Thanks again, for your help and contribution.

[MrC]

Let me clarify a few things.  The pscriptor utility performs certain specified actions on the files within MC.  It uses the files you select in the view as its input, and performs operations depending upon the selected scriptlet (or in-line script code you supply).

The MakeBDChapters scriptlet is one such scriptlet.  It is supplied (by pscriptor) with the selection you've made in MC, and this file list selection would be the blu-ray file entry in MC, the MPLS files (created by MC), and later also the dummy files (created by MakeBDChapters in the first pass).  It does not access or need the Blu-ray disc itself, ever.

The scriptlet MakeBDChapters does essentially two things: 1) it creates the chapter-entries in MC based upon the Blu-ray MPLS file you've selected and particlizes these, and 2) it uses online chapter look-ups from ChapterDB, or uses the MediaInfo tool, to get as much information about chapters as possible (start points, names, title, etc.).

The MakeBDChapters scriptlet uses another scriptlet called Particlize.  Particlize is what does the magic if creating the particles, and it can work on any file type; MKVs are perfect candidates.

So there is no reason why MKV's cannot be used.  So long as chapter start points are available, your MKVs can be particlized to provide you with specific playback points of the whole MKV file.   It appears that the MediaInfo tool does provide these chapter start points (so long as they were in the MKV in the first place), but these won't have chapter names.  But also, ChapterDB lookups can be used, which often have chapter names - this would be done be specify the specific chapter DB entry you want to use (with Blu-ray playlists, the lookup is often automatic, based on the MPLS file you've selected in MC).  There are other tools that might help populate chapter names if they are absent.

You won't be "ripping" music tracks from your MKVs - rather, you'll simple use particles to play the individual "chapters".

The script doesn't care where the files reside.  MC already knows this and makes the information available in the view.

It would be best to try working with particlized media in MC, so that you can learn about the pros and cons of particles, how the selected Media Type affects playback and which external programs and tools you might use can work with particles (since these are not actual "files", but references to an existing file in MC but having a specific Playback Range set).

I'm not clear about how your MKVs are related to your FLAC files you mention (unless you are converting your MKVs to create audio-only FLAC).

[Ekpen]

I'm not clear about how your MKVs are related to your FLAC files you mention (unless you are converting your MKVs to create audio-only FLAC).

This was my thinking.. to create Flac from the mkv rips.
I will try it.

Thanks.
George

[Ekpen]

a specific Playback Range set).

I'm not clear about how your MKVs are related to your FLAC files you mention (unless you are converting your MKVs to create audio-only FLAC).

MrC,

Sorry for the delay in getting back to you in a fairly reasonable time.

[Maybe I am over thinking this by making it more difficult than it seems.]  I will try to read your notes again carefully. I am still having  problems with my  new pair of glasses.

Here is my thought on this. I already have mkv ripped files on my hard drives. They are are music concerts- e.g Yanni live at El Morro. These are either DTS or Dolby audio contained within the ripped files.  I was looking for a way to extract or convert the track to a new format by using a

program like dbpoweramp to convert to Flac. (Yes, I want to convert the audio files within the mkv or the mkv file to flac)

Thanks.

George

[hulkss]

The MakeBDChapters scriptlet uses another scriptlet called Particlize.  Particlize is what does the magic of creating the particles, and it can work on any file type; MKVs are perfect candidates.

So there is no reason why MKV's cannot be used.  So long as chapter start points are available, your MKVs can be particlized to provide you with specific playback points of the whole MKV file.

OK, I have over 100 BD concerts ripped into MKVs.  Chapters are in place for each song and are titled. Now how would I proceed to particlize them automatically? This would be great for making play lists of individual concert songs. Thanks for any guidance on this.

[audiocrazy]

When I run this command after the First Pass I get this error.
I also watched the video tutorial and there is no album info in the tutorial also. How can i resolve this.
Check the attached pic.

c:\perl64\bin\perl pscriptor.pl -E MakeBDChapters -f "dummyfield" -v --filekey
Uncaught exception from user code:
        The field Album is required in the file list, and must have a value.
        Stopped at Scriptlets/MakeBDChapters.pm line 204.
        MakeBDChapters::do_preprocessing(undef, ARRAY(0x3693a98), ARRAY(0x369ee18)) called at pscriptor.pl line 163

[audiocrazy]

When i update the Album and Artist Name manually i get this error

Use of uninitialized value $current_mpls in hash element at
        Scriptlets/MakeBDChapters.pm line 269 (#1)
Use of uninitialized value $current_mpls in hash element at
        Scriptlets/MakeBDChapters.pm line 270 (#1)
Use of uninitialized value in string eq at Scriptlets/Particlize.pm line 103 (#1)
Uncaught exception from user code:
        Failed to unlink file: C:\Users\malik\Desktop\MakeBDChapters\Chris_Botti_In_Boston\00003\Chris Botti in Boston - Title 1 Chapter 01.mp3: No such file or directory
        Stopped at Scriptlets/Particlize.pm line 113.
        Particlize::do_perfile(undef, "dummyfield", HASH(0x30482b0)) called at Scriptlets/MakeBDChapters.pm line 275
        MakeBDChapters::do_perfile(undef, "dummyfield", HASH(0x30482b0)) called at pscriptor.pl line 213
        main::process_user_tracks(ARRAY(0x3f69778)) called at pscriptor.pl line 170

[jmone]

As there is an updated Script Package, I'm currently in the process of updating the instructions and screenshots.  It is now much easer to use.  I'll advise when done.

[jmone]

Guide is now updated.  This is a big change as the core Script has been updated to MCUtils that has greatly simplified the process from the prior version.  Any feedback welcome!
Thanks
Nathan