Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Matt]

With the help of Andrew Van Til (senior editor of Missing Remote), we're adding commercial skipping support to the video playback engine.

Tonight's build will look for a sidecar EDL file with the same name as a video, and support mute and seek edits.

Future enhancements may include putting the commercial regions on the progress bar display, and supporting additional commercial skip formats.

I have a few technical questions with regards to how to implement the EDL support, so I'll follow-up this post with the details.

[Matt]

Imagine a simple example:

30 minute TV show
EDL has two skips:
A: 10:00-11:30
B: 20:00-22:30

Playing forward would perform each skip.  If you back up after a skip, it should not skip again.

But what should happen if a bookmark starts playback at 12:00, and you seek back?  Should it skip A?  What if you seek to the middle of A?  What if you seek to 0:00?

Should seeking backwards more than some fixed size (say 10 minutes) reset all the skips?

Also, should the program display something on the OSD when it does an auto-skip.  For example, something like "Skipping 4:00"?

Any other little details to consider?

[JimH]

Andy is also author of a couple of important pieces of software.  More on his site:
http://babgvant.com/

We've added him to the beta team.  Welcome!

[babgvant]

Playing forward would perform each skip.  If you back up after a skip, it should not skip again.

Yes.  In skipping should be temporarily disabled until the position that you skipped back from has been passed.  The use case is a place where detection is wrong, usually where the last couple seconds of a show segment has been misidentified. 

But what should happen if a bookmark starts playback at 12:00, and you seek back?  Should it skip A?  What if you seek to the middle of A?  What if you seek to 0:00?

Should seeking backwards more than some fixed size (say 10 minutes) reset all the skips?

My preference is for forward only skipping.  So the answer to all of these questions would be no.  That said, there may be other "right" answers.  Perhaps a way to reset/renable after skipping back in the UI?

Also, should the program display something on the OSD when it does an auto-skip.  For example, something like "Skipping 4:00"?

I think you should just show the scrub when an automated skip occurs.

Any other little details to consider?

1) I'd like the scrub to mark the show/commercial spans, but that's a nice to have.

2) Skipping back and forth using the EDL should be exposed through the dpad; left goes to the previous marker, right goes to the next marker, maybe use up for resetting and down to disable forward skips (if the edl is junk for e.g.).

3) Configurable pad to the start/end times (some people want to see a second or two on either side of the commercial segment just to make sure they aren't missing something).

4) Option to trigger scanning (i.e. run a custom exe) if an edl file doesn't exist for a TV file

Thanks for doing this BTW. 

[babgvant]

We've added him to the beta team.  Welcome!

Thanks!

[Osho]

Three things regarding commercial skipping that I would like to see:

1. Commercial skipping meta-data files should automatically be created by invoking the appropriate program by MC. User should not have to set-up some other program and/or scheduled tasks in Windows to generate meta-data files.
2. It should be possible to select if the meta-data is generated as soon as the program recording completes or at some user-configurable time when machine is not in interactive use.
3. Forward commercial skipping should be automatic. I shouldn't have to press any button on the remote to skip through the commercials.

Thanks,
Osho

[babgvant]

1. Commercial skipping meta-data files should automatically be created by invoking the appropriate program by MC. User should not have to set-up some other program and/or scheduled tasks in Windows to generate meta-data files.

I don't think anyone would mind if the feature is there as long as it was configurable. 

I prefer to use a custom app (DTB) to watch for new TV files and do interesting things with them (beyond commercial analysis).  Also, I'm not sure how you get around the need for end users to install/configure 3rd party applications like comskip and ShowAnalyzer (SA) so what's the harm in another step.  There isn't a one size fits all approach to this problem, some will want live scanning, some will want delayed and some will want to batch it up.

[rpalmer68]

This is excitng, but how do we generate the EDL file?   comskip and Showanalayser don't know what to do with a jtv file.

Thanks
Richard

[babgvant]

This is excitng, but how do we generate the EDL file?   comskip and Showanalayser don't know what to do with a jtv file.

Thanks
Richard

Can you use TS instead?

[rpalmer68]

Can you use TS instead?

Maybe, I haven't tried.  But JTV allows me to watch a show being from a clent while it's recording on the server, ts won't.

[babgvant]

They should be to workaround that issue with a custom file source filter.  I'm not familiar with the JTV format; anyone know if it's documented anywhere?

[Matt]

Media Center installs a source filter for JTV files.  It's the same one we use for playback.

[glynor]

My preference is for forward only skipping.  So the answer to all of these questions would be no.  That said, there may be other "right" answers.  Perhaps a way to reset/renable after skipping back in the UI?

I agree completely here.

Reverse skipping gets frustrating quickly if the comskip EDL generator doesn't quite work right.

[Matt]

I made it so that it only does a skip once.

But if you jump back more than 10 minutes, it'll reset and do skips again.  My thinking is that at some point you're not jumping back to correct a wrong skip but instead jumping back to watch the show over.

[babgvant]

I made it so that it only does a skip once.

But if you jump back more than 10 minutes, it'll reset and do skips again.  My thinking is that at some point you're not jumping back to correct a wrong skip but instead jumping back to watch the show over.

That is a valid use case.  I should have been clearer, re the forward only thing; it should only be relevant for the current session.  Once you start a new session it should be forgotten.

[babgvant]

Media Center installs a source filter for JTV files.  It's the same one we use for playback.

Do clients support "Live TV" when TS is used?  What are the benefits of JTV over TS? 

[raym]

Perhaps you could provide an option to toggle the feature on and off via the OSD. that ways, if someone really wants to watch a scene over and over, they can flick skipping off for that session.

[KingSparta]

I remember Some DVR's Has some legal issues in this area, first off with ReplayTV.

[babgvant]

I remember Some DVR's Has some legal issues in this area, first off with ReplayTV.

Call it something different then, like advance bookmarking

All the feature does is use a text file to move around in a file, what the spans mean is up to the user.  For e.g. some people use a similar feature in DTB to simulate chapters in Windows Media Center.

[babgvant]

Tonight's build will look for a sidecar EDL file with the same name as a video, and support mute and seek edits.

Initial testing looks great   Thanks

[raym]

Future enhancements may include putting the commercial regions on the progress bar display, and supporting additional commercial skip formats.

Any chance MC could read a VideoRedo .vprj cut file too? Xbox MC and others support this format as it's quite popular.

I've attached a sample.

[Osho]

Any chance MC could read a VideoRedo .vprj cut file too? Xbox MC and others support this format as it's quite popular.

I've attached a sample.

Just my 2 cents: before supporting many different formats for meta-data for TV recordings, MC17 should 'standardize' on one such format - the one that it generates automatically after a TV recording is completed and analyzed (at a potentially user-configurable time and priority). Let's make sure that one method of commercial detection and skipping works well before opening up to other usage flows.

Thanks,
Osho

[raym]

Just my 2 cents: before supporting many different formats for meta-data for TV recordings, MC17 should 'standardize' on one such format - the one that it generates automatically after a TV recording is completed and analyzed (at a potentially user-configurable time and priority). Let's make sure that one method of commercial detection and skipping works well before opening up to other usage flows.

Thanks,
Osho

I actually came back here to re-tract my request but you responded before I could do so. I completely agree with you. EDL (as produced by comskip and the likes) is a far more widespread format I believe and certainly the correct choice to focus on first.

I've messed with this quite a bit today and I've gotta say, WOW! It's so nice to be able to finally use this feature inside MC. Well done guys.

[raym]

It just occured to me that perhaps EDL is not a standardised format? When compared to Matt's example above, the EDL that comskip produces looks a bit different, eg:

123.36   333.52   0
843.64   978.08   0
1412.08   1569.32   0
1889.28   2014.08   0
2868.80   2994.76   0


This appears to work fine so... I'm not sure...

[raym]

I made it so that it only does a skip once.

But if you jump back more than 10 minutes, it'll reset and do skips again.  My thinking is that at some point you're not jumping back to correct a wrong skip but instead jumping back to watch the show over.

I don't like this much. The family often jumps back briefly to hear a line in the dialog that was missed or whatever. Problem is, the next jump point is ignored.

So, my pref would be a) ignore back skips entirely (as others have said) and b) enable forward skips ALWAYS - with an OSD option to enable/disable commercial skipping if the sidecar exists.

[Matt]

I don't like this much. The family often jumps back briefly to hear a line in the dialog that was missed or whatever. Problem is, the next jump point is ignored.

I'm not sure I understand.  They seek back more than ten minutes to briefly hear a line?  Wouldn't that normally be seeking back a minute or less?

Thanks for any clarification.

[Matt]

It just occured to me that perhaps EDL is not a standardised format? When compared to Matt's example above, the EDL that comskip produces looks a bit different, eg:

123.36   333.52   0
843.64   978.08   0
1412.08   1569.32   0
1889.28   2014.08   0
2868.80   2994.76   0


This appears to work fine so... I'm not sure...

I wasn't posting EDL, just something human readable.

You're example is EDL.  It's like this:
[Start Time] Tab [End Time] Tab [Edit Type] Newline

I think the edit type of 0 means cut, not commercial break (which is 3).  But it looks like real-world examples use the two interchangeably, so we treat them the same.

[babgvant]

I think the edit type of 0 means cut, not commercial break (which is 3).  But it looks like real-world examples use the two interchangeably, so we treat them the same.

FWIW, I've only seen 0 in the last position.

[raym]

I'm not sure I understand.  They seek back more than ten minutes to briefly hear a line?  Wouldn't that normally be seeking back a minute or less?

Thanks for any clarification.

I'm not sure I understand either    The way I read your description of this and what I was sure I observed myself initially was jumping back LESS than 10 mins would cancel auto skipping. That made no sense. But I can't reproduce this now so I'm not sure what I was thinking .

Thanks for the clarification on EDL formats.

Once again, this is an awesome addition!

[raym]

So I'm not going crazy... I just started watching a show and MC skipped forward the first 2 times as expected.   About 20 mins in I backed up the video about 30 secs (NOT 10 mins) and MC did not skip the next ad. Stopping and restarting playback fixed it.

I dunno. I understand the problems with this but not sure what to suggest.  For example, what happens when a skip point happens too early? You go back a bit but will MC skip at the wrong point again?

Again, a way to toggle the ad skipping function in the OSD if an edl is present would be a good idea as a temp "fix" for issues like this perhaps.

Thanks.

[raym]

After a few days of ad-free bliss, I can make some real comments.   

1/ We definitely need the cut points represented on the OSD somehow. Perhaps as vertical lines along the time bar? When one is missed, for example, it would be helpful to see if the expected cut point actually existed as well as seeing where the next one is.

2/ The EDL sidecars aren't getting cleaned up when I delete the source file. This is pretty important as I already have numerous orphaned files floating around and it's likely to get out of hand fast!

Thanks. 

[Matt]

Do you still think it's skipping skips?  I could not explain why that would happen from your post above, but obviously the code hasn't been fire tested a lot yet.

Maybe you could provide a step-by-step and an EDL if you think there's a problem.

Thanks.

[babgvant]

Ran through a full test last night.  Very happy with the implementation.  If the scrub was marked it would be just about perfect

[babgvant]

Did some more testing and it would be really nice if there was an easier way to skip back to the last skip point (1) and skip forward (2) to the end of the current commercial segment.

Use case 1:

- the last bit of a show is incorrectly marked as a commercial so the skip occurs before the end of the show, when the skip occurs we should be able to get back to the skip point via a single key press

Use case 2:

- after returning to the show segment from 1, we should be able to reexecute the skip to the beginning of the next show segment

Thanks

[JustinChase]

With the help of Andrew Van Til (senior editor of Missing Remote), we're adding commercial skipping support to the video playback engine.

Wow, what crazy timing!

I seriously just walked away from the HTPC in the living room after confirming that I now have hands free commercial skipping working with WMC and wtv files, using Andrew's DTBAddin plugin and comskip!

I walked in here, wrote a thank you to the author of MC-TVConverter for making the change that allowed it to happen.

I popped over here to see what was new, and there's this post.  Incredible timing.

Good news is, that workflow can be very easily modified to get me the edl for ts recordings, a single digit change I think. 

...skipping should be temporarily disabled until the position that you skipped back from has been passed.  The use case is a place where detection is wrong, usually where the last couple seconds of a show segment has been misidentified. 

another use case is this

... I just started watching a show and MC skipped forward the first 2 times as expected.   About 20 mins in I backed up the video about 30 secs (NOT 10 mins) and MC did not skip the next ad. Stopping and restarting playback fixed it.

In this case, they skipped back not because of a wrong detection, but just to go back and see something.  If MC had disabled skipping only until playback got past the point they skipped back from, it seems commercials would have been correctly skipped; since stopping and starting indicated the skip point was there/working.  I'm guessing that upon skipping back MC disabled the next 'skip ahead' point instead?  Maybe just a bug?

I think if MC could look back maybe 15 or 20 seconds to see if it's recently passed a 'end skip' marker when you press the back button.  If not, you're just going back to see something again and skipping should not be affected at all.  If there was recently a skip, take me back maybe 4 seconds (or 2) before where it 'skipped ahead' from, then disable that 'skip ahead' point, by disabling skipping until *after* that point.  That would give me 20 seconds to notice that I missed part of the show, find the remote, maybe from another room, and hit back and go right back to wherever it skipped from and let me watch for whenever I might need to skip ahead.

BUT; that only works when it 'skipped ahead' too early.  What if the 'end skip' was the problem?  Then I need to go back maybe 20 seconds before that 'end skip' and see if that's commercial, or if I need back again to get back to the show.

I'm not sure how to distinguish which reason the back button was pressed.  Perhaps a long-press to go back before the last 'skip ahead' point, and a regular back press will go back XX seconds before the last 'end skip' point, and 'skip ahead' points will be ignored until after where the back press happened?

I'm looking forward to giving it a go, once the build is released.  I'll set some recordings to have some test files to work with.

I think I'll be happier working to fine-tune the comskip profile to better identify commercials, and not have to use the back button

Anyone know how to setup channel-specific profiles for comskip?

[pauly139]

Is there any commercial detection software around that will analyze JTV files and produce the EDLs?

[raym]

Is there any commercial detection software around that will analyze JTV files and produce the EDLs?

Don't think so. This might be coming though. In the meantime, you could record in .ts format and gen the edl via comskip.

[raym]

I think I'll be happier working to fine-tune the comskip profile to better identify commercials, and not have to use the back button Anyone know how to setup channel-specific profiles for comskip?

Yeah, I think the "issues" I was having had more to do with my comskip config rather than anything MC was doing.

Anyone know how to setup channel-specific profiles for comskip?

You can use the "--ini=<file>" switch in comskip to define channel specific configurations. Depending on your scripting skills, this can be quite powerful.

[AVTechMan]

What about files recorded in .wtv format? That would be really cool to play all of the .wtv files in MC and have the commercial skipping. Unfortunately I still have to use WMC7 to do the recording on the Ceton but I can transfer the recordings to the server.

My second HTPC which I am doing a new installation on I have a spare tuner card (not Ceton) that I can test fully with MC. I guess I will have to wait until I can pay the upgrade price to 17 to try it out.

[babgvant]

What about files recorded in .wtv format? That would be really cool to play all of the .wtv files in MC and have the commercial skipping. Unfortunately I still have to use WMC7 to do the recording on the Ceton but I can transfer the recordings to the server.

I'd like to get DCT working with MC's recording engine at some point

[nwboater]

I have a bunch of TV recordings I imported from SageTV. Most of them have been marked by Comskip which was properly working in Sage.

Is there some way I can set up MC17 to skip commercials on these?

It's great that Commercial Skipping has come to MC!!!

Thanks for any help.

Rod

[babgvant]

It should just work if the EDL files are next to the recordings (same as SageTV).

[nwboater]

It should just work if the EDL files are next to the recordings (same as SageTV).

It is working on the imported SageTV files. Thanks very much. In fact sorry I didn't get back here sooner to thank you.

It's not real clear from the posts above if Comskip should be working on MC TV recordings yet, but it doesn't sound like it. On our MC recording it is not working. For some reason we can only record jtv files (have a post on this in the TV section of the Forum) and perhaps it does now work on ts files. Does it?

Some clarification on the status and direction with MC on Comskip may be helpful to many.

Thanks,
Rod

[imugli]

Comskip does work for ts files.

I'm currently working to get it working on my Ubuntu server under Wine whenever a new file is detected in the recordings folder. 

[nwboater]

Comskip does work for ts files.

I'm currently working to get it working on my Ubuntu server under Wine whenever a new file is detected in the recordings folder. 

Just to be sure I understand correctly: In MC17 Comskip files are automatically generated while recording in the ts format. So when playing back commercials are skipped. Right?

If that's the case I sure need to get it worked out with Yaobing why I can't record in ts format with my HDPVR.

I don't see anything in setup to enable/disable Comskip. Is it just there?

Thanks very much for the response.

Rod

[imugli]

OK so I think we have different meanings for Comskip, and how MC treats / does this 

When I say "Comskip" works for ts files, I mean comskip the program (http://www.kaashoek.com/comskip/, not comskip as in "Commercial Skipping".

Secondly, I don't believe MC automatically generates the edl file. I think you still need to run a third party app (comskip) to generate it (thus the reason .jtv cannot be comskipped, because no 3rd party apps support it), but once generated MC honours the edl file. Thus my attempts to get comskip monitoring a folder and running automatically.

[pauly139]

Does anyone have Comskip working well for Australian television? If so, would you be willing to share your configuration file? Also, what application are you using to analyse the .ts recordings - eg Lifeextender or something else?

Thx
Paul

[nwboater]

OK so I think we have different meanings for Comskip, and how MC treats / does this 

When I say "Comskip" works for ts files, I mean comskip the program (http://www.kaashoek.com/comskip/, not comskip as in "Commercial Skipping".

Secondly, I don't believe MC automatically generates the edl file. I think you still need to run a third party app (comskip) to generate it (thus the reason .jtv cannot be comskipped, because no 3rd party apps support it), but once generated MC honours the edl file. Thus my attempts to get comskip monitoring a folder and running automatically.

Thanks for the clarification. I have Comskip Donators Version and it was working automatically in SageTV with a plug-in. Can somebody let us know if there is a way to have it run automatically in MC, or is that something that is being developed?

For those of you that are using Comskip to generate the edl files what are you doing to run it?

Thanks,
Rod

[JustinChase]

For those of you that are using Comskip to generate the edl files what are you doing to run it?

I have a utility that watches a Recorded TV folder, and when WMC is done recording a .wtv file (usually about 10-15 minutes after), or when MC17 finishes recording a .ts file (actually usually during, I think the way MC records in eparts in extra folders confuses the utility?) it will 'convert' it to .ts into another folder, on the network, then it runs comskip on it, creates the .edl file* and an .xml file located in a third location, which WMC and the DTBAddin read from and I get automatic commercial skipping when watching the original WMC files in WMC, and I get automatic commercial skipping of the ts recordings (the converted ones and the originals) in MC17.

* the .edl file is currently still being misnamed to match the original filename, not the renamed/converted file.  it takes 2 seconds to fix, then it's automatic.  I'm hopeful this is resolved in the next day or 2.

This does result in duplicates of your files.  Your workflow might cause you to want to delete either copy, and this will do that for you.  it still involves a bit of work with the comskip.ini, but you need to do that to get accurate skipping anyway.  That's on my list, but it's pretty close as it is now.

a 30 minute SD program takes about 5 minutes to completely finish the process and move all the files, YMMV

a 30 minute HD program takes about 13 minutes to completely finish.

MC-TVConverter  the current version can be had here  -  http://mctvconverter.vivolum.net/

He also has a beta board where you can download a version that has commercial cutting separated from commercial skipping, if you want to just get the .edl, but not actually cut the converted file (comskip isn't perfect)

http://mctvconverter.userboard.org/closed-beta-discussion-f2/tvconverter-beta-t6.html

password is beta

If you want a newer version that cleans up after itself better, please let me know, I'm sure he won't mind if I share it with you.

He's been pretty responsive to my feedback, but his last effort failed to fix the renaming issue, yet again

[Osho]

I have a utility that watches a Recorded TV folder, and when WMC is done recording a .wtv file (usually about 10-15 minutes after), or when MC17 finishes recording a .ts file (actually usually during, I think the way MC records in eparts in extra folders confuses the utility?) it will 'convert' it to .ts into another folder, on the network, then it runs comskip on it, creates the .edl file* and an .xml file located in a third location, which WMC and the DTBAddin read from and I get automatic commercial skipping when watching the original WMC files in WMC, and I get automatic commercial skipping of the ts recordings (the converted ones and the originals) in MC17.

* the .edl file is currently still being misnamed to match the original filename, not the renamed/converted file.  it takes 2 seconds to fix, then it's automatic.  I'm hopeful this is resolved in the next day or 2.

This does result in duplicates of your files.  Your workflow might cause you to want to delete either copy, and this will do that for you.  it still involves a bit of work with the comskip.ini, but you need to do that to get accurate skipping anyway.  That's on my list, but it's pretty close as it is now.

a 30 minute SD program takes about 5 minutes to completely finish the process and move all the files, YMMV

a 30 minute HD program takes about 13 minutes to completely finish.

MC-TVConverter  the current version can be had here  -  http://mctvconverter.vivolum.net/

He also has a beta board where you can download a version that has commercial cutting separated from commercial skipping, if you want to just get the .edl, but not actually cut the converted file (comskip isn't perfect)

http://mctvconverter.userboard.org/closed-beta-discussion-f2/tvconverter-beta-t6.html

password is beta

If you want a newer version that cleans up after itself better, please let me know, I'm sure he won't mind if I share it with you.

He's been pretty responsive to my feedback, but his last effort failed to fix the renaming issue, yet again

Awesome - this is very detailed and very useful ..

Now, if only we could integrate this whole process into MC17 itself

Thanks,
Osho