Option to deinterlace interlaced material (for extras)? material, standard, problem, camera

andy

Unlike the main film that is shot at 24 fps, most of the extras are shot with a standard NTSC camera at 29.97 fps.  The NTSC extras are interlaced with 60 unique fields per second.  This is unlike film material that is interlaced, which is constructed from a progressive 24 fps (using Telecine).  In that case, each field does not correspond to a unique moment in time, as does NSTC.

The problem with true interlaced material (such as those found in extras), is that you can't apply 3:2 pulldown on it and feed it into CCE as a progressive stream.

I've found that CCE does a really poor job with interlaced material, even if you tell it to encode per field.  So I'm wondering if an option in DVDRB to deinterlace these interlaced extras (using some smart deinterlacing algorithm) before feeding them into CCE would be beneficial.  I'm not sure how this is being handled in DVDRB right now.  The quality of extras really takes a hit when quot;half-d1, half qualityquot; is selected, and deinterlacing the source first will greatly improve the encoded output.

That has been discussed in these threads.. try quot;searchquot;  

My own personal opinion is that you should never try to deinterlace a true interlaced source.  You should reencode it in the same manner as it was recorded.

The one exception would be when you will always play it back on a computer (never a standalone player).hi jdobbs,

i respect your recommendation.  if there were no re-encoding involved, then yes it makes perfect sense to leave the video interlaced.  However, when we know that the MPEG2 encoder (CCE) will not handle the interlaced material well and output a poor quality encoded file, then why leave it as interlaced?

I suppose their are provisions in MPEG2 to do field based encoding.  In this case it could do motion estimation on succesive fields and do DCT quantization on the individual fields.  This way the sharp jaggies that result from interlaced frames will not introduce unnecessary noise, since we know MPEG2 does a lousy job with high frequencies, or sharp areas of a picture.  Perhaps CCE is doing just this, but it doesn't seem to do a good job at it.

When I apply FieldDeinterlace() of the DECOMB filter package before encoding it in CCE the encoded video is remarkably better in quality.  Looks much better on my standalone DVD player as well.  I know I'll be adding FieldDeinterlace to all of my interlaced extras in future RB builds.  Would be nice to have an option to do it automatically. :-)

even good deinterlacers tend to have issues duplicating high motion scenes. i used to deinterlace but i switched. i prefer to have perhaps a slightly lower visual quality in trade for haveing smoother motion (at viewing distances i notice poor motion much easier than a tad more mosquito noise)

Your right, CCE really isnt the best at interlaced material.  And if the dvd (or extras) are poorly interlaced or hybrid or telecined.  Then some sort of processing can usually be done and have the result be very nice.  However, if the source is true interlaced then a FieldDeinterlace (or any other deinterlacing filters) can serve to distort the picture more than is good for it.  I think mostly JDobbs point is that unless your using your PS2 to play your dvd's, then your dvd players hardware deinterlacing chip will do a much better job at doing the deinterlacing.  So for the most part deinterlacing isnt really recommended because either your tv or your dvd player will do better job at it than some avs filter.  Now again this doesnt apply to really tricky source where they might trip up your dvd player, but for the most part on truly interlaced source you really shouldnt deinterlace as it is just that much more changes your making to the original video stream.  I feel the less change the better (usually).

EDIT:  also, for tough interlaced encodes you could try using Procoder.

I wouldn't assume that your TV's line doubling/ deinterlacing scheme is more sophisticated than the avs filter.  The filters implemented in software, where it doesn't have to be done in real-time, can be far more effective.

But even if they were equally as effective, doing the deinterlacing prior to encode, which spares CCE from butchering the encode is where the real savings comes from.

This is a mildly off-topic question for jdobbs, which has probably been answered before.  From what I remember, most segments of the VOBs are stored at 24 fps w/ an appropriate flag.  If someone is watching this DVD on his old TV that doesn't support progressive scan, the DVD player will apply telecine to create the interlaced 29.97 fps before sending it out.  But don't DVD manufacturers get tricky and store some segments at 29.97 fps (telecined), even though the original was 24 fps for this segment?  I suppose the DVD player will apply 3:2 pulldown, so the progressive frames can be reconstructed for TVs that support it.  This switching between progressive and interlacing was largely to deter bootleggers I imagine.

So the question is, how does RB handle such segments?  Ideally 3:2 pulldown would be applied to these segments before sending it off to CCE for encode, and then the flag would be changed to progressive.  This would greatly improve compression, since only 24 fps rather than 29.97 fps are needed to be encoded and since CCE doesn't handle interlaced material well anyways.

dvdrb encodes frame for frame what was in the original. so if its 23.976 that is soft telecined to be 29.97 (pulldown) then 23.976 is encoded and the pulldown is added. if its 29.97 then it is encoded at 29.97.. if the source is mixed it doesnt matter .. all real frames are encoded and then the flags are reapplied.

So the question is, how does RB handle such segments? Ideally 3:2 pulldown would be applied to these segments before sending it off to CCE for encode, and then the flag would be changed to progressive. This would greatly improve compression, since only 24 fps rather than 29.97 fps are needed to be encoded and since CCE doesn't handle interlaced material well anyways.

now there is the scenario that i think you are referring to (although didnt describe well.. you would NOT want to 3:2 pulldown *telecine* material before sending it to cce) --when 23.976 material is encoded at 29.97 (hard telecine) in these cases since encoding all 'real' frames leads to 29.97 output.. that is what you get. what i think your trying to say is it should be inverse-telecined (ivtc) which *should* get you back to 23.976. this has been discussed in other threads (search for ivtc and target the dvdrb forum)

I wouldn't assume that your TV's line doubling/ deinterlacing scheme is more sophisticated than the avs filter.
I don't think that was his point.

A true interlaced source was recorded as interlaced.  That means that the fields are also recorded in real time -- there is a temporal difference between fields.  So any type of blending or deinterlacing is going to distort the original and result in a less-than-optimal picture.  The hardware in the DVD player will play that back exactly as recorded (a field at a time) -- so it is better left alone.

As for hard-telecined sources that were originally film -- yes, theoretically performing inverse telecining could give you a better picture,  but that assumes a clean source and perfect pulldown detection.  The chances are that if the original authors cared about quality it wouldn't be hard telecined to begin with...

For most cases it is better to let DVD-RB do it's job and leave the decision to the software.

However, when we know that the MPEG2 encoder (CCE) will not handle the interlaced material well and output a poor quality encoded file, then why leave it as interlaced?

I mainly backup DVD's of TV shows and I have noticed no difference between the interlacing on the orginal versus the backup.  I'm using a cheapo progressive DVD player and you can definetely see the interlacing effects, but like I said its identical on both the original and the backup.  I'm using CCE version 2.7.  Is it possible that you're using an older version that doesn't handle interlacing correctly?  CCE has come a long way since 2.5

Originally posted by jdobbs
A true interlaced source was recorded as interlaced.  That means that the fields are also recorded in real time -- there is a temporal difference between fields.

Completely understood, as my first post indicated when I mentioned that each field is shot at a unique moment in time.

So any type of blending or deinterlacing is going to distort the original and result in a less-than-optimal picture.

I don't contend that deinterlacing creates an optimal picture.  What I contend is that quot;Interlaced Source-gt;Deinterlacer-gt;CCE-gt;watching on your TVquot; is better than quot;Interlaced Source-gt;CCE-gt;watching on your TVquot;.  If CCE re-encoding was not part of the equation, than there would certainly be no reason to deinterlace.

The hardware in the DVD player will play that back exactly as recorded (a field at a time) -- so it is better left alone.

This is incorrect.  If you happen to still be watching your DVDs through an interlaced output (such as Svideo or composite), into an older TV, then yes, you will be watching a movie in which every field is drawn 1/60 second (in which case you could benefit from leaving the material as interlaced).  But if you're watching a DVD at 480p (60 full frames per second), then any interlaced material must be constructed into progressive frames.  So your DVD player will have to apply the same quot;less-than-optimalquot; techniques when constructing progressive frames out of truly interlaced material.  Of course, even progressive films need to played at the 60 Hz VSYNC of any TV.  So 24 progressive fps need to map to 60 progressive fps.  For telecined 29.97 fps material, inverse-telecine would first be applied to construct 24 progressive fps, and then those frames would be replicated to map to 60 fps.

As for hard-telecined sources that were originally film -- yes, theoretically performing inverse telecining could give you a better picture,  but that assumes a clean source and perfect pulldown detection.  The chances are that if the original authors cared about quality it wouldn't be hard telecined to begin with...

I was going on memory, so I may be wrong that most DVDs mix some hard telecined segments into the DVD.  In your experience jdobbs, have you seen much of this?  That's good if there isn't much of that.

But if you're watching a DVD at 480p (60 full frames per second), then any interlaced material must be constructed into progressive frames.
I'm pretty sure interlaced sources don't typically play back at 480p, that only works for progressive sources -- I wouldn't argue that, though, as I'm sure there could be different ways to do it.  No matter what -- you never get 60 full frames per second from a DVD -- only 30 (29.97).

As for hard telecined -- pretty much only in the previews, and I think you'll find them very difficult to inverse telecine.  There are a few older movies that you may see it -- not much any more.  Overall it's rare (at least here in Region 1).

Originally posted by jdobbs
I'm pretty sure interlaced sources don't typically play back at 480p, that only works for progressive sources -- I wouldn't argue that, though, as I'm sure there could be different ways to do it.  No matter what -- you never get 60 full frames per second from a DVD -- only 30 (29.97).

If you're watching a DVD with a progressive scan DVD player with YPbPr output to a progressive-scan capable TV (e.g. an HDTV), then the DVD will be played at 480p 60 frames per second.  Of course the DVD material is not stored at 60 fps, so the DVD player has to do some trickery to create 60 fps.  60 / 24 is 2.5, so for every 2 frames of DVD material it must create 5.  It does this by showing the first frame 3 times, the second frame 2 times, with this pattern repeated.  It must output 60 fps, as the vertical sync of the TV is 60 Hz.

As for hard telecined -- pretty much only in the previews, and I think you'll find them very difficult to inverse telecine.  There are a few older movies that you may see it -- not much any more.  Overall it's rare (at least here in Region 1).

That's good news.

Originally posted by bennynihon
If you're watching a DVD with a progressive scan DVD player with YPbPr output to a progressive-scan capable TV (e.g. an HDTV), than the DVD will be played at 480p 60 frames per second.  Of course the DVD material is not stored at 60 fps, so the DVD player has to do some trickery to create 60 fps.  60 / 24 is 2.5, so for every 2 frames of DVD material it must create 5.  It does this by showing the first frame 3 times, the second frame 2 times, with this pattern repeated.  It must output 60 fps, as the vertical sync of the TV is 60 Hz.
  Ok, that works.  Of course you're talking about film output (24fps).  But (being the sceptic I am) you know what I call it when you repeat the same frame twice at 60fps?  I call it 30.

bennynihon-

The most common DVDs where I've seen a mix of soft telecine and hard telecine (interlaced encoding mixed with progressive encoding of what was originally film sources) are anime DVDs. It has nothing to do with deterring bootleggers though, as you speculated earlier. For the reasons you described earlier, these can benefit from IVTC prior to encoding, as can film sources hard telecined and encoded entirely interlaced. I'm doing one now, following the nice guide that Sir Didymus wrote here:

showthread.php?s=amp;threadid=88266

However, he's wrong about one thing, and I do  another thing differently. I wouldn't reencode a DVD-RB produced .m2v, because then you're following an encode with another reencode of what was already encoded. Does that make any sense? I would run the whole DVD through DVD-RB first, and then go back to the original source vobs to do my second IVTC'd encode, make it the same size as the movie .m2v that DVD-RB produced, mux it back into the DVD-RB produced DVD,and pocket the 20% improvement in quality and the better playback on most Progressive Scan DVD Players.

The place where he's wrong (he's a PAL guy after all ), is in saying in point 2 that the cell times have to be reduced by 0.8. They don't, and doing so messes up the chapters. They finish way before the end of the movie. Since the film stays the same length, the cell times also stay the same.

However, other than that, it's a nice and accurate guide. It does mean encoding the movie (or extras, or whatever else should and can be IVTC'd) twice, and requires extra time and work, but, for me anyway, the quality boost is worth it. In addition, if you have a crummy Progressive Scan DVD player (as evidently you do), it'll make for much nicer viewing on your HDTV. I've done this several times now, and it's fairly easy, following the steps as laid out by Sir Didymus.

SD, if you read this, much thanks to you for laying it out so clearly for this DVD backup noob. And as you also said, thanks to jsoto for making VobBlanker and PGCDemux, without which this whole process would be much harder. And thanks to mpucoder for muxman, and, last but by no means least, to jdobbs for DVD-RB (surprised, jdobbs?).

Hi manono !

Well, it sound very nice to ear at least one person in the world is getting real-and-practical benefits [quality benefits] from that little guide...

... The place where he's wrong (he's a PAL guy after all ), is in saying in point 2 that the cell times have to be reduced by 0.8. They don't, and doing so messes up the chapters. They finish way before the end of the movie. Since the film stays the same length, the cell times also stay the same.

Yeah! Thank you for understanding my very troublesome position as a PAL man discussing of IVTC with really expert people in this nice forum... And thanks a lot for pointing out this. That's obvious [now you are evidencing it...]. Chapters points refere to displayed frames, not to encoded... I will edit asap the other thread... BIG theorical fault on my side...

I wouldn't reencode a DVD-RB produced .m2v, because then you're following an encode with another reencode of what was already encoded. Does that make any sense? I would run the whole DVD through DVD-RB first, and then go back to the original source vobs to do my second IVTC'd encode, make it the same size as the movie .m2v that DVD-RB produced, mux it back into the DVD-RB produced DVD,and pocket the 20% improvement in quality and the better playback on most Progressive Scan DVD Players.

Infact. Any suggestion to encode in cascade a movie is a mistake!!! Nothing wrost than this!!!. The way I suggested [well, the way I intended to suggest...] should involve a single encoding session (not two) and it seems DVD-RB being not necessary in the process. In other terms, what is not clear to me in the way you operate is the reason for using DVD-RB: just for having an idea of the movie size ? [Or maybe just for building up a new title skeleton for the process ? Mhhh...]
In my intention, the process should be as follow:

1. Just analyse the source (eventually demuxing the original and counting its frames, you know immediately the average bitrate you need for a target m2v fitting a DVD5)...
2. Apply IVTC to the original source vobs, encoding its video asset based on the bitrate needed;
3. Substitute the cells of A [FILE MODE RIPPED COPY OF] the original, with VobBlanker, as described in the guide...

Cheers,
SD

Edit: just changed the guide; I undestood what happened [again a fault on my side]: I was supposing, erroneously, that it was possible to carry out the IVTC process without reencoding the movie... So I suggested to encode it [just once] after the IVTC, using DVD-RB... But this was simply a mistake: since in IVTC you need to remove some frames [just the ones, hopefully, containing redundant video], then the re-encoding of the video asset is necessary... Now the guide is corrected, I hope...Hi Sir Didymus-

I figured you might stumble across this thread. Maybe I did misunderstand you. When you said in point 8, quot;8. Having preprocessed in such a way the source title, it should be 20% smaller... You may now try to use DVD-RB or some other compression toolquot;, that said to me that you create a fresh IVTC'd and pulled down .m2v and mux it back into the DVD, and then run the DVD through DVD-RB.

For example, I have now a DVD whose original size is about 7.5 GB, with 5.5 GB for the movie and about 2 GB of extras. Yes, if I do a movie only backup, I think I can shrink it down to DVD5 with nearly the same quality as the original size just with the IVTC alone (or close to DVD5, anyway). But if I want the complete DVD, I'll still run it through DVD-RB. That will mean encoding the main movie files twice, since DVD-RB doesn't (yet) support IVTC.

I see now. You've been editing while I've been writing. Yes, IVTC requires a complete reencode. I'm a lazy guy, and could reencode all the video (movie + extras) following your guide, thus encoding just once, but for the extras, that I'm not so particular about, I'd let DVD-RB do it. So, doing it that way, DVD-RB does the whole DVD once, and then later I go back and encode the movie from the original vobs and fit it back into the DVD-RB created DVD. That's one way, anyway.

I'm not complaining. The best way to learn is by doing. When I tried that 0.8 thing on the cell times, after muxing, the audio was synched and played fine, the subs were synched and played fine, and only the chapter points were off. So then I got to thinking that 1) cell times corresponded to chapter points (didn't know that before), and 2) cell times are based either on time stamps, or on the telecined frame numbers (not the IVTC'd frame count), and shouldn't be adjusted unless the film length changes (such as with a PAL to NTSC conversion). I was going to PM you with my experiences, but then this thread came up, and I thought I'd thank you in public. Thanks.

Originally posted by manono
...I'd let DVD-RB do it. So, doing it that way, DVD-RB does the whole DVD once, and then later I go back and encode the movie from the original vobs and fit it back into the DVD-RB created DVD. That's one way, anyway.

Ok. Now I see and understand. What you do is totally proper...
At the end, the core of the process is still DVD-RB...  
What's really amazing is the flexibility of this great application from Jdobbs...