Problem with X264's predicted motion vectors avc, textures, average, desired

andy

For a long time I was baffled by a problem many of my avc cqm's faced when used in X264. They would preserve detail and texture quite well, but there would often be sliding textures and missplaced blocks in the encodes. I beleive this is caused by the higher average quant these cqm's require to hit a desired bitrate, at higher quants X264 uses a higher amount of predicted mv's istead of actual mv's, which results in motion errors. Is there anyway to reduce the amount of predicted mv's X264 is allowed to use?

x264_cqmskip.0.diff
I tried this when I first implemented cqm, and for the jvt matrix it produced way too few skips, but maybe it's better with current code or current matrices.

I really don't have any idea how to compile X264, would it be possible for this to be implemented as a switch in the cvs untill a better alternative is found?

I have noticed a similar problem to *.mp4 guy, but like him, I am in the same boat when it comes to patching and compiling x264 (and making it work, I can produce a dud compile just fine )

I think having it as a switch was a good suggestion, providing it won't break a whole lot of other things   I'd certainly be willing to test it out.

Cheers

I tested some more, and even when I use the same exact settings and cq mode low bitrate matrices still use a lot less predicted mv's, why is that?

You mean Skip blocks, right? Skip = predicted mv and no residual.
All sane cqms, low-bitrate or not, use a smaller quantizer for dc than the flat matrix does. Smaller quantizer means the quantized dc is more likely to be nonzero. Nonzero coefficients imply not a Skip block. This might or might not be counterbalanced by the larger quantizer of some of the other coefficients, but apparently not in your cqm.


Originally Posted by akupenguinYou mean Skip blocks, right? Skip = predicted mv and no residual.
All sane cqms, low-bitrate or not, use a smaller quantizer for dc than the flat matrix does. Smaller quantizer means the quantized dc is more likely to be nonzero. Nonzero coefficients imply not a Skip block. This might or might not be counterbalanced by the larger quantizer of some of the other coefficients, but apparently not in your cqm.

My problem is that a high bitrate matrix of mine gets more skiped blocks at the same cq then a low bitrate one, even though all of its ac and dc coeficients are lower.

While I don't know what caused your extra skips...
Why would you even make a matrix with all of the coefficients less than 16? You're just confusing everything that thinks it knows what qp is (e.g. the skip decision, the deblocking filter, rate-distortion optimization, ratecontrol, any potential post-processing filter someone might write, etc).
All sane cqms should produce approximately the same bitrate at the same qp. If they don't, then increase the cqm coefficients and decrease the qp.
(Of course the converse the converse also applies to quot;low bitratequot; matrices. quot;low bitrate matrixquot; should mean quot;tuned for high qpquot;, not quot;produce lower bitrate at normal qpquot;.)

There was a legitimate reason in mpeg4asp to mess with the meaning of a qp: Ratecontrol was very coarse if you had to jump between qp=1,2,3, so you might want to halve the cqm and double the qp, and then you're at qp=2,4,6, which have extra steps in between. That excuse does not apply to h264, with its logarithmic qp scale.


Originally Posted by akupenguinWhile I don't know what caused your extra skips...
Why would you even make a matrix with all of the coefficients less than 16? You're just confusing everything that thinks it knows what qp is (e.g. the skip decision, the deblocking filter, rate-distortion optimization, ratecontrol, any potential post-processing filter someone might write, etc).
All sane cqms should produce approximately the same bitrate at the same qp. If they don't, then increase the cqm coefficients and decrease the qp.
(Of course the converse the converse also applies to quot;low bitratequot; matrices. quot;low bitrate matrixquot; should mean quot;tuned for high qpquot;, not quot;produce lower bitrate at normal qpquot;.)

There was a legitimate reason in mpeg4asp to mess with the meaning of a qp: Ratecontrol was very coarse if you had to jump between qp=1,2,3, so you might want to halve the cqm and double the qp, and then you're at qp=2,4,6, which have extra steps in between. That excuse does not apply to h264, with its logarithmic qp scale.

I was refering to my own low bitrate matrix, not the standard one, so not all of the values were lower then 16. However it was significantly less aggressive. akupenguin you do realise that if you have a matrix that has has coeficients that are exactly half of another matrice's coeficients that at the exact same bitrate they will produce significantly different pictures that are not the result of ratecontrol differences in mpeg4-asp. It apears the same is true of mpeg4-avc, though it is hard to tell with all of the related dificulties.

[Edit]
Here is an example, The settings used in the encodes are exactly the same in both cases, cq mode was used(to avoid ratecontrol issues), the filesizes are close enough to make no difference on such a short clip. I would have made an example using X264, but it has the afformentioned problems that would make comparisons inconclusive.Code:
8,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,33

8,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
all32's@Q4 example clipCode:
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8

8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
all8's@Q9.65 example clip

The reason that the inter DC coeficient in the all 32's matrix is 8 is because intra dc coeficients are locked at 8 in mpeg4-asp, so it makes the comparison more accurate by keeping all the DC coeficients at 8.


Originally Posted by *.mp4 guyyou do realise that if you have a matrix that has has coeficients that are exactly half of another matrice's coeficients that at the exact same bitrate they will produce significantly different pictures that are not the result of ratecontrol differences in mpeg4-asp. It apears the same is true of mpeg4-avc, though it is hard to tell with all of the related dificulties.

Yes, it's not just ratecontrol, but also due to
Originally Posted by akupenguinthe skip decision, the deblocking filter, rate-distortion optimization
and in mpeg4-asp also your locked dc coef.

That said, all of those issues are avoidable in h264 if I were to tune the encoder with full accounting for cqms. An encode with the default matrix is exactly the same as an encode with half the cqm entries and +6 qp and -3 deblocking strength and modified lambda, deadzone, chroma_qp_offset, skip and decimation thresholds, etc.
My point was that if you modify cqm too far and don't modify all the other parameters to match (and they aren't all tweakable in x264), then you'll get a suboptimal encode.


Originally Posted by akupenguinYes, it's not just ratecontrol, but also due to  and in mpeg4-asp also your locked dc coef.

That said, all of those issues are avoidable in h264 if I were to tune the encoder with full accounting for cqms. An encode with the default matrix is exactly the same as an encode with half the cqm entries and +6 qp and -3 deblocking strength and modified lambda, deadzone, chroma_qp_offset, skip and decimation thresholds, etc.
My point was that if you modify cqm too far and don't modify all the other parameters to match (and they aren't all tweakable in x264), then you'll get a suboptimal encode.

Well I suppose it would be nice if some of those options (Deadzone and decimation thresholds, would be particularly usefull)  were available, it would make it easier to make X264 behave properly at high bitrates. I realize in theory  having overal lower matrix values shouldn't change anything, but in practice its the only way to affect things like deadzone and decimation that have a significant effect on picture quality. As long as those options are locked and X264 goes into fits when you try to use a matrix to affect them it makes things more difficult then they have to be.

[edit] For instance being able to tweak the deadzone parameter would help with blocking and lack of detail at higher bitrates.

I would like to agree with *.mp4.guy.
I have also encountered the same problem.  At first I thought it might have been a problem with the source I was using and/or a decoder issue.  I never got round to finding the real cause of the problem.

I was testing a very high datarate matrix.Code:
INTRA4X4_LUMA =
1,2,3,5,
2,3,5,6,
3,5,6,7,
5,6,7,8

INTRA4X4_CHROMAU =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTRA4X4_CHROMAV =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTER4X4_LUMA =
1,2,3,5,
2,3,5,6,
3,5,6,7,
5,6,7,8

INTER4X4_CHROMAU =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTER4X4_CHROMAV =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTRA8X8_LUMA =
1,1,2,3,3,4,4,5,
1,1,3,3,4,4,5,6,
2,3,3,4,4,5,6,7,
3,3,4,4,5,6,7,8,
3,4,4,5,6,7,8,9,
4,4,5,6,7,8,9,10,
4,5,6,7,8,9,10,11,
5,6,7,8,9,10,11,12

INTER8X8_LUMA =
1,1,2,3,3,4,4,5,
1,1,3,3,4,4,5,6,
2,3,3,4,4,5,6,7,
3,3,4,4,5,6,7,8,
3,4,4,5,6,7,8,9,
4,4,5,6,7,8,9,10,
4,5,6,7,8,9,10,11,
5,6,7,8,9,10,11,12
The result was blocking.

edit: here is a sample
files/17235790/source.rar.html

The result is not blocking, but moving of pixels.
Not sure excatly how to explain.


Originally Posted by AudionutI was testing a very high datarate matrix.

No, you were using a quot;lets screw the codec over and see if it can still produce a recognizable picturequot; matrix.

quot;High bitrate matrixquot; is not at all related to quot;matrix with small coefficientsquot;! (in h264, of course. I don't speak for xvid.)

A quot;high bitratequot; matrix is designed to preserve all the film grain, noise, and other crap that some people think makes a movie look authentic once you've encoded all the real content with acceptable quality. A quot;low bitratequot; matrix is designed to filter out all the details while leaving a minimum amount of artifacts. That design is all about selecting a balance between coefficients, and nothing to do with the absolute value of the coefficients. Very large or very small absolute values just confuse the codec, nothing more, nothing less.
No, I don't plan to change that. RDAQ could sorta hide your errors, but it's not the right answer.P.S. The exact failure mode in your case was: In order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so. Then it's encoding a frame at qp=50, and x264 says quot;hey, I know how to encode at qp=50, that's where I have to throw away all the information I possibly can because this video is way too complex for the target bitrate.quot; So during motion estimation, it very agressively picks large partitions and small mv deltas and so on.

Crap, I forgot to post the settings and log file.Code:
Starting job job1 at 2:41:46 PM
Job is a video job. encoder commandline:
--crf 18 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 7 --trellis 2--analyse all  --8x8dct --ipratio 1.2 --qcomp 1.0 --direct auto --me umh --threads 2 --thread-input --cqmfile quot;F:\AVC_HIGH.cfgquot;--progress --output quot;E:\STAR_WARS_EPISO\entire movie\source.mkvquot; quot;E:\STAR_WARS_EPISO\entire movie\source.avsquot;
successfully started encoding
Processing ended at 2:46:52 PM
----------------------------------------------------------------------------------------------------------

Log for job job1

avis [info]: 704x288 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:9     Avg QP:17.78  size: 56250  PSNR Mean Y:49.44 U:51.34 V:51.22 Avg:49.97 Global:49.89
x264 [info]: slice P:255   Avg QP:18.02  size: 28736  PSNR Mean Y:48.65 U:49.99 V:49.82 Avg:49.03 Global:49.00
x264 [info]: slice B:737   Avg QP:19.69  size:  1738  PSNR Mean Y:46.22 U:49.16 V:48.83 Avg:46.95 Global:46.83
x264 [info]: mb I  I16..4:  0.2% 64.4% 35.4%
x264 [info]: mb P  I16..4:  0.0%  1.6%  0.1%  P16..4: 55.3% 22.6% 19.4%  0.7%  0.3%    skip: 0.0%
x264 [info]: mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  2.2%  0.7%  1.4%  direct: 0.2%  skip:95.5%
x264 [info]: 8x8 transform  intra:73.3%  inter:88.8%
x264 [info]: direct mvs  spatial:91.6%  temporal:8.4%
x264 [info]: ref P  63.9% 13.3% 11.3%  5.0%  6.5%
x264 [info]: ref B  82.1%  8.9%  4.7%  2.4%  2.0%
x264 [info]: PSNR Mean Y:46.869 U:49.394 V:49.101 Avg:47.510 Global:47.307 kb/s:1821.15

Actual bitrate after encoding without container overhead: 1822.40
desired video bitrate of this job: 18 kbit/s - obtained video bitrate: 1825.22617382617 kbit/s
----------------------------------------------------------------------------------------------------------

I take it you did not look at the sample I posted.

To me, a high bitrate matrix is one that requires more bitrate at a constant quant, then the default matrix.
But you're saying that, a high bitrate matrix is one that preserves noise etc.  To me, that would be a quot;matrix that preserves detailsquot;.
Originally Posted by akupenguinVery large or very small absolute values just confuse the codec, nothing more, nothing less.

So the real question then is.  The acceptable range for coefficients is not 1-255, but instead it is quot;insert your answer herequot;.

edit:  orOriginally Posted by akupenguinP.S. The exact failure mode in your case was: In order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so. Then it's encoding a frame at qp=50, and x264 says quot;hey, I know how to encode at qp=50, that's where I have to throw away all the information I possibly can because this video is way too complex for the target bitrate.quot; So during motion estimation, it very agressively picks large partitions and small mv deltas and so on.

Are you saying that when engaged in CRF mode, x264 still tries to maintain a datarate based on the datarate that the standard matrix produces?


Originally Posted by akupenguinquot;High bitrate matrixquot; is not at all related to quot;matrix with small coefficientsquot;!Originally Posted by akupenguinIn order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so.

Clarification.  Why would x264 have to increase the qp to 40 or 50 with my matrix if it isn't the result of small coefficients?

Increasing the qp at constant bitrate is a direct result of the small coefficients.
But just because your matrix produces a huge bitrate at constant qp, does not mean that it looks better than the flat matrix when given the same huge bitrate.

Or am I misusing the term, and everyone else says quot;high bitrate matrixquot; to mean quot;looks like crap at low bitratesquot; without any reference to what it might actually be good for?

So the real question then is. The acceptable range for coefficients is not 1-255, but instead it is quot;insert your answer herequot;.

Off the top of my head: range should be maybe [8,40], with an average of no less than 16. The high frequencies might tolerate a little bigger, but if you set them too high you might as well downscale instead.

Others opinion's as to what constitutes a high bitrate matrix would be welcome.

As I stated before, my idea of a high bitrate matrix, is one that requires a higher bitrate than the standard matrix at any given quant.Originally Posted by akupenguinIncreasing the qp at constant bitrate is a direct result of the small coefficients.

Agreed.  Or put differently.  Using smaller coefficients in a matrix requires more bitrate at the same quantizer than higher coefficients.  Hence, why I called my matrix quot;high bitratequot;, because it uses small coefficients, hence higher bitrate at a constant quantizer, or put differently, higher quantizer at a constant bitrate.Originally Posted by akupenguinBut just because your matrix produces a huge bitrate at constant qp, does not mean that it looks better than the flat matrix when given the same huge bitrate.

I never stated otherwise.  I mean no disrespect to yourself, as I greatly appreciate your contribution to x264.

However, it seems as though you are of the opinion that high bitrate matrix = high quality.
I belive that there is a difference between the two.

High bitrate, means excatly that, More bits, nothing more, nothing less.

High quality, means, to my eyes, the resulting video looks better.

I never gave the impression that my matrix was the, quot;be all, and end all of matricesquot;, quot;the shitquot;, quot;why use anyone else'squot;.
I posted in this thread to inform yourself, that when using my matrix, errors in the video stream were the result.Originally Posted by akupenguinyou were using a quot;lets screw the codec over and see if it can still produce a recognizable picturequot; matrix.

One of the reasons for having open source software is to test it's limitations, and what it can and can't do.  Correct?Originally Posted by akupenguinOff the top of my head: range should be maybe [8,40], with an average of no less than 16. The high frequencies might tolerate a little bigger, but if you set them too high you might as well downscale instead.

Thankyou.  Now I know.  Fell free to bitch slap me if this has already been posted.

High bitrate, means excatly that, More bits, nothing more, nothing less

You do agree, however, that that concept is completely useless for h264, since quantizer 1 for the default matrix already gives an insane bitrate. With ASP, quantizer 2 was sometimes not able to deliver a wanted quality, hence a high bitrate matrix might have been useful. With AVC, such a concept is useless.

With ASP, a properly tuned quot;high bitratequot; matrix could still provide good quot;qualityquot; at high quantizers.
So that while say the h.263 matrix was usefull for quants 2-4 with reasonable quality, a tuned matrix was usefull for quants 2-6.
Didee pointed out for example that his sixofnine matrix provided extra (quality) steps.

What I am trying to do is, create a matrix for AVC that does virtually the same thing.
If you want to encode at quant 1 or even quant 10 which is very high, you have a matrix that is going to keep all the fine details and produce an outstanding encode.  But at the same time, will not punish you for using quants up to say 30.

Then reread the thread, especially when aku clearly underlines the fact that the quot;sixofninequot; concept is completely useless in AVC, since the quantizer scale is logarithmic and quite fine.

You seem not to be aware of the following : whatever the quantizer and the matrix, raising the quantizer by six will multiply the coefficient by 2 and the size (roughly) by 2. Dividing the coefficients by 2 won't give you a higher precision. It'll simply act as though you were using a -6 quantizer.