Hi
I've made some little test files for a movie that I want to compress (at 512x288).
Now I want to compare this tests. I've searched the forum and came to know about the possibility to make comparisons using avisynth. So I've prepared this script (using the latest release of avisynth 2.5):Code:
clip1=AviSource(quot;D:\test5.aviquot;)
clip2=AviSource(quot;D:\test8.aviquot;)
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;)
Then I opened Virtualdubmod 1.5.1.1, loaded the script, and then clicked on play. Then the log file appeared. Is everything OK till now? did I make any mistake?
Now I'm here with my log file, but I can't understand if it's correct and how to read it.
I mean, hov can I understand wich of the two files is the better in quality? I know that is difficult to judge the relative quality of two files, especially when the differences in used settings are very few. I've read many discussions (in this forum) about PNSR, and about how it can be considered an objective parameter fore someone, and how not for someothers.
So I ask yur help. How can I have an idea of the best encoding result, apart from using my eyes?
thanks a lot
P.S. here I post the results for the firts 30 files, so that you can have an idea and comment my results (is it mine a god encoding? I mean, looking at the PSNR)...
Comparing channel(s) YUV
Mean Max Max Absolute Mean Pos. Neg. Frame Dev. Dev. Dev. Dev. PSNR (dB)
----------------------------------------------------- 0 0.2783 +0.2064 5 -4 53.0141 1 0.2826 +0.2023 6 -5 52.9264 2 0.3052 +0.2256 6 -6 52.5922 3 0.3167 +0.2356 5 -5 52.4680 4 0.3282 +0.2543 5 -5 52.3366 5 0.3159 +0.2350 7 -6 52.4559 6 0.3128 +0.2332 7 -6 52.5817 7 0.3141 +0.2365 6 -5 52.5178 8 0.3259 +0.2524 5 -5 52.3511 9 0.3260 +0.2484 5 -6 52.4049 10 0.3252 +0.2475 6 -6 52.3900 11 0.3147 +0.2316 6 -6 52.5380 12 0.3145 +0.2316 5 -6 52.5381 13 0.3270 +0.2456 12 -10 52.3657 14 0.3331 +0.2602 4 -6 52.3248 15 0.3225 +0.2431 5 -6 52.4776 16 0.3194 +0.2437 5 -6 52.4627 17 0.3209 +0.2291 5 -5 52.3673 18 0.3362 +0.2593 4 -5 52.2516 19 0.3392 +0.2644 5 -5 52.2149 20 0.3338 +0.2601 5 -5 52.2999 21 0.3300 +0.2539 4 -4 52.3591 22 0.3243 +0.2469 5 -4 52.3914 23 0.3385 +0.2645 5 -5 52.2216 24 0.3383 +0.2527 5 -4 52.2091 25 0.3422 +0.2468 5 -4 52.1016 26 0.3348 +0.2414 5 -5 52.2133 27 0.3386 +0.2503 5 -5 52.1951 28 0.3434 +0.2678 6 -4 52.1553 29 0.3474 +0.2725 6 -5 52.1113 30 0.3323 +0.2359 5 -4 52.3372
for help, though you've missed the prototype of the function:
Compare(clip_filtered clip, clip_original clip, string channels, string logfile, bool show_graph)
Therefore, you have to compare your original source (as second clip), ie the material you feed XviD with. If it's a true video source (from AviSource/...) then you can go on. If it's an avisynth script, you have to recreate your source, ie:
[code]
clip_filtered = AviSource(quot;D:\test5.aviquot;)
clip_original = Somesource(quot;somevideo.someextensionquot;).SomeFilterChain(someargs)
Compare(clip_filtered, clip_original, quot;YUVquot;, quot;compare.logquot;)
This way you get to compare all 3 channels of test5.avi with the source. For faster checking, quot;Yquot; should be sufficient, as it's the part that most often matters to the eye.
In that regard, the PSNR values you report are meaningless: they just compare two results of different encoding to each other, while the reference should the original clip.
For your information:
- below 38dB (last column) is generally a very bad result (it depends on your goal though, but video quality clearly tells this is not for storage)
- between 38 and 42, it's average (mean quantizer generally =gt; 3)
- above 42dB, your video should be OK
A good encode should be around 44-46. You hardly can go beyong 48dB (except in particular situations, like black frames) with regular quantizer 2 using MPEG matrix.
Don't take those values for a rule of thumbs: they are just statistical evaluation of the coding efficiency, and can change for a same visual quality (and vice-versa).
for help, though you've missed the prototype of the function:
Compare(clip_filtered clip, clip_original clip, string channels, string logfile, bool show_graph)
Yeah that's the correct syntax
Therefore, you have to compare your original source (as second clip), ie the material you feed XviD with. If it's a true video source (from AviSource/...) then you can go on. If it's an avisynth script, you have to recreate your source, ie:
[code]
clip_filtered = AviSource(quot;D:\test5.aviquot;)
clip_original = Somesource(quot;somevideo.someextensionquot;).SomeFilterChain(someargs)
Compare(clip_filtered, clip_original, quot;YUVquot;, quot;compare.logquot;)
This way you get to compare all 3 channels of test5.avi with the source. For faster checking, quot;Yquot; should be sufficient, as it's the part that most often matters to the eye.
I know this. But the problem is that my video source is in mpeg2 and coded at a different resolution, and avisynth doesn't want to compare the two movies. I even tryed to convert the original movie to RGB uncompressed avi, using the same resolution as test5.avi, but nothing to do. Anyhow that's not the real problem.
In that regard, the PSNR values you report are meaningless: they just compare two results of different encoding to each other, while the reference should the original clip.
For your information:
- below 38dB (last column) is generally a very bad result (it depends on your goal though, but video quality clearly tells this is not for storage)
- between 38 and 42, it's average (mean quantizer generally =gt; 3)
- above 42dB, your video should be OK
A good encode should be around 44-46. You hardly can go beyong 48dB (except in particular situations, like black frames) with regular quantizer 2 using MPEG matrix.
That's what I was looking for, so let's say you reached the goal
I'm new to this kind of... mmm can we call them experiments? So I just wanted to have an idea of what I was doing and which values to refer to. You gave me a comparison meter (hope this is the right expression!!!), and now I can have an idea.
Is there any app that can compare an xvid encoded movie with an mpeg2 and give a PSNR value? I suppose that it's meaningless to compare two movies at different resolutions, isn't it?
Thanks and regards
Movie Maniac®
Originally posted by Movie Maniac®
That's not right, my intention is not to discuss about avisynth usage. I know how to use it and the compare function. I just wanted someone to explain the meaning of that log file in terms of video quality.
OK, though I still believe it fits more in that forum than XviD's one. Yet moderators didn't move it, but I think it would have had more replies and sooner in the AVS usage forum, as it is first bound to an Avisynth function, then a general coding matter that could be used not only for XviD (ie the DivX3/4 forum could have been as suited).
Yeah that's the correct syntax
Thanks, what grade will you give me ?
More seriously, I still don't know what test5 and test8 are: if they are the lossy encoding of the same source file, it won't work. If one is a lossless conversion (keeping the colorspace is important too) of the input to the codec, then fine (as I guess you've done).
I know this. But the problem is that my video source is in mpeg2 and coded at a different resolution, and avisynth doesn't want to compare the two movies.
As you have figured, this is obvious. Compare though may accept (I'm not sure) clips of different famelength/fps.
I even tryed to convert the original movie to RGB uncompressed avi, using the same resolution as test5.avi, but nothing to do. Anyhow that's not the real problem.
I do think it is. But it all depends on what are test5 and test8. I'll let you clarify on that, as my example script seems not to have solved your problem.
You gave me a comparison meter (hope this is the right expression!!!), and now I can have an idea.
I'm glad I've at least solved that problem. Just remember PSNR is an objective quality metric: a Real Video 9 encode may have a better PSNR than the corresponding XviD encode, but in the end you may prefer blocks and sharpness over blurriness. I think you have already this in mind, but you should always double check. For instance, an average PSNT may be good, but if it comes at the cost of a great variation (check the dev field of the input) of PSNR during the comparison, then you may prefer the one of lower PSN value, but lesser deviation.
Is there any app that can compare an xvid encoded movie with an mpeg2 and give a PSNR value? I suppose that it's meaningless to compare two movies at different resolutions, isn't it?
Completely. But compare is totally suited to do this job afaik, you just need to check what you want to compare and if it can be compared.
You'll probably think I'm repeating myself, but I have the feeling you misunderstood me. If you want to check on the (lossy) coding efficiency of a codec at different settings, then you must compare the input (filtered, resized clip, same colorspace as stored by XviD, ie YV12 as probably done by avisynth) that was fed to the codec, to the output of this codec.
Resizing and colorspace conversions introduces loss of details/rounding errors when converted back to original clip's settings (RGB24-gt;YV12 is lossy for instance), so you wouldn't compare the codec's result. My suggestion is then to reuse the script you wrote to encode into XviD, but instead, encode it to VBLE (or maybe HuffYUV, hoping that the ConvertToYV12() you'll need to add for the comparison won't introduce some kind of error), and then compare the VBLE lossless encode to the XviD encode.
Originally posted by Kurosu
If it's an avisynth script, you have to recreate your source, ie:
[code]
clip_filtered = AviSource(quot;D:\test5.aviquot;)
clip_original = Somesource(quot;somevideo.someextensionquot;).SomeFilterChain(someargs)
Compare(clip_filtered, clip_original, quot;YUVquot;, quot;compare.logquot;)
Originally posted by Movie Maniac®
I know this. But the problem is that my video source is in mpeg2 and coded at a different resolution, and avisynth doesn't want to compare the two movies.
---
Is there any app that can compare an xvid encoded movie with an mpeg2 and give a PSNR value? I suppose that it's meaningless to compare two movies at different resolutions, isn't it?
try loading the original full .avs script, you fed xvid with as clip_original!
should work (if i understood everything right )
Originally posted by Kurosu
OK, though I still believe it fits more in that forum than XviD's one. Yet moderators didn't move it, but I think it would have had more replies and sooner in the AVS usage forum, as it is first bound to an Avisynth function, then a general coding matter that could be used not only for XviD (ie the DivX3/4 forum could have been as suited).
Seems now we're getting OT. So you're right. I think that the only reason that I posted in this forum is that I was using xvid to encode my movie
Thanks, what grade will you give me ?
A+
More seriously, I still don't know what test5 and test8 are: if they are the lossy encoding of the same source file, it won't work. If one is a lossless conversion (keeping the colorspace is important too) of the input to the codec, then fine (as I guess you've done).
Well, you're right. They're both lossy encodings. But I wasn't considering the PNSR for the comparison, just the deviation, to see how much the two encodings differ one from the other. Then I tryed to obtain a PNSR comparing each one with the original, but nothing to do... as i sayd.
Avisynth doesn't want let me compare an avi whith an mpeg2, even at the same resolution. So I'll try a looseless compression, as you suggest, and compare the two results.
Compare though may accept (I'm not sure) clips of different famelength/fps.
I can grant for different framelenght.
I'm glad I've at least solved that problem. Just remember PSNR is an objective quality metric: a Real Video 9 encode may have a better PSNR than the corresponding XviD encode, but in the end you may prefer blocks and sharpness over blurriness. I think you have already this in mind, but you should always double check.
Hehehe, ok ok, in the end no one can just tell you: this is a good encoding! since it's you that whatch the movie, and you to decide whether you like a certain quality or not. I think that blur helps a lot, but I don't like to watch movies that looks like if there was an opaque plexiglas film between me and the screen.. if you know what I mean
For instance, an average PSNT may be good, but if it comes at the cost of a great variation (check the dev field of the input) of PSNR during the comparison, then you may prefer the one of lower PSN value, but lesser deviation.
Mmmm, things are getting darker for me. I just know what PNSR is, and now you talk about those strange values... what's this? Can you suggest any (very simple and basilar) document which I can read and learn from? Please consider that I'm an autodidact, not an engineer , so I need basilar infos to start....Completely. But compare is totally suited to do this job afaik, you just need to check what you want to compare and if it can be compared.
My suggestion is then to reuse the script you wrote to encode into XviD, but instead, encode it to VBLE (or maybe HuffYUV, hoping that the ConvertToYV12() you'll need to add for the comparison won't introduce some kind of error), and then compare the VBLE lossless encode to the XviD encode.
As I sayd, I'll try this way. Here it is the script i used to encode:
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
mpeg2source(quot;D:\video.d2vquot;)
crop(4,72,536,432)
LanczosResize(512,288)
so YV12 is built inside the mpegdec3.dll, non eed to convert from colorspace to colorspace. The source is YV12...
I'll let you know any result
Till then, good GP (I'm going to whatch it on TV...)
Originally posted by bond
try loading the original full .avs script, you fed xvid with as clip_original!
should work (if i understood everything right )
Ok
I'll try this asap. Now my computer is at work and the race has just started
Bye
regards
Movie Maniac®
Originally posted by Movie Maniac®
Well, you're right. They're both lossy encodings. But I wasn't considering the PNSR for the comparison, just the deviation, to see how much the two encodings differ one from the other. Then I tryed to obtain a PNSR comparing each one with the original, but nothing to do... as i sayd.
In fact, I don't recall if all of the deviations printed in the logfile are meant in pixels or also in PSNR. More on this later in this reply.
I think that blur helps a lot, but I don't like to watch movies that looks like if there was an opaque plexiglas film between me and the screen.. if you know what I mean
Indeed. Although it's nice to have a PSNR when we can hardly make a difference
Mmmm, things are getting darker for me. I just know what PNSR is, and now you talk about those strange values... what's this? C
Sorry for being too complex to read. First (I might be wrong in this very case), I consider deviation as how much the values in average vary from the average value:
- {10 15 20} have an average of 15
- {5 15 25} has the same average but a greater deviation
The result (with a more reduce scale, for instance PSNR going from 38 to 48) for our case is that the quality changes a lot in the encoded video. This can be as bad as a video getting blocky with an over-quantized I-frame, and thus blocky, and getting smoothed with the following frames. In average for all those frames, you may have a better PSNR than another encode without that problem, but you'll better notice the 3dB PSNR loss on one frame than the average 0.2dB win on all frames.
PSNR is already an average measure quality over a frame: some parts can suffer a lot from lossy coding, while other may have very little differences. And those differences may or may not cause much visual quality loss.
(this was intended as a description for quality measure, not a PSNR bashing - PSNR is still a reliable metric for lossy coding efficiency).
you suggest any (very simple and basilar) document which I can read and learn from? Please consider that I'm an autodidact, not an engineer , so I need basilar infos to start....
I'm very sorry on that point, it's really only math stuff, but I hope the above description made it clearer for you, enough to the point you don't need reading math stuff.
As I sayd, I'll try this way. Here it is the script i used to encode:
The I suggest 2 things:
- as I said before, encode it to VBLE once and for all if you have room on your Hard Drive; you may not gain much in encoding time from using the VBLE encode considering the low level of filtering (decoding VBLE vs filtering)
- use the following scripts (with the modification to the resizer)
#Test5-compare.avs
#To test how well test5 was encoded
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1 = avisource(quot;test5.aviquot;)
clip2 = mpeg2source(quot;D:\video.d2vquot;).LanczosResize(512,288,4,72,536,432)
Compare(clip1, clip2, quot;YUVquot;,quot;test5.logquot;)
#Test8-compare.avs
#To test how well test5 was encoded
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1 = avisource(quot;test8.aviquot;)
clip2 = mpeg2source(quot;D:\video.d2vquot;).LanczosResize(512,288,4,72,536,432)
Compare(clip1, clip2, quot;YUVquot;,quot;test8.logquot;)
(clip2 could have been in both cases Avisource(quot;VBLE-lossless.aviquot;)
Then check the 2 logs.
I think we now got to the solution
While MPEG2DEC3 delivers YV12 I don't think Compare has any support for that yet. So you may have to ConvertToYUY2() on both clips for compare. At least that's what I had to do recently in a similar circumstance.
- Tom
Originally posted by trbarry
[B]While MPEG2DEC3 delivers YV12 I don't think Compare has any support for that yet. So you may have to ConvertToYUY2()...B]
Seems so
Now I'm using this scriptCode:
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1=ConvertToYUY2(AviSource(quot;D:\encoded.aviquot;))
clip2=ConvertToYUY2(mpeg2source(quot;D:\original.d2vquot;).LanczosResize(512,288,4,72,536,432))
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;,show_graph=true)
And I'm waitin' for my logfile. I've taken a look at a piece of the log, and seems that the PNSR is near 50, so it may be a good encoding .The strange thing is that I've tryed the trick of loading as an avi file another avs script, this way:Code:
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1=ConvertToYUY2(AviSource(quot;D:\encoded.aviquot;))
clip2=ConvertToYUY2(AviSource(quot;D:\original.avsquot;))
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;,show_graph=true)
where the original was:
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
mpeg2source(quot;D:\original.d2vquot;)
crop(4,72,536,432)
LanczosResize(512,288)
but this trick seems not to work, since values were all too large, and PNSR was below 10 (while the movie looks good)...
I'll let you know the results.
But, do you think the way I'm trying now is correct?
Regards
Movie Maniac®
P.S. this instruction quot;show_graph=truequot; should show a graph... but I can't see any.. 
P.P.S. @Moderator
since it seems we're getting OT, feel free to move this post wherever you think it should be.
Originally posted by Movie Maniac®
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1=ConvertToYUY2(AviSource(quot;D:\encoded.aviquot;))
clip2=ConvertToYUY2(AviSource(quot;D:\original.avsquot;))
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;,show_graph=true)
you have to write:
clip1=AviSource(quot;D:\encoded.aviquot;).ConvertToYUY2.trim(1,0)
clip2=Import(quot;D:\original.avsquot;).ConvertToYUY2
and you dont need to load the mpeg2dec3 in this script!
Originally posted by bond
and you dont need to load the mpeg2dec3 in this script!
Sorry
I knew this, I just forgot to dlete that line.
I'll try your script.
At the moment I've got the results for the first comparison:
This is the script I used
LoadPlugin(quot;D:\MPEG2Dec3.dllquot;)
clip1=ConvertToYUY2(AviSource(quot;D:\encoded.aviquot;))
clip2=ConvertToYUY2(mpeg2source(quot;D:\original.d2vquot;).LanczosResize(512,288,4,72,536,432))
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;,show_graph=true)
and these are the results
Total frames processed: 109495
---------------------------Minimum Average Maximum
Mean Absolute Deviation: 0.0762 0.8628 2.1441
--------Mean Deviation: -0.2401 +0.0057 +0.2173
-----------------PSNR: 38.9391 46.1734 59.2167
Seems a medium quality, isn't it?
Now I've got the results of the second comparison, same clips, but different script:
clip1=AviSource(quot;D:\encoded.aviquot;).ConvertToYUY2.trim(1,0)
clip2=Import(quot;D:\original.avsquot;).ConvertToYUY2
Compare(clip1, clip2, quot;YUVquot;,quot;compare.logquot;,show_graph=true)
Total frames processed: 109495
---------------------------Minimum Average Maximum
Mean Absolute Deviation: 0.0756 3.2548 72.3725
--------Mean Deviation: -52.2118 +0.0061 +68.1686
------------------PSNR: 8.2079 35.2652 59.2488
there's a lot of difference between the two results. So what's the matter? And why use that trim(1,0) function?
Thanks for you rpatience
regards
P.S. can't understand why text formatting ain't preserved.
I highly recommend to create a lossless avi first (in huffyuv maybe?), then encode your xvid from this lossless file, then compare using this file.
This way, you're sure that you have the same file, including colorspace conversions.
And a small trick: comparsion can be done at higher speed if you have a fast cpu. Add .assumefps(50) after avisource(), then load the script to mplayer2. You won't see much, but the logfile will be there
Radek
Originally posted by sysKin
I highly recommend to create a lossless avi first (in huffyuv maybe?), then encode your xvid from this lossless file, then compare using this file.
This way, you're sure that you have the same file, including colorspace conversions.
I wasn't searching the absolute quality. I just wanted to compare two different encodings of the same movie, to decide wich one is better, since by my eyes the difference is not clear. Anyhow I'll try this way in the future
And a small trick: comparsion can be done at higher speed if you have a fast cpu. Add .assumefps(50) after avisource(), then load the script to mplayer2. You won't see much, but the logfile will be there
Radek
This is a pretty good idea, thanks for your suggestion
regards
Originally posted by Movie Maniac®
---------------------------Minimum Average Maximum
Mean Absolute Deviation: 0.0762 0.8628 2.1441
--------Mean Deviation: -0.2401 +0.0057 +0.2173
-----------------PSNR: 38.9391 46.1734 59.2167
Seems a medium quality, isn't it?
No, I believe those are very good results. But quot;goodquot; also depends on how you managed to get down the 1st pass size: there is no miracle in having a 1st pass of 700MB, for a target size of 690, and having excellent quality.
---------------------------Minimum Average Maximum
Mean Absolute Deviation: 0.0756 3.2548 72.3725
--------Mean Deviation: -52.2118 +0.0061 +68.1686
------------------PSNR: 8.2079 35.2652 59.2488
Those figures look busted. I think the same are identical, ie one is offseted by 1 frame compared to the other. That would explain why you still get a high maximum (black compared against black), an low but still OK (I think encoding at around quant 31 can get you down to 24-28dB) PSNR (most often, consecutive frames are very similar) and a very low PSNR (scene change).
there's a lot of difference between the two results. So what's the matter? And why use that trim(1,0) function?
See above for an explaination.
trim(1,0) cuts from frame 1 till end (as 0 isn't a valid length of frames). But the index starts at 1, therefore the one frame offset comes from here.
Besides, and although I guess it is now better handled, always try to us functions, ie not clip.ConvertToYUY2 but clip.ConvertToYUY2()
At some time with avisynth 1.0x, that could introduce a noticable slowdown. It may still do.
One last suggestion:
Originally posted by Syskin
And a small trick: comparsion can be done at higher speed if you have a fast cpu. Add .assumefps(50) after avisource(), then load the script to mplayer2.
And what about setting vdub_mod to direct stream copy, disable displays (hit F9 and F10) and then do a test (ctrl+F5). Then, the test will go as fast as it can go (provided you set priority high enough to make it take most of the CPU time, ie normal/higher)
Originally posted by Movie Maniac®
P.S. can't understand why text formatting ain't preserved.
This is html formatting, it eats spaces and tabs, so you should use the control codes [code] or [quote] (maybe I'm missing a better suited one), or (more bothersome) the code # to insert monospaces.
Originally posted by Kurosu
No, I believe those are very good results. But quot;goodquot; also depends on how you managed to get down the 1st pass size: there is no miracle in having a 1st pass of 700MB, for a target size of 690, and having excellent quality.
Those figures look busted. I think the same are identical, ie one is offseted by 1 frame compared to the other. That would explain why you still get a high maximum (black compared against black), an low but still OK (I think encoding at around quant 31 can get you down to 24-28dB) PSNR (most often, consecutive frames are very similar) and a very low PSNR (scene change).
You were right. I deleted that trim... and the risults were as good as the other.
So I've learned something new. Thanks to everyone who answered my question and managed to let things become clearer.
P.S. I apologize for my very bad english I'm not mother tongue!
@Movie Maniac®
the trim(1,0) trick was introduced by MF to get arround the Decoder Lag in XviD when using b-frames, so if you use b-frames you should use the trim(1,0) trick or as Kurosu said you would start with a wrong frame and the endresult would be meaningles.
Originally posted by CruNcher
@Movie Maniac®
the trim(1,0) trick was introduced by MF to get arround the Decoder Lag in XviD when using b-frames, so if you use b-frames you should use the trim(1,0) trick or as Kurosu said you would start with a wrong frame and the endresult would be meaningles. That's a good Tip.
Thanks a lot mate
Regards
Movie Maniac® |
Post at 2011-4-15 17:21
| This Author's Posts Only
