Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 2-pass encoding vs. constant quality (Read 25669 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

2-pass encoding vs. constant quality

Reply #25 –
You don't understand what I am trying to say. No analyzers are needed.

Quote
try 1pass quants 3-4 and try 2pass quants 3-4.....(now look for differences)

No! I mean that the algorithm decides on the span (2-3, 3-4, 4-5, etc.) Nothing is fixed by the user.

Quote
the first premise(hit the target bitrate) says that "two quantizers used" is a silly concept to say the least......just too narrow span!

Maybe it is not the optimal concept, but it is definitely possible. Find the greatest n, so that a constant quantizer n encoding is still larger than the specified target size. Now you can do an encoding with quantizers varying between only n and n+1 which reaches the target.

(That does not mean that Xvid's algorithm is implemented this way.)

2-pass encoding vs. constant quality

Reply #26 –
Quote
Maybe it is not the optimal concept, but it is definitely possible.

It certainly could be done, but why should it be done? Why should the codec limit itself to only two quantizer values? Am I missing something here?

Quote
(That does not mean that Xvid's algorithm is implemented this way.)


Nope. With my 2-pass Xvid encodes the quantizer values usually span the entire range of 2-10, with most hits on the values 2-4. I could limit the quantizer range, but I can't see any reason why I should do it.

2-pass encoding vs. constant quality

Reply #27 –
It should be clear that we only take p-frame quants into account? Because b-frames should be 1.5x the p-frame quant by definition. And I-frames are boosted to be lower than p-frame quant by most people. Also some 'small' quants may come out as quant=2 even if the rest is more highly compressed; this is wanted and expected in special cases, as I wrote above. (I certainly hope nobody is counting the quants used for the ending credits of the movie...  )

2-pass encoding vs. constant quality

Reply #28 –
continuum,this holds no water...(i said that tg's words will confuse!)

Quote
No! I mean that the algorithm decides on the span (2-3, 3-4, 4-5, etc.) Nothing is fixed by the user.

so you're saying that we give codec a 2-31 span and then let it decide what 2quants to pick?
is that it?
what if i'm doing moto-race?
and there are some still scenes?
whole concept goes down the toilet!

this is not implemented in that way,neither will it ever be....(even in theory it sounds non-realistic)

Quote
Now you can do an encoding with quantizers varying between only n and n+1 which reaches the target.

i told you to try "n+1" qunt span(ie.2 quants) so you can see for yourself that you can't reach the target this way....
it's either too small(too high overall quants) or too big(too low overall quants)....
too small means looks like crap,and too big means  "dad...i want dvd burner..." 
i hate big quants,and my dad doesn't have the money...same as i don't...hehe
(not untill this cdrw burner is working anyway...  )

i really think tg shouldn't of mentioned "almost" const.quantizer,as after that you got this weird ideas(  )
and he didn't really mean it that way anyhow..he didn't ment it literally....(as you saw by his later response)
and also as he mentions,when you include b-frames it all just collapses...
video compression simply doesn't work in that way,and i hope it never will(  )
no codec will be picking 2quants on my stuff,let me tell you...

but all this stuff is reminding me about huffyuv;we should go otherway around;smaller filesizes and better quality,instead of thinking about these ultra high bitrate situations....(as i hope you wouldn't do 4-5...for sure you wouldn't let codec pick those two?  )
although in that other thread skywalker just said he doesn't see a difference between quant 2 and 5...(  ....  )


Quote
Why should the codec limit itself to only two quantizer values?

ask continuum...(  )....
but ok.one could say this;if mpeg4 was so effective that quant2 has the bitrate of quant31 we would all be doing quant2 encodings only...(  )


Quote
Nope. With my 2-pass Xvid encodes the quantizer values usually span the entire range of 2-10, with most hits on the values 2-4.

as expected and normal thing codecs do...

2-pass encoding vs. constant quality

Reply #29 –
Quote
(i said that tg's words will confuse!) [...] after that you got this weird ideas

Look at the first post in this thread, it was actually the question.   

Quote
what if i'm doing moto-race?
and there are some still scenes?
whole concept goes down the toilet!

Why? The same quantizer will use more bits on the complex race scenes and less on the easy still scenes. Which (I assume) is the intended behaviour.

Quote
i told you to try "n+1" qunt span(ie.2 quants) so you can see for yourself that you can't reach the target this way....

You CAN ALWAYS reach the target with only two quantizer values!

The whole argument boils down to this: is the quantizer correlated to the perceived quality?

I don't know the internals of a video codec; I suppose the quantizer is a threshold for the perceptual entropy compression, and thus maybe the best guess of the algorithm, how the final picture will look like. But maybe the situation is completely different (as your analyzed distributions indicate), and that's why I asked the question in the first place: How does the codec decide on the used quantizer (based on the wish of constant quality)?
[span style='font-size:8pt;line-height:100%']
edit: spelling[/span]

2-pass encoding vs. constant quality

Reply #30 –
Quote
Quote
i told you to try "n+1" qunt span(ie.2 quants) so you can see for yourself that you can't reach the target this way....

You CAN ALWAYS reach the target with only two quantizer values!

LOL! This smilie says it very well!

Really, why is it so hard to understand for i4004 that a codec can reach its target filesize by switching between two quantizers? DivX5 for example goes to extremes in order to try and do this: The 'n' passes are for finding a quantizer distribution with more constant quality (less quantizer fluctuation) that still reaches the target filesize. This 'n' passes-thing was all the (marketing) rage for some short time - I mean, could it have gotten past anyone how it works?

Let me try again to explain it: In dumb CBR mode you have a fixed span of time or frames for which the codec tries to keep the target bitrate; because the codec cannot know how big the next upcoming frame is, it'll have to guess a quantizer. If the frame comes out too big, it'll have to compensate by using a higher quantizer on some other frame. Let's say the codec used quant=2 on a frame too big, and now it has to use several times frame=4 to compensate for that. But that's CBR. In 2-pass it is possible to calculate which (theoretical) average quantizer would have to be used in order to reach the target. It is well possible, to stay with that example, to reach the same filesize that CBR reached by using quants 2-4-4-4 by using quants 3-4-3-4.  What the actual size of a compressed frame is can't be foretold, so there still will be fluctuation necessary in order to compensate, unless you do n-passes.

As there are no odd frame quantizers and the adaptive quantization of individual macroblocks within a frame can't be forced by two-pass, an alternation between at least two adjacent (anything but adjacent wouldn't make sense) quantizers must be used in order to reach that average quantizer. Nobody has claimed that XviD does this strictly or perfectly; but on the other hand I don't think fluctuation is important enough to justifiy doing several passes.

It is really not that hard to understand! Just use an odd quantizer in XviD's single pass mode, it's the same.

Hope that one broke through the wall.

Edited out: some harshness

2-pass encoding vs. constant quality

Reply #31 –
i reverted to page1 and i can only see that you started to ttalk about 2 quants after i ressurected the thread,ie. after tgdeck mentioned "almost constant" thingie....
you asked about cq. versus 2pass....

and you asked;
Quote
I read somewhere that the target of 2-pass encoding is to use a quantizer as constant as possible, is that true?

nope..the target of 2pass is to hit the target bitrate by varying the quants...so quant. as constant as possible is not used in 2pass modes (or ABR 1pass)
cannot be used as that breakes filesize expectancy....

Quote
Why? The same quantizer will use more bits on the complex race scenes and less on the easy still scenes. Which (I assume) is the intended behaviour.


balance is lost;still scenes will look like crap on higher quants,and we can't really use quants that low as then filesize rockets....

Quote
You CAN ALWAYS reach the target with only two quantizer values!

ahhh..yes,i see....you're saying that target filesize is reachable if we pick the right quants............fair enough,your thoughts,but i'm saying that it doesn't really work!
because of your next sentence;
Quote
The whole argument boils down to this: is the quantizer correlated to the perceived quality?

off course it is;quantizer is THE most important thing for the quality of the frame...
so 2quants are as i already said;either lookin' crappy or overboard with filesizes...
neither is really acceptable......

Quote
How does the codec decide on the used quantizer (based on the wish of constant quality)?

no..it decides based on the bitrate wish...codec is not clever at all!

if you're interested in quantizer explanation,i have one;
god damn,where are the links when you need'em....
http://virtualdub.everwicked.com/index.php...44&hl=quantizer
(i better bookmark this one)

Quote
Really, why is it so hard to understand for i4004 that a codec can reach its target filesize by switching between two quantizers?

no it's not,but i want a solid proof;drfanalyzed file report....
1 or 2 pass encoding with 2 quants that reaches desired filesize.....shouldn't be too hard for 2pass guys,right?but make the footage demanding....

Quote
Let me try again to explain it: In dumb CBR mode you have a fixed span of time or frames for which the codec tries to keep the target bitrate;

offcourse!
for 1pass ABR(as i said no need to call it CBR) we wan't this buffer as big as possible...indeed...

Quote
It is well possible, to stay with that example, to reach the same filesize that CBR reached by using quants 2-4-4-4 by using quants 3-4-3-4.

yes...but i already said that i like still scenes on low quants;but you explained 2pass method well;it will tend to use overall higher quants.....and i said i hate that.... 

Quote
It is really not that hard to understand!

when you produce drfanalyzed report that says xvid(or other mpeg4) did 2quant encoding in 2pass mode and hit the target filesize, i'll believe you;untill then i won't.....
i hope that's fair
(offcurse invitation is opened to anyone reading this.....)

but how will you do that,beats me...(as codec desn't have the slightest clue what quants to use.....ie you would have to limit them manually......in fact i would go so far to say that codec only knows that increasing the quant makes the bitrate go down....all the tweaking of the codec settings we do,proves it;codec are a dumb things!
i only said that 1pass with limited quants can be made indistuinguishable from 2pass encoding...if you push the right buttons....i tried to show it on that 1pass versus 2pass drfanalyzed files...doing a 3-9 span for 1pass would bring 1pass even closer to 2pass encoding....but as i said,i'm not doing 3-x (as is usual 2pass practice) as i wan't a "picasso"(heh) on still scenes.....i compared still scenes on my 1pass versus still scenes on 2pass...i liked 1pass stuff better....exactly as undercut said;no bits are wasted....it's just where they are spent.....i like to use 2 as much as i can...and motion can be slightly worse(but as i said,not too bad;untill quants 6 till 9))

Quote
Hope that one broke through the wall.

i posted a solid proof for the things i'm saying.....now you post a solid proof on your concepts.....i didin't saw it yet,and your words alone won't do....

you explained the 1pass versus 2pass mode well,but i believe you are mistaking if you say that target bitrate will be reached by 2 only quants....(as you clearly agreed with continuum when he said it!)
we all know that it would take many test encoding to pick the right 2 quants,and that's not it!that's not automated process.....not useable...not appropriate to do 10 test encodings to see what 2 quants to use.....

as i usually say;encode more,talk less... 
and encode more to reach better quality.....we neeed not theorize this much when we have actuall codecs to test it with...

why won't i do this tests for you?
1-i have "old man and the sea" to cap now
2-my hdd is full again,so more job for nandub...
3-2pass 2quants won't work/doesn't work... 

i'm leaving this thread here....i believe i repeated my stuff few times too many.....
if you post a solid proof that you can do the stuff you claimed are possible,i'll apologize and accept it....no problems...

cheers

/ivo

edit;"cannot be used as that breakes filesize expectancy....

2-pass encoding vs. constant quality

Reply #32 –
Quote
balance is lost;still scenes will look like crap on higher quants,and we can't really use quants that low as then filesize rockets....

Not true.
1) Just because a frame is small this doesn't mean it will look worse than a big frame at the same quant. The thing is, it is not necessary to quantize it highly. If you switch off b-frames (not recommended) encode at >70% compressiblity and don't use a boost, you could have such a 2nd pass where everything stays within the range of two quants. But this is not desirable. Having most p-frames not jump too much is.

I have said that  before, I get the feeling you don't read thoroughly.

2) All the time we've been talking about encoding at sane quantizers like 2-3 or 3-4. Small frames don't look like crap at these quants and nothing rockets nowhere.

If you'd use quantizers like 8-10, having small frames compressed at quant=2 wouldn't save such an encode, either...

Quote
as i usually say;encode more,talk less...
and encode more to reach better quality.....we neeed not theorize this much when we have actuall codecs to test it with...

From my experience with encoding and encoding-related forums - which date back quite some years (haven't seen you around back in Nando's days discussing DRF-limitations - which Nando agreed were a very bad idea) - I'm inclined to say that  many mass-encoders don't think much about their outdated methods and produce a lot of low-quality output while interested amateurs progress nicely with newer software using default-settings.

I've always found the fun in encoding lies in experimenting and re-encoding again and again to test new features. Usually a movie stays on my HD for weeks, as do several encoded versions of it, done with different custom-matrices etc. till I eventually decide to keep one. Dear colleague, there is a big difference between simply talking too much (often decorated with lots of smilies and emotion) and the development of a consistent stream of thought.

Edited out: some harshness
Edited: made it even simpler and easier to understand

2-pass encoding vs. constant quality

Reply #33 –
just to add a further explanation on my thoughts mentioned here
http://www.hydrogenaudio.org/forums/index....ndpost&p=190501
("artificial cbr in audio too" issue)

here's a thread on vdub forum that i remembered while i wrote the above linked stuff....so i went ahead and corrected the stuff with vwpaint's thoughts...here
http://virtualdub.everwicked.com/index.php...=45&#entry24837
(it's enough to read last 2posts on vdub....)

2-pass encoding vs. constant quality

Reply #34 –
The quote from Very WetPaint is about MP3-encoding. You should know that video is quite something else; as the first sentence from that quote already implies:
Quote
By their nature, audio compression schemes tend more toward CBR than image compression.


It's an unrelated quote, dear colleague. I seem to recall that kind of unrelated quotes and links that don't contribute anything to the clarification of the subject from previous discussions.

Edit: To post something actually related, here are some quantizer-statistics from encoding the Matrix Reloaded trailer; without b-frames (in order not to mess the statistics up due to the b-quant ratio and offset) also without adaptive quantization. Results are as expected, deviation within 1 quantizer at 70% of the 1st pass' size, within 2 quants at 50%:

no b-frames, no aq
50% of 1st pass (54,7 MB; deviation: -818 kb):
quant2=16.4%
quant3=53.2%
quant4=29.5%

no b-frames, no aq
70% of 1st pass (77,4 MB; deviation: -243 kb):
quant2=54,9%
quant3=45,1%

The quant-distribution is quite the same for i- and p-frames. XviD 1.0 RC3. VHQ=4, GMC and trellis were used; 10% i-frame boost and the default 5% for overflow treatment. Interpretation: The 2-pass code seems to work nicely in reaching a rather conservative quant-distribution.

Maybe I'll add stats with b-frames and aq later, but as DRF-analyzer doesn't distinguish between frame-types, I'd have to use the XviD-status window which in turn doesn't output percentages per quant per frame-type. So I'd have to guess percentages which isn't quite a scientific method. 

---------------

Edit: In spite of my doubts mentioned above, I've encoded now with b-frames and aq  to about the same filesize (slight deviation in such short samples makes for a lot of difference) as the first video without b-frames. Aim was to see whether any more constant distribution of p-frame quantizers could be traded for using b-frames and aq. The result seems to support this notion only partly:

b-frames, AQ
55,2 MB (deviation: -298 kb):
i-frames 2-3 (average 2.40)
p-frames: 2-4 (average 2.79, about 80% are quant=3, about 5% are quant=4)
b-frames: 4-7 (average 4.66, less than 5% are quant=6 or 7)

Interpretation: We see that p-frame quants now tend towards a solid '3' and only deviate one quant up or down in a 20% of all cases. [edit]This must be taken with a grain of salt, though, as AQ means in a good portion of those frames some macroblocks have a higher quantizer.[/edit] So although the result seems unimpressive at first, the reduction of quant=4 p-frames from 30% to 5% should not be valued lowly. It must also be remembered that the DRF-analyzer counted i- and p-frame quants in the non-b-frame encodes as one and the same.

Counting i- and p-frames together the same way for the b-frame-enabled encode for one thing would mean a further slight shift towards the lower quants for this latter encode. Furthermore, in the non-b-frame encode i-frame quant-distribution followed p-frame quant-distribution, meaning that we also had many quant=4 i-frames. There are no quant=4 i-frames in this last encode.

Edit: To come back full circle to the initial question/topic ("Are there advantages of two-pass over single-pass?"): When you encode small clips like the one above, there is no real reason to use two-pass, single pass will do fine. But in most cases you have some filesize limitations (a CD-R, a DVD-R...) for the video you work with and hence you should use XviD's two-pass like it was meant to be (default two-pass settings, no quantizer-restrictions) because it quite accurately hits the target filesize and reaches about the best quality possible for that filesize. This means, an even quality is reached by a quantizer distribution that comes very close to using a fixed quantizer in single-pass. XviD's two-pass algo has been coded by someone who really knew what he did.

2-pass encoding vs. constant quality

Reply #35 –
my 2 cents...
Quote
Average Frame quality is  MEDIUM [Average DRF/quantizer is 3.80]
Standard Deviation: Quality is MEDIUM [Std. Deviation is 1.17]


standard deviation 1.17... that means 95% of all frames are within 2.34 quants of the average (remember stats?)

out of 31 possible values i'd say that's "more or less" constant  it's a pretty steep bell-curve at any rate.

2-pass encoding vs. constant quality

Reply #36 –
too bad i dont quite follow this discussion that goes a bit too far technically for me